[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:20:26.623869  1517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.123.126:32847
I20260709 12:20:26.625911  1517 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:20:26.627167  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:26.644806  1523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:26.647317  1526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:26.648545  1517 server_base.cc:1061] running on GCE node
W20260709 12:20:26.645023  1524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:27.863404  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:27.863749  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:27.864022  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599627863994 us; error 0 us; skew 500 ppm
I20260709 12:20:27.865021  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:27.872777  1517 webserver.cc:533] Webserver started at http://127.1.123.126:43775/ using document root <none> and password file <none>
I20260709 12:20:27.873873  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:27.874141  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:27.874671  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:27.881323  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/master-0-root/instance:
uuid: "6190226b862b4c9abdc05e358228ec56"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-08l4"
I20260709 12:20:27.890483  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:20:27.896497  1532 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.897976  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:20:27.898402  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/master-0-root
uuid: "6190226b862b4c9abdc05e358228ec56"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-08l4"
I20260709 12:20:27.898957  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:27.965314  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:27.967254  1517 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:20:27.967823  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:28.054184  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.126:32847
I20260709 12:20:28.054256  1583 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.126:32847 every 8 connection(s)
I20260709 12:20:28.063710  1584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.088071  1584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56: Bootstrap starting.
I20260709 12:20:28.095330  1584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.097503  1584 log.cc:826] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56: Log is configured to *not* fsync() on all Append() calls
I20260709 12:20:28.103286  1584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56: No bootstrap required, opened a new log
I20260709 12:20:28.126340  1584 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6190226b862b4c9abdc05e358228ec56" member_type: VOTER }
I20260709 12:20:28.127027  1584 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:28.127231  1584 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6190226b862b4c9abdc05e358228ec56, State: Initialized, Role: FOLLOWER
I20260709 12:20:28.128228  1584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6190226b862b4c9abdc05e358228ec56" member_type: VOTER }
I20260709 12:20:28.128775  1584 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:20:28.129034  1584 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:20:28.129369  1584 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:28.134310  1584 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6190226b862b4c9abdc05e358228ec56" member_type: VOTER }
I20260709 12:20:28.135192  1584 leader_election.cc:304] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6190226b862b4c9abdc05e358228ec56; no voters: 
I20260709 12:20:28.137377  1584 leader_election.cc:290] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:20:28.137883  1587 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:28.141314  1587 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [term 1 LEADER]: Becoming Leader. State: Replica: 6190226b862b4c9abdc05e358228ec56, State: Running, Role: LEADER
I20260709 12:20:28.142447  1587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6190226b862b4c9abdc05e358228ec56" member_type: VOTER }
I20260709 12:20:28.143700  1584 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:20:28.155943  1589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6190226b862b4c9abdc05e358228ec56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6190226b862b4c9abdc05e358228ec56" member_type: VOTER } }
I20260709 12:20:28.157068  1589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:28.159071  1588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6190226b862b4c9abdc05e358228ec56. Latest consensus state: current_term: 1 leader_uuid: "6190226b862b4c9abdc05e358228ec56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6190226b862b4c9abdc05e358228ec56" member_type: VOTER } }
I20260709 12:20:28.160820  1588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:28.163444  1597 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:20:28.175678  1597 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:20:28.178503  1517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:20:28.196498  1597 catalog_manager.cc:1383] Generated new cluster ID: a7bd9dcb240541e4b8d99c146526d400
I20260709 12:20:28.196899  1597 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:20:28.226769  1597 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:20:28.229099  1597 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:20:28.244263  1597 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56: Generated new TSK 0
I20260709 12:20:28.245832  1597 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:20:28.310307  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:28.320166  1606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:28.321316  1607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:28.326601  1517 server_base.cc:1061] running on GCE node
W20260709 12:20:28.327108  1609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:28.328238  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:28.328547  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:28.328758  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599628328741 us; error 0 us; skew 500 ppm
I20260709 12:20:28.329650  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:28.334177  1517 webserver.cc:533] Webserver started at http://127.1.123.65:44525/ using document root <none> and password file <none>
I20260709 12:20:28.334930  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:28.335188  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:28.335564  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:28.337208  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-0-root/instance:
uuid: "b074c91c6ea54bea8aae038429fc5a38"
format_stamp: "Formatted at 2026-07-09 12:20:28 on dist-test-slave-08l4"
I20260709 12:20:28.344693  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:20:28.350469  1614 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:28.351464  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:20:28.351832  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-0-root
uuid: "b074c91c6ea54bea8aae038429fc5a38"
format_stamp: "Formatted at 2026-07-09 12:20:28 on dist-test-slave-08l4"
I20260709 12:20:28.352238  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:28.388582  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:28.390620  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:28.393831  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:28.399539  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:28.399887  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:28.400231  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:28.400476  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:28.506174  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.65:46227
I20260709 12:20:28.506235  1676 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.65:46227 every 8 connection(s)
I20260709 12:20:28.515022  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:28.528486  1681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:28.537732  1682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:28.548331  1517 server_base.cc:1061] running on GCE node
I20260709 12:20:28.550839  1677 heartbeater.cc:344] Connected to a master server at 127.1.123.126:32847
I20260709 12:20:28.551476  1677 heartbeater.cc:461] Registering TS with master...
W20260709 12:20:28.551971  1684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:28.552938  1677 heartbeater.cc:507] Master 127.1.123.126:32847 requested a full tablet report, sending...
I20260709 12:20:28.553332  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:28.553647  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:28.553954  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599628553927 us; error 0 us; skew 500 ppm
I20260709 12:20:28.554826  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:28.557229  1549 ts_manager.cc:194] Registered new tserver with Master: b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227)
I20260709 12:20:28.558687  1517 webserver.cc:533] Webserver started at http://127.1.123.66:43211/ using document root <none> and password file <none>
I20260709 12:20:28.559535  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:28.559830  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:28.560204  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:28.561193  1549 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50212
I20260709 12:20:28.562040  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-1-root/instance:
uuid: "265a51ff1cc7461e82200790ac4907c3"
format_stamp: "Formatted at 2026-07-09 12:20:28 on dist-test-slave-08l4"
I20260709 12:20:28.569402  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:28.574940  1689 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:28.575932  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:20:28.576333  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-1-root
uuid: "265a51ff1cc7461e82200790ac4907c3"
format_stamp: "Formatted at 2026-07-09 12:20:28 on dist-test-slave-08l4"
I20260709 12:20:28.576706  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:28.613369  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:28.615006  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:28.616578  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:28.619851  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:28.620095  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:28.620366  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:28.620568  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:28.729876  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.66:38587
I20260709 12:20:28.730014  1751 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.66:38587 every 8 connection(s)
I20260709 12:20:28.738276  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:28.747336  1755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:28.750223  1756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:28.758739  1752 heartbeater.cc:344] Connected to a master server at 127.1.123.126:32847
I20260709 12:20:28.759212  1752 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:28.760169  1752 heartbeater.cc:507] Master 127.1.123.126:32847 requested a full tablet report, sending...
I20260709 12:20:28.761899  1517 server_base.cc:1061] running on GCE node
I20260709 12:20:28.762897  1549 ts_manager.cc:194] Registered new tserver with Master: 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
W20260709 12:20:28.763700  1758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:28.764763  1549 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56450
I20260709 12:20:28.764952  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:28.765204  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:28.765406  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599628765386 us; error 0 us; skew 500 ppm
I20260709 12:20:28.766249  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:28.769214  1517 webserver.cc:533] Webserver started at http://127.1.123.67:37289/ using document root <none> and password file <none>
I20260709 12:20:28.769822  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:28.770049  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:28.770340  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:28.771509  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-2-root/instance:
uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
format_stamp: "Formatted at 2026-07-09 12:20:28 on dist-test-slave-08l4"
I20260709 12:20:28.776757  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 12:20:28.781014  1763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:28.781960  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:20:28.782302  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-2-root
uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
format_stamp: "Formatted at 2026-07-09 12:20:28 on dist-test-slave-08l4"
I20260709 12:20:28.782632  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599626433458-1517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:28.803879  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:28.805326  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:28.807755  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:28.811156  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:28.811410  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:28.811689  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:28.811889  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:28.920018  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.67:40083
I20260709 12:20:28.920085  1825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.67:40083 every 8 connection(s)
I20260709 12:20:28.938194  1826 heartbeater.cc:344] Connected to a master server at 127.1.123.126:32847
I20260709 12:20:28.938642  1826 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:28.939630  1826 heartbeater.cc:507] Master 127.1.123.126:32847 requested a full tablet report, sending...
I20260709 12:20:28.941996  1549 ts_manager.cc:194] Registered new tserver with Master: d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:28.942332  1517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016758599s
I20260709 12:20:28.943538  1549 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56464
I20260709 12:20:29.007375  1549 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56476:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:20:29.011633  1549 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:20:29.098151  1791 tablet_service.cc:1511] Processing CreateTablet for tablet 012b99b449bc4f02a31b9c18fa87475a (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:20:29.100193  1791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012b99b449bc4f02a31b9c18fa87475a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.111043  1790 tablet_service.cc:1511] Processing CreateTablet for tablet b0ededca7aa44f16b1e09004b3c06605 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:20:29.112872  1790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0ededca7aa44f16b1e09004b3c06605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.118458  1642 tablet_service.cc:1511] Processing CreateTablet for tablet 012b99b449bc4f02a31b9c18fa87475a (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:20:29.118839  1641 tablet_service.cc:1511] Processing CreateTablet for tablet b0ededca7aa44f16b1e09004b3c06605 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:20:29.120137  1642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012b99b449bc4f02a31b9c18fa87475a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.121184  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0ededca7aa44f16b1e09004b3c06605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.122372  1640 tablet_service.cc:1511] Processing CreateTablet for tablet c9c09f11fdd44984937a807c10c5d5d3 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:20:29.123924  1640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9c09f11fdd44984937a807c10c5d5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.146050  1715 tablet_service.cc:1511] Processing CreateTablet for tablet c9c09f11fdd44984937a807c10c5d5d3 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:20:29.149266  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9c09f11fdd44984937a807c10c5d5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.145607  1717 tablet_service.cc:1511] Processing CreateTablet for tablet 012b99b449bc4f02a31b9c18fa87475a (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:20:29.141837  1716 tablet_service.cc:1511] Processing CreateTablet for tablet b0ededca7aa44f16b1e09004b3c06605 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:20:29.143864  1639 tablet_service.cc:1511] Processing CreateTablet for tablet a3cba7d3b0e14ea68eba21b6298dbfa7 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:20:29.145196  1789 tablet_service.cc:1511] Processing CreateTablet for tablet c9c09f11fdd44984937a807c10c5d5d3 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:20:29.155488  1789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9c09f11fdd44984937a807c10c5d5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.158825  1717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012b99b449bc4f02a31b9c18fa87475a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.158767  1788 tablet_service.cc:1511] Processing CreateTablet for tablet a3cba7d3b0e14ea68eba21b6298dbfa7 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:20:29.153609  1714 tablet_service.cc:1511] Processing CreateTablet for tablet a3cba7d3b0e14ea68eba21b6298dbfa7 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:20:29.166417  1716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0ededca7aa44f16b1e09004b3c06605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.168924  1639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3cba7d3b0e14ea68eba21b6298dbfa7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:29.169021  1596 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:29.170996  1714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3cba7d3b0e14ea68eba21b6298dbfa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.168291  1788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3cba7d3b0e14ea68eba21b6298dbfa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.168296  1638 tablet_service.cc:1511] Processing CreateTablet for tablet 7a99c2cf2385417a86196fc38ccfbab0 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:20:29.179796  1638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a99c2cf2385417a86196fc38ccfbab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.178264  1713 tablet_service.cc:1511] Processing CreateTablet for tablet 7a99c2cf2385417a86196fc38ccfbab0 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:20:29.184494  1787 tablet_service.cc:1511] Processing CreateTablet for tablet 7a99c2cf2385417a86196fc38ccfbab0 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:20:29.185431  1713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a99c2cf2385417a86196fc38ccfbab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.187815  1787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a99c2cf2385417a86196fc38ccfbab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.199043  1637 tablet_service.cc:1511] Processing CreateTablet for tablet 8d944dcaa2834f528cbe940a7cf041c9 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:20:29.202231  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d944dcaa2834f528cbe940a7cf041c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.213320  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 8d944dcaa2834f528cbe940a7cf041c9 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:20:29.216792  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d944dcaa2834f528cbe940a7cf041c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.216560  1786 tablet_service.cc:1511] Processing CreateTablet for tablet 8d944dcaa2834f528cbe940a7cf041c9 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:20:29.234287  1786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d944dcaa2834f528cbe940a7cf041c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.276986  1844 tablet_bootstrap.cc:492] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c: Bootstrap starting.
I20260709 12:20:29.281615  1636 tablet_service.cc:1511] Processing CreateTablet for tablet c31f514965854ca89d1b161aa8e5c802 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:20:29.283388  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c31f514965854ca89d1b161aa8e5c802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.298799  1845 tablet_bootstrap.cc:492] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38: Bootstrap starting.
I20260709 12:20:29.299875  1711 tablet_service.cc:1511] Processing CreateTablet for tablet c31f514965854ca89d1b161aa8e5c802 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:20:29.301493  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c31f514965854ca89d1b161aa8e5c802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.304878  1642 tablet_service.cc:1511] Processing CreateTablet for tablet 1842932f5ba24f0ca6c62e1acfeb017e (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:20:29.306556  1642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1842932f5ba24f0ca6c62e1acfeb017e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.303500  1790 tablet_service.cc:1511] Processing CreateTablet for tablet c31f514965854ca89d1b161aa8e5c802 (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:20:29.310480  1790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c31f514965854ca89d1b161aa8e5c802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.338544  1710 tablet_service.cc:1511] Processing CreateTablet for tablet 1842932f5ba24f0ca6c62e1acfeb017e (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:20:29.349189  1710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1842932f5ba24f0ca6c62e1acfeb017e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.368896  1791 tablet_service.cc:1511] Processing CreateTablet for tablet 1842932f5ba24f0ca6c62e1acfeb017e (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:20:29.370561  1791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1842932f5ba24f0ca6c62e1acfeb017e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.376861  1844 tablet_bootstrap.cc:654] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.386240  1845 tablet_bootstrap.cc:654] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.396729  1714 tablet_service.cc:1511] Processing CreateTablet for tablet 677bc6e8dade496dad5352691517e43f (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:20:29.398197  1714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 677bc6e8dade496dad5352691517e43f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.413862  1637 tablet_service.cc:1511] Processing CreateTablet for tablet 677bc6e8dade496dad5352691517e43f (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:20:29.415416  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 677bc6e8dade496dad5352691517e43f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.433298  1788 tablet_service.cc:1511] Processing CreateTablet for tablet 677bc6e8dade496dad5352691517e43f (DEFAULT_TABLE table=test-workload [id=545250fc7354445fa04ed69260ac61c7]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:20:29.434968  1788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 677bc6e8dade496dad5352691517e43f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.438664  1846 tablet_bootstrap.cc:492] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3: Bootstrap starting.
I20260709 12:20:29.465369  1844 tablet_bootstrap.cc:492] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c: No bootstrap required, opened a new log
I20260709 12:20:29.466482  1844 ts_tablet_manager.cc:1403] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c: Time spent bootstrapping tablet: real 0.190s	user 0.042s	sys 0.043s
I20260709 12:20:29.466543  1845 tablet_bootstrap.cc:492] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38: No bootstrap required, opened a new log
I20260709 12:20:29.467552  1845 ts_tablet_manager.cc:1403] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38: Time spent bootstrapping tablet: real 0.169s	user 0.016s	sys 0.009s
I20260709 12:20:29.469877  1846 tablet_bootstrap.cc:654] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.477427  1844 raft_consensus.cc:359] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:29.480544  1844 raft_consensus.cc:385] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.480990  1844 raft_consensus.cc:740] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.479733  1845 raft_consensus.cc:359] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:29.483507  1845 raft_consensus.cc:385] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.483896  1845 raft_consensus.cc:740] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.481925  1844 consensus_queue.cc:260] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:29.484760  1845 consensus_queue.cc:260] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:29.489967  1846 tablet_bootstrap.cc:492] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3: No bootstrap required, opened a new log
I20260709 12:20:29.490483  1846 ts_tablet_manager.cc:1403] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3: Time spent bootstrapping tablet: real 0.069s	user 0.008s	sys 0.009s
I20260709 12:20:29.493623  1846 raft_consensus.cc:359] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.494514  1846 raft_consensus.cc:385] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.494822  1846 raft_consensus.cc:740] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.495602  1846 consensus_queue.cc:260] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.505056  1752 heartbeater.cc:499] Master 127.1.123.126:32847 was elected leader, sending a full tablet report...
I20260709 12:20:29.506026  1677 heartbeater.cc:499] Master 127.1.123.126:32847 was elected leader, sending a full tablet report...
I20260709 12:20:29.506706  1826 heartbeater.cc:499] Master 127.1.123.126:32847 was elected leader, sending a full tablet report...
I20260709 12:20:29.510267  1845 ts_tablet_manager.cc:1434] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38: Time spent starting tablet: real 0.042s	user 0.011s	sys 0.004s
I20260709 12:20:29.510833  1846 ts_tablet_manager.cc:1434] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.005s
I20260709 12:20:29.511526  1845 tablet_bootstrap.cc:492] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38: Bootstrap starting.
I20260709 12:20:29.511902  1846 tablet_bootstrap.cc:492] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3: Bootstrap starting.
W20260709 12:20:29.514815  1678 tablet.cc:2406] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:20:29.521378  1846 tablet_bootstrap.cc:654] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.522037  1845 tablet_bootstrap.cc:654] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.527236  1844 ts_tablet_manager.cc:1434] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c: Time spent starting tablet: real 0.060s	user 0.011s	sys 0.014s
I20260709 12:20:29.528650  1844 tablet_bootstrap.cc:492] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c: Bootstrap starting.
I20260709 12:20:29.539767  1845 tablet_bootstrap.cc:492] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38: No bootstrap required, opened a new log
I20260709 12:20:29.540536  1845 ts_tablet_manager.cc:1403] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.005s
I20260709 12:20:29.548208  1844 tablet_bootstrap.cc:654] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.548964  1846 tablet_bootstrap.cc:492] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3: No bootstrap required, opened a new log
I20260709 12:20:29.549553  1846 ts_tablet_manager.cc:1403] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.017s
I20260709 12:20:29.552789  1845 raft_consensus.cc:359] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.553166  1846 raft_consensus.cc:359] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:29.553930  1846 raft_consensus.cc:385] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.553745  1845 raft_consensus.cc:385] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.554312  1846 raft_consensus.cc:740] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.554484  1845 raft_consensus.cc:740] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.555356  1845 consensus_queue.cc:260] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.558277  1845 ts_tablet_manager.cc:1434] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.004s
I20260709 12:20:29.555292  1846 consensus_queue.cc:260] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:29.562479  1846 ts_tablet_manager.cc:1434] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:20:29.563799  1846 tablet_bootstrap.cc:492] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3: Bootstrap starting.
I20260709 12:20:29.559607  1845 tablet_bootstrap.cc:492] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38: Bootstrap starting.
I20260709 12:20:29.585922  1845 tablet_bootstrap.cc:654] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.583024  1846 tablet_bootstrap.cc:654] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.608371  1846 tablet_bootstrap.cc:492] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3: No bootstrap required, opened a new log
I20260709 12:20:29.608357  1844 tablet_bootstrap.cc:492] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c: No bootstrap required, opened a new log
I20260709 12:20:29.609398  1846 ts_tablet_manager.cc:1403] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.005s
I20260709 12:20:29.611490  1845 tablet_bootstrap.cc:492] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38: No bootstrap required, opened a new log
I20260709 12:20:29.612015  1845 ts_tablet_manager.cc:1403] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.016s
I20260709 12:20:29.609402  1844 ts_tablet_manager.cc:1403] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent bootstrapping tablet: real 0.081s	user 0.024s	sys 0.032s
I20260709 12:20:29.612955  1846 raft_consensus.cc:359] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.613906  1846 raft_consensus.cc:385] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.614259  1846 raft_consensus.cc:740] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.615206  1845 raft_consensus.cc:359] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.615132  1846 consensus_queue.cc:260] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.615948  1845 raft_consensus.cc:385] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.616276  1845 raft_consensus.cc:740] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.617065  1845 consensus_queue.cc:260] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.618101  1846 ts_tablet_manager.cc:1434] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:29.619120  1846 tablet_bootstrap.cc:492] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3: Bootstrap starting.
I20260709 12:20:29.619585  1845 ts_tablet_manager.cc:1434] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:29.620574  1845 tablet_bootstrap.cc:492] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38: Bootstrap starting.
I20260709 12:20:29.628197  1846 tablet_bootstrap.cc:654] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.630054  1845 tablet_bootstrap.cc:654] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.624217  1844 raft_consensus.cc:359] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.630461  1844 raft_consensus.cc:385] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.630764  1844 raft_consensus.cc:740] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.631723  1844 consensus_queue.cc:260] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.633991  1844 ts_tablet_manager.cc:1434] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:20:29.634972  1844 tablet_bootstrap.cc:492] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c: Bootstrap starting.
I20260709 12:20:29.644816  1846 tablet_bootstrap.cc:492] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3: No bootstrap required, opened a new log
I20260709 12:20:29.645462  1846 ts_tablet_manager.cc:1403] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.007s
I20260709 12:20:29.649533  1846 raft_consensus.cc:359] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.650315  1844 tablet_bootstrap.cc:654] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.650435  1846 raft_consensus.cc:385] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.650974  1846 raft_consensus.cc:740] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.651890  1846 consensus_queue.cc:260] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.656991  1846 ts_tablet_manager.cc:1434] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:20:29.661052  1845 tablet_bootstrap.cc:492] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38: No bootstrap required, opened a new log
I20260709 12:20:29.661759  1845 ts_tablet_manager.cc:1403] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.014s
I20260709 12:20:29.663395  1846 tablet_bootstrap.cc:492] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3: Bootstrap starting.
I20260709 12:20:29.665096  1845 raft_consensus.cc:359] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.665933  1845 raft_consensus.cc:385] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.666235  1845 raft_consensus.cc:740] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.667169  1845 consensus_queue.cc:260] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.668439  1844 tablet_bootstrap.cc:492] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c: No bootstrap required, opened a new log
I20260709 12:20:29.668979  1844 ts_tablet_manager.cc:1403] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.009s
I20260709 12:20:29.671183  1846 tablet_bootstrap.cc:654] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.672518  1845 ts_tablet_manager.cc:1434] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:20:29.673463  1845 tablet_bootstrap.cc:492] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38: Bootstrap starting.
I20260709 12:20:29.680634  1844 raft_consensus.cc:359] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.681586  1844 raft_consensus.cc:385] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.681944  1844 raft_consensus.cc:740] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.683228  1846 tablet_bootstrap.cc:492] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3: No bootstrap required, opened a new log
I20260709 12:20:29.682968  1844 consensus_queue.cc:260] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.683730  1846 ts_tablet_manager.cc:1403] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.001s
I20260709 12:20:29.685009  1845 tablet_bootstrap.cc:654] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.687958  1844 ts_tablet_manager.cc:1434] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:20:29.687978  1846 raft_consensus.cc:359] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.688880  1846 raft_consensus.cc:385] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.689178  1846 raft_consensus.cc:740] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.689254  1844 tablet_bootstrap.cc:492] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c: Bootstrap starting.
I20260709 12:20:29.689983  1846 consensus_queue.cc:260] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.694618  1852 raft_consensus.cc:493] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.695209  1852 raft_consensus.cc:515] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.698513  1852 leader_election.cc:290] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:29.715306  1844 tablet_bootstrap.cc:654] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.716025  1845 tablet_bootstrap.cc:492] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38: No bootstrap required, opened a new log
I20260709 12:20:29.716603  1845 ts_tablet_manager.cc:1403] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.007s
I20260709 12:20:29.720393  1845 raft_consensus.cc:359] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.721472  1845 raft_consensus.cc:385] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.726542  1845 raft_consensus.cc:740] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.727552  1845 consensus_queue.cc:260] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.732915  1844 tablet_bootstrap.cc:492] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c: No bootstrap required, opened a new log
I20260709 12:20:29.733621  1844 ts_tablet_manager.cc:1403] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.007s
I20260709 12:20:29.735191  1845 ts_tablet_manager.cc:1434] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.005s
I20260709 12:20:29.738618  1845 tablet_bootstrap.cc:492] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38: Bootstrap starting.
I20260709 12:20:29.738497  1844 raft_consensus.cc:359] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.747010  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:29.748005  1844 raft_consensus.cc:385] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.749131  1844 raft_consensus.cc:740] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.749820  1846 ts_tablet_manager.cc:1434] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3: Time spent starting tablet: real 0.066s	user 0.023s	sys 0.030s
I20260709 12:20:29.750185  1844 consensus_queue.cc:260] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.750957  1846 tablet_bootstrap.cc:492] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3: Bootstrap starting.
I20260709 12:20:29.751539  1801 raft_consensus.cc:2468] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 0.
I20260709 12:20:29.752632  1850 raft_consensus.cc:493] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.753226  1850 raft_consensus.cc:515] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:29.755870  1692 leader_election.cc:304] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:29.756295  1850 leader_election.cc:290] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:29.756934  1867 raft_consensus.cc:2804] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:29.757349  1867 raft_consensus.cc:493] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.757720  1867 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.765223  1867 raft_consensus.cc:515] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.767491  1867 leader_election.cc:290] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:29.769948  1846 tablet_bootstrap.cc:654] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.771574  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:29.772441  1652 raft_consensus.cc:2468] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 0.
I20260709 12:20:29.774073  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:29.774968  1651 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.777453  1845 tablet_bootstrap.cc:654] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.783912  1844 ts_tablet_manager.cc:1434] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.004s
I20260709 12:20:29.787840  1651 raft_consensus.cc:2468] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 1.
I20260709 12:20:29.799089  1693 leader_election.cc:304] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:29.800256  1867 raft_consensus.cc:2804] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:29.800781  1867 raft_consensus.cc:697] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:29.789265  1844 tablet_bootstrap.cc:492] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c: Bootstrap starting.
I20260709 12:20:29.801651  1867 consensus_queue.cc:237] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.808197  1846 tablet_bootstrap.cc:492] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3: No bootstrap required, opened a new log
I20260709 12:20:29.808734  1846 ts_tablet_manager.cc:1403] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.025s
I20260709 12:20:29.811756  1846 raft_consensus.cc:359] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.812741  1846 raft_consensus.cc:385] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.813155  1846 raft_consensus.cc:740] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.814254  1846 consensus_queue.cc:260] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.820315  1846 ts_tablet_manager.cc:1434] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:20:29.821494  1846 tablet_bootstrap.cc:492] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3: Bootstrap starting.
I20260709 12:20:29.827579  1850 raft_consensus.cc:493] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.828330  1850 raft_consensus.cc:515] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.829212  1846 tablet_bootstrap.cc:654] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.831182  1850 leader_election.cc:290] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:29.831034  1547 catalog_manager.cc:5697] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:29.832679  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:29.833415  1801 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.841964  1846 tablet_bootstrap.cc:492] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3: No bootstrap required, opened a new log
I20260709 12:20:29.842518  1846 ts_tablet_manager.cc:1403] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.007s
I20260709 12:20:29.847139  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:29.847941  1651 raft_consensus.cc:2468] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 0.
I20260709 12:20:29.850934  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:29.851979  1651 raft_consensus.cc:2468] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 0.
I20260709 12:20:29.853657  1867 raft_consensus.cc:493] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.854419  1867 raft_consensus.cc:515] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.858101  1867 leader_election.cc:290] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:29.858400  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
W20260709 12:20:29.860175  1693 leader_election.cc:343] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.856921  1846 raft_consensus.cc:359] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.862555  1846 raft_consensus.cc:385] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.862864  1846 raft_consensus.cc:740] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.864917  1845 tablet_bootstrap.cc:492] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38: No bootstrap required, opened a new log
I20260709 12:20:29.865636  1845 ts_tablet_manager.cc:1403] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38: Time spent bootstrapping tablet: real 0.127s	user 0.034s	sys 0.030s
I20260709 12:20:29.865265  1846 consensus_queue.cc:260] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.866219  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:29.867753  1846 ts_tablet_manager.cc:1434] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:20:29.868338  1767 leader_election.cc:304] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:29.868826  1846 tablet_bootstrap.cc:492] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3: Bootstrap starting.
I20260709 12:20:29.869845  1845 raft_consensus.cc:359] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.870791  1845 raft_consensus.cc:385] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.872529  1845 raft_consensus.cc:740] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.871608  1850 raft_consensus.cc:2804] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:29.873898  1850 raft_consensus.cc:493] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.874310  1850 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.873597  1845 consensus_queue.cc:260] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.881618  1845 ts_tablet_manager.cc:1434] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:20:29.883212  1850 raft_consensus.cc:515] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:29.888121  1845 tablet_bootstrap.cc:492] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38: Bootstrap starting.
I20260709 12:20:29.888779  1850 leader_election.cc:290] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:29.884042  1846 tablet_bootstrap.cc:654] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.871402  1801 raft_consensus.cc:2468] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 1.
I20260709 12:20:29.872048  1767 leader_election.cc:304] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
W20260709 12:20:29.890807  1692 leader_election.cc:343] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.891338  1692 leader_election.cc:304] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3; no voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:29.892764  1867 raft_consensus.cc:2749] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:29.893222  1850 raft_consensus.cc:2804] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:29.893644  1850 raft_consensus.cc:493] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.893978  1850 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.896274  1867 raft_consensus.cc:493] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.896879  1867 raft_consensus.cc:515] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.900492  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:29.901185  1867 leader_election.cc:290] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:29.901346  1651 raft_consensus.cc:2468] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 0.
I20260709 12:20:29.901474  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:29.902127  1652 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.902945  1693 leader_election.cc:304] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:29.903446  1850 raft_consensus.cc:515] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.904367  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:29.905683  1867 raft_consensus.cc:2804] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:29.885262  1844 tablet_bootstrap.cc:654] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.910275  1852 raft_consensus.cc:493] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.910434  1727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:29.910874  1852 raft_consensus.cc:515] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.911167  1726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:29.911397  1727 raft_consensus.cc:2468] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 0.
I20260709 12:20:29.911955  1726 raft_consensus.cc:2468] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 0.
I20260709 12:20:29.913215  1852 leader_election.cc:290] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:29.913026  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:29.913822  1725 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.914331  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:29.914891  1867 raft_consensus.cc:493] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.916127  1651 raft_consensus.cc:2468] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 0.
I20260709 12:20:29.916342  1867 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.917914  1693 leader_election.cc:304] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:29.914672  1850 leader_election.cc:290] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:29.919286  1852 raft_consensus.cc:2804] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:29.919670  1852 raft_consensus.cc:493] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.919984  1852 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.921944  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:29.924165  1846 tablet_bootstrap.cc:492] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3: No bootstrap required, opened a new log
I20260709 12:20:29.925050  1846 ts_tablet_manager.cc:1403] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.008s
I20260709 12:20:29.922966  1651 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:29.922927  1692 leader_election.cc:343] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.922140  1652 raft_consensus.cc:2468] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 1.
I20260709 12:20:29.935750  1725 raft_consensus.cc:2468] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 1.
I20260709 12:20:29.931203  1852 raft_consensus.cc:515] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.941028  1846 raft_consensus.cc:359] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.941700  1651 raft_consensus.cc:2468] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 1.
I20260709 12:20:29.942327  1767 leader_election.cc:304] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:29.944391  1850 raft_consensus.cc:2804] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:29.944608  1852 leader_election.cc:290] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:29.942668  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:29.944658  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:29.942011  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:29.942891  1846 raft_consensus.cc:385] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.947548  1846 raft_consensus.cc:740] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.948508  1851 raft_consensus.cc:493] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.948637  1846 consensus_queue.cc:260] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.947742  1852 raft_consensus.cc:493] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.949891  1801 raft_consensus.cc:2393] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 265a51ff1cc7461e82200790ac4907c3 in current term 1: Already voted for candidate d6c3f73078c047c7a7b8a53cd63d674c in this term.
I20260709 12:20:29.949980  1851 raft_consensus.cc:515] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.951512  1846 ts_tablet_manager.cc:1434] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:20:29.951443  1725 raft_consensus.cc:2393] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d6c3f73078c047c7a7b8a53cd63d674c in current term 1: Already voted for candidate 265a51ff1cc7461e82200790ac4907c3 in this term.
I20260709 12:20:29.952754  1846 tablet_bootstrap.cc:492] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3: Bootstrap starting.
I20260709 12:20:29.947197  1767 leader_election.cc:304] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:29.951452  1652 raft_consensus.cc:2393] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 265a51ff1cc7461e82200790ac4907c3 in current term 1: Already voted for candidate d6c3f73078c047c7a7b8a53cd63d674c in this term.
I20260709 12:20:29.950218  1852 raft_consensus.cc:515] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.959137  1851 leader_election.cc:290] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:29.954665  1867 raft_consensus.cc:515] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:29.978302  1846 tablet_bootstrap.cc:654] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.980419  1867 leader_election.cc:290] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:29.988018  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:29.989267  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:29.986106  1852 leader_election.cc:290] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:29.990094  1651 raft_consensus.cc:2468] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 0.
I20260709 12:20:29.990449  1845 tablet_bootstrap.cc:654] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.993220  1693 leader_election.cc:304] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:29.994058  1652 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.002633  1850 raft_consensus.cc:697] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Becoming Leader. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:30.006568  1883 raft_consensus.cc:2804] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.007157  1883 raft_consensus.cc:493] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:30.002357  1767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:46227, user_credentials={real_user=slave}} blocked reactor thread for 55555.3us
I20260709 12:20:30.009369  1883 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.018210  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:30.014204  1884 raft_consensus.cc:2804] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.021692  1884 raft_consensus.cc:697] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Becoming Leader. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:30.022702  1884 consensus_queue.cc:237] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.027760  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:30.031821  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:30.017694  1850 consensus_queue.cc:237] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:30.033540  1844 tablet_bootstrap.cc:492] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c: No bootstrap required, opened a new log
W20260709 12:20:30.034970  1692 leader_election.cc:343] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.035542  1844 ts_tablet_manager.cc:1403] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent bootstrapping tablet: real 0.247s	user 0.025s	sys 0.010s
I20260709 12:20:30.036710  1882 raft_consensus.cc:493] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.038514  1882 raft_consensus.cc:515] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
W20260709 12:20:30.042517  1692 leader_election.cc:343] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.046677  1845 tablet_bootstrap.cc:492] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38: No bootstrap required, opened a new log
I20260709 12:20:30.048359  1845 ts_tablet_manager.cc:1403] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38: Time spent bootstrapping tablet: real 0.160s	user 0.014s	sys 0.010s
I20260709 12:20:30.051518  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:30.052312  1651 raft_consensus.cc:2468] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 0.
I20260709 12:20:30.053333  1882 leader_election.cc:290] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:30.053033  1652 raft_consensus.cc:2468] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 1.
I20260709 12:20:30.057241  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:30.058126  1725 raft_consensus.cc:2468] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 0.
I20260709 12:20:30.058425  1693 leader_election.cc:304] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:30.051578  1844 raft_consensus.cc:359] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.057931  1767 leader_election.cc:304] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:30.061148  1852 raft_consensus.cc:2804] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.059361  1883 raft_consensus.cc:515] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.062538  1852 raft_consensus.cc:697] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:30.062008  1844 raft_consensus.cc:385] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.064147  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:30.064782  1882 raft_consensus.cc:2804] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.065483  1882 raft_consensus.cc:493] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.064096  1852 consensus_queue.cc:237] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.064918  1844 raft_consensus.cc:740] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.068266  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:30.068737  1652 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.069509  1883 leader_election.cc:290] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:30.067111  1882 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.068674  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:30.071506  1725 raft_consensus.cc:2468] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 0.
I20260709 12:20:30.071420  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
W20260709 12:20:30.074944  1692 leader_election.cc:343] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.072398  1844 consensus_queue.cc:260] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.076196  1845 raft_consensus.cc:359] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.076049  1618 leader_election.cc:304] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:30.079005  1845 raft_consensus.cc:385] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.080147  1851 raft_consensus.cc:2804] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.080133  1845 raft_consensus.cc:740] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.080958  1851 raft_consensus.cc:493] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.082646  1851 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.082859  1845 consensus_queue.cc:260] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.086056  1844 ts_tablet_manager.cc:1434] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.001s
I20260709 12:20:30.087167  1844 tablet_bootstrap.cc:492] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c: Bootstrap starting.
I20260709 12:20:30.088790  1652 raft_consensus.cc:2468] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 1.
I20260709 12:20:30.090523  1693 leader_election.cc:304] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:30.091511  1852 raft_consensus.cc:2804] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.092059  1852 raft_consensus.cc:697] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:30.093017  1852 consensus_queue.cc:237] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.094213  1846 tablet_bootstrap.cc:492] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3: No bootstrap required, opened a new log
I20260709 12:20:30.094703  1846 ts_tablet_manager.cc:1403] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3: Time spent bootstrapping tablet: real 0.142s	user 0.038s	sys 0.013s
I20260709 12:20:30.096150  1882 raft_consensus.cc:515] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.096598  1851 raft_consensus.cc:515] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.098227  1800 raft_consensus.cc:2393] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b074c91c6ea54bea8aae038429fc5a38 in current term 1: Already voted for candidate d6c3f73078c047c7a7b8a53cd63d674c in this term.
I20260709 12:20:30.098585  1692 leader_election.cc:304] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3; no voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:30.099620  1867 raft_consensus.cc:2749] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:30.100937  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:30.101902  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:30.102654  1652 raft_consensus.cc:2393] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d6c3f73078c047c7a7b8a53cd63d674c in current term 1: Already voted for candidate b074c91c6ea54bea8aae038429fc5a38 in this term.
I20260709 12:20:30.105203  1882 leader_election.cc:290] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:30.105792  1726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:30.101521  1725 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.106897  1846 raft_consensus.cc:359] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.107604  1767 leader_election.cc:304] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6c3f73078c047c7a7b8a53cd63d674c; no voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:30.108769  1882 raft_consensus.cc:2749] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:30.110080  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:30.107838  1846 raft_consensus.cc:385] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.110879  1800 raft_consensus.cc:2393] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b074c91c6ea54bea8aae038429fc5a38 in current term 1: Already voted for candidate d6c3f73078c047c7a7b8a53cd63d674c in this term.
I20260709 12:20:30.110913  1846 raft_consensus.cc:740] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.115245  1725 raft_consensus.cc:2468] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 1.
I20260709 12:20:30.113610  1545 catalog_manager.cc:5697] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c reported cstate change: term changed from 0 to 1, leader changed from <none> to d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67). New cstate: current_term: 1 leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.117667  1618 leader_election.cc:304] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:30.116127  1846 consensus_queue.cc:260] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.121109  1846 ts_tablet_manager.cc:1434] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.002s
I20260709 12:20:30.122800  1851 leader_election.cc:290] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 1 election: Requested vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:30.125916  1852 raft_consensus.cc:493] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.126447  1852 raft_consensus.cc:515] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.126878  1892 raft_consensus.cc:2804] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.127439  1892 raft_consensus.cc:697] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 1 LEADER]: Becoming Leader. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:30.128918  1852 leader_election.cc:290] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:30.130239  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
W20260709 12:20:30.131893  1692 leader_election.cc:343] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.128355  1892 consensus_queue.cc:237] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.135025  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:30.125255  1844 tablet_bootstrap.cc:654] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.137780  1845 ts_tablet_manager.cc:1434] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38: Time spent starting tablet: real 0.084s	user 0.007s	sys 0.010s
I20260709 12:20:30.138970  1845 tablet_bootstrap.cc:492] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38: Bootstrap starting.
I20260709 12:20:30.145288  1882 raft_consensus.cc:493] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.146064  1882 raft_consensus.cc:515] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.149122  1882 leader_election.cc:290] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:30.149412  1845 tablet_bootstrap.cc:654] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:30.150482  1693 leader_election.cc:343] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.150552  1726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:30.151069  1693 leader_election.cc:304] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3; no voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:30.151296  1726 raft_consensus.cc:2468] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 0.
I20260709 12:20:30.152675  1767 leader_election.cc:304] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:30.152882  1844 tablet_bootstrap.cc:492] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c: No bootstrap required, opened a new log
I20260709 12:20:30.153599  1852 raft_consensus.cc:2749] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:30.154227  1844 ts_tablet_manager.cc:1403] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent bootstrapping tablet: real 0.067s	user 0.012s	sys 0.007s
I20260709 12:20:30.153601  1850 raft_consensus.cc:2804] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.154738  1850 raft_consensus.cc:493] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.155130  1850 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.132987  1547 catalog_manager.cc:5697] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.158568  1844 raft_consensus.cc:359] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.159404  1844 raft_consensus.cc:385] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.159833  1844 raft_consensus.cc:740] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.160669  1844 consensus_queue.cc:260] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.156998  1545 catalog_manager.cc:5697] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c reported cstate change: term changed from 0 to 1, leader changed from <none> to d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67). New cstate: current_term: 1 leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:30.163448  1844 ts_tablet_manager.cc:1434] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 12:20:30.160087  1547 catalog_manager.cc:5697] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.163794  1850 raft_consensus.cc:515] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.164609  1844 tablet_bootstrap.cc:492] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c: Bootstrap starting.
W20260709 12:20:30.165158  1618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:38587, user_credentials={real_user=slave}} blocked reactor thread for 47621.1us
I20260709 12:20:30.167315  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:30.168224  1652 raft_consensus.cc:2468] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 0.
I20260709 12:20:30.172683  1850 leader_election.cc:290] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:30.173820  1726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:30.174479  1726 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.177356  1844 tablet_bootstrap.cc:654] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.183596  1844 tablet_bootstrap.cc:492] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c: No bootstrap required, opened a new log
I20260709 12:20:30.183596  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:30.184393  1652 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.184441  1844 ts_tablet_manager.cc:1403] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.004s
I20260709 12:20:30.188539  1844 raft_consensus.cc:359] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.189289  1844 raft_consensus.cc:385] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.189602  1844 raft_consensus.cc:740] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.189697  1845 tablet_bootstrap.cc:492] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38: No bootstrap required, opened a new log
I20260709 12:20:30.190274  1845 ts_tablet_manager.cc:1403] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.005s
I20260709 12:20:30.190554  1844 consensus_queue.cc:260] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.192435  1844 ts_tablet_manager.cc:1434] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:20:30.193414  1844 tablet_bootstrap.cc:492] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c: Bootstrap starting.
I20260709 12:20:30.193518  1845 raft_consensus.cc:359] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.194255  1845 raft_consensus.cc:385] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.194519  1845 raft_consensus.cc:740] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.194681  1726 raft_consensus.cc:2468] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 1.
I20260709 12:20:30.195202  1845 consensus_queue.cc:260] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.196352  1767 leader_election.cc:304] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:30.197321  1850 raft_consensus.cc:2804] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.196165  1549 catalog_manager.cc:5697] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 reported cstate change: term changed from 0 to 1, leader changed from <none> to b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "b074c91c6ea54bea8aae038429fc5a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.197469  1845 ts_tablet_manager.cc:1434] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:20:30.197858  1850 raft_consensus.cc:697] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Becoming Leader. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:30.198516  1845 tablet_bootstrap.cc:492] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38: Bootstrap starting.
I20260709 12:20:30.198725  1850 consensus_queue.cc:237] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
W20260709 12:20:30.201627  1596 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:30.212293  1652 raft_consensus.cc:2468] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 1.
I20260709 12:20:30.232138  1845 tablet_bootstrap.cc:654] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.234241  1844 tablet_bootstrap.cc:654] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.247325  1844 tablet_bootstrap.cc:492] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c: No bootstrap required, opened a new log
I20260709 12:20:30.247859  1844 ts_tablet_manager.cc:1403] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.028s
I20260709 12:20:30.249979  1845 tablet_bootstrap.cc:492] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38: No bootstrap required, opened a new log
I20260709 12:20:30.250478  1845 ts_tablet_manager.cc:1403] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.005s
I20260709 12:20:30.251412  1844 raft_consensus.cc:359] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.252133  1844 raft_consensus.cc:385] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.252422  1844 raft_consensus.cc:740] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.253098  1844 consensus_queue.cc:260] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.255520  1844 ts_tablet_manager.cc:1434] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:20:30.255293  1548 catalog_manager.cc:5697] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c reported cstate change: term changed from 0 to 1, leader changed from <none> to d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67). New cstate: current_term: 1 leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:30.256469  1844 tablet_bootstrap.cc:492] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c: Bootstrap starting.
I20260709 12:20:30.256695  1845 raft_consensus.cc:359] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.257372  1845 raft_consensus.cc:385] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.257673  1845 raft_consensus.cc:740] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.258378  1845 consensus_queue.cc:260] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.261184  1845 ts_tablet_manager.cc:1434] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:20:30.263630  1844 tablet_bootstrap.cc:654] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.272006  1844 tablet_bootstrap.cc:492] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c: No bootstrap required, opened a new log
I20260709 12:20:30.272536  1844 ts_tablet_manager.cc:1403] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:20:30.273265  1892 raft_consensus.cc:493] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.273821  1892 raft_consensus.cc:515] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.275455  1844 raft_consensus.cc:359] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.276098  1892 leader_election.cc:290] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:30.276170  1844 raft_consensus.cc:385] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.276567  1844 raft_consensus.cc:740] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.276921  1726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:30.277195  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:30.277638  1726 raft_consensus.cc:2468] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 0.
I20260709 12:20:30.277393  1844 consensus_queue.cc:260] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.277853  1800 raft_consensus.cc:2468] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 0.
I20260709 12:20:30.279328  1617 leader_election.cc:304] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:30.279888  1844 ts_tablet_manager.cc:1434] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:30.282401  1892 raft_consensus.cc:2804] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.282877  1892 raft_consensus.cc:493] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.283311  1892 raft_consensus.cc:3060] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.286443  1852 consensus_queue.cc:1048] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.292011  1892 raft_consensus.cc:515] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.294971  1892 leader_election.cc:290] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 1 election: Requested vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:30.295684  1726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:30.296475  1726 raft_consensus.cc:3060] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.299178  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:30.299834  1801 raft_consensus.cc:3060] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.309654  1726 raft_consensus.cc:2468] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 1.
I20260709 12:20:30.311084  1618 leader_election.cc:304] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:30.312021  1892 raft_consensus.cc:2804] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.312700  1892 raft_consensus.cc:697] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 1 LEADER]: Becoming Leader. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:30.313862  1892 consensus_queue.cc:237] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.320796  1801 raft_consensus.cc:2468] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 1.
I20260709 12:20:30.322932  1883 consensus_queue.cc:1048] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:30.337211  1549 catalog_manager.cc:5697] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 reported cstate change: term changed from 0 to 1, leader changed from <none> to b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "b074c91c6ea54bea8aae038429fc5a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.344625  1852 raft_consensus.cc:493] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.345295  1852 raft_consensus.cc:515] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.348524  1852 leader_election.cc:290] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:30.349357  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677bc6e8dade496dad5352691517e43f" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:30.349924  1801 raft_consensus.cc:2468] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 0.
I20260709 12:20:30.350813  1692 leader_election.cc:304] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:30.351253  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677bc6e8dade496dad5352691517e43f" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:30.351635  1852 raft_consensus.cc:2804] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.352049  1852 raft_consensus.cc:493] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.352015  1652 raft_consensus.cc:2468] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 0.
I20260709 12:20:30.352558  1852 raft_consensus.cc:3060] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.358657  1852 raft_consensus.cc:515] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.360852  1852 leader_election.cc:290] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:30.361819  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677bc6e8dade496dad5352691517e43f" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:30.362013  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677bc6e8dade496dad5352691517e43f" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:30.362286  1652 raft_consensus.cc:3060] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.362624  1801 raft_consensus.cc:3060] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.367311  1652 raft_consensus.cc:2468] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 1.
I20260709 12:20:30.368438  1693 leader_election.cc:304] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:30.369166  1852 raft_consensus.cc:2804] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.369482  1801 raft_consensus.cc:2468] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 1.
I20260709 12:20:30.369649  1852 raft_consensus.cc:697] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:30.371158  1852 consensus_queue.cc:237] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.379573  1549 catalog_manager.cc:5697] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.420820  1850 consensus_queue.cc:1048] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:30.434993  1882 consensus_queue.cc:1048] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.441905  1882 consensus_queue.cc:1048] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.460343  1884 consensus_queue.cc:1048] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.476459  1850 raft_consensus.cc:493] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.477116  1850 raft_consensus.cc:515] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.479678  1850 leader_election.cc:290] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:30.480917  1801 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.482992  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:30.483520  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:30.484524  1652 raft_consensus.cc:2393] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d6c3f73078c047c7a7b8a53cd63d674c in current term 1: Already voted for candidate 265a51ff1cc7461e82200790ac4907c3 in this term.
I20260709 12:20:30.486188  1767 leader_election.cc:304] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6c3f73078c047c7a7b8a53cd63d674c; no voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:30.487141  1884 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.488780  1850 raft_consensus.cc:493] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.489357  1850 raft_consensus.cc:515] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.493803  1852 consensus_queue.cc:1048] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:30.495635  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:30.496389  1652 raft_consensus.cc:2468] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 1.
I20260709 12:20:30.497910  1767 leader_election.cc:304] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:30.498796  1882 raft_consensus.cc:2804] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:30.499238  1882 raft_consensus.cc:493] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.499635  1882 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.499732  1884 raft_consensus.cc:2749] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:30.504522  1726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:30.503985  1850 leader_election.cc:290] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:30.507886  1882 raft_consensus.cc:515] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.509253  1800 raft_consensus.cc:1240] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Rejecting Update request from peer 265a51ff1cc7461e82200790ac4907c3 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:20:30.510074  1882 leader_election.cc:290] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 2 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:30.510787  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:30.511479  1652 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.511965  1726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:30.513453  1852 consensus_queue.cc:1059] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.514415  1867 raft_consensus.cc:3055] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:30.514789  1867 raft_consensus.cc:740] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:30.515780  1867 consensus_queue.cc:260] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.520211  1652 raft_consensus.cc:2468] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 2.
I20260709 12:20:30.520738  1867 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.521740  1767 leader_election.cc:304] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:30.523052  1882 raft_consensus.cc:2804] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:30.523622  1882 raft_consensus.cc:697] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 LEADER]: Becoming Leader. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:30.525029  1882 consensus_queue.cc:237] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.534154  1852 consensus_queue.cc:1048] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.555980  1883 consensus_queue.cc:1048] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.556465  1547 catalog_manager.cc:5697] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c reported cstate change: term changed from 1 to 2, leader changed from 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66) to d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67). New cstate: current_term: 2 leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:30.559101  1801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a"
dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
mode: GRACEFUL
new_leader_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:53222
I20260709 12:20:30.560506  1801 raft_consensus.cc:606] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Received request to transfer leadership to TS b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:30.569010  1800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605"
dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
mode: GRACEFUL
new_leader_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:53222
I20260709 12:20:30.569813  1800 raft_consensus.cc:606] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Received request to transfer leadership to TS b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:30.590447  1882 consensus_queue.cc:1048] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.596832  1725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
mode: GRACEFUL
new_leader_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:40494
I20260709 12:20:30.597504  1725 raft_consensus.cc:606] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Received request to transfer leadership to TS b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:30.604848  1800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7"
dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
mode: GRACEFUL
new_leader_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:53222
I20260709 12:20:30.605671  1800 raft_consensus.cc:606] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Received request to transfer leadership to TS b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:30.612764  1800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0"
dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
mode: GRACEFUL
new_leader_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:53222
I20260709 12:20:30.613499  1800 raft_consensus.cc:606] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 LEADER]: Received request to transfer leadership to TS 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:30.620244  1884 consensus_queue.cc:1048] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.637451  1894 consensus_queue.cc:1048] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.639057  1892 consensus_queue.cc:1048] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:30.643121  1850 raft_consensus.cc:993] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c: : Instructing follower b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:30.643719  1850 raft_consensus.cc:1081] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Signalling peer b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:30.647562  1652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:52560
I20260709 12:20:30.648512  1652 raft_consensus.cc:493] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:30.649088  1652 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.678149  1652 raft_consensus.cc:515] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.682046  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:30.682809  1725 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.701449  1892 consensus_queue.cc:1048] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.696347  1725 raft_consensus.cc:2468] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 2.
I20260709 12:20:30.703186  1652 leader_election.cc:290] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 election: Requested vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:30.703269  1651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
mode: GRACEFUL
new_leader_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:52568
I20260709 12:20:30.703562  1618 leader_election.cc:304] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:30.704021  1651 raft_consensus.cc:606] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 1 LEADER]: Received request to transfer leadership to TS 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:30.705018  1892 raft_consensus.cc:2804] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:30.705507  1892 raft_consensus.cc:697] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Becoming Leader. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:30.706403  1892 consensus_queue.cc:237] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.712020  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:30.712764  1800 raft_consensus.cc:3055] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:30.713110  1800 raft_consensus.cc:740] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:30.714205  1651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
mode: GRACEFUL
new_leader_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:52568
I20260709 12:20:30.714841  1651 raft_consensus.cc:606] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 1 LEADER]: Received request to transfer leadership to TS 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:30.718506  1800 consensus_queue.cc:260] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.719993  1800 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.726263  1726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "677bc6e8dade496dad5352691517e43f"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
mode: GRACEFUL
new_leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
 from {username='slave'} at 127.0.0.1:40494
I20260709 12:20:30.727058  1726 raft_consensus.cc:606] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Received request to transfer leadership to TS d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:30.728799  1800 raft_consensus.cc:2468] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 2.
I20260709 12:20:30.737622  1548 catalog_manager.cc:5697] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 reported cstate change: term changed from 1 to 2, leader changed from d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67) to b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "b074c91c6ea54bea8aae038429fc5a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.739095  1894 raft_consensus.cc:993] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38: : Instructing follower 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:30.739744  1894 raft_consensus.cc:1081] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 1 LEADER]: Signalling peer 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:30.744851  1727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:40490
I20260709 12:20:30.745697  1727 raft_consensus.cc:493] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:30.746204  1727 raft_consensus.cc:3060] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.755954  1894 consensus_queue.cc:1048] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:30.761886  1727 raft_consensus.cc:515] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.766186  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:30.767225  1651 raft_consensus.cc:3055] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:30.767462  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:30.767714  1651 raft_consensus.cc:740] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:30.769627  1727 leader_election.cc:290] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 2 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:30.768996  1651 consensus_queue.cc:260] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.770435  1651 raft_consensus.cc:3060] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.778465  1651 raft_consensus.cc:2468] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 2.
I20260709 12:20:30.781383  1693 leader_election.cc:304] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:30.782383  1867 raft_consensus.cc:2804] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:30.783001  1867 raft_consensus.cc:697] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 2 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:30.783938  1867 consensus_queue.cc:237] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.792716  1548 catalog_manager.cc:5697] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 1 to 2, leader changed from b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65) to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.808470  1867 consensus_queue.cc:1048] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:30.816740  1852 raft_consensus.cc:993] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3: : Instructing follower d6c3f73078c047c7a7b8a53cd63d674c to start an election
I20260709 12:20:30.817308  1852 raft_consensus.cc:1081] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Signalling peer d6c3f73078c047c7a7b8a53cd63d674c to start an election
I20260709 12:20:30.823043  1800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "677bc6e8dade496dad5352691517e43f"
dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
 from {username='slave'} at 127.0.0.1:53210
I20260709 12:20:30.823292  1852 consensus_queue.cc:1048] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.823745  1800 raft_consensus.cc:493] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:30.824162  1800 raft_consensus.cc:3060] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.835589  1800 raft_consensus.cc:515] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.838006  1800 leader_election.cc:290] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 2 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:30.840469  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677bc6e8dade496dad5352691517e43f" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:30.841169  1727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677bc6e8dade496dad5352691517e43f" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:30.841944  1727 raft_consensus.cc:3055] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:30.841274  1651 raft_consensus.cc:3060] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.842273  1727 raft_consensus.cc:740] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:30.843183  1727 consensus_queue.cc:260] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.844611  1727 raft_consensus.cc:3060] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.852869  1651 raft_consensus.cc:2468] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 2.
I20260709 12:20:30.852869  1727 raft_consensus.cc:2468] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 2.
I20260709 12:20:30.855044  1767 leader_election.cc:304] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:30.855953  1884 raft_consensus.cc:2804] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:30.856518  1884 raft_consensus.cc:697] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 2 LEADER]: Becoming Leader. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:30.857414  1884 consensus_queue.cc:237] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.866850  1547 catalog_manager.cc:5697] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c reported cstate change: term changed from 1 to 2, leader changed from 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66) to d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67). New cstate: current_term: 2 leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:30.931777  1852 raft_consensus.cc:993] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3: : Instructing follower b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:30.932314  1867 raft_consensus.cc:1081] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Signalling peer b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:30.934101  1651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:52556
I20260709 12:20:30.934721  1651 raft_consensus.cc:493] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:30.935137  1651 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.940804  1651 raft_consensus.cc:515] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.942826  1651 leader_election.cc:290] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 election: Requested vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:30.944254  1727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:30.944248  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:30.945185  1727 raft_consensus.cc:3055] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:30.945247  1800 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.945688  1727 raft_consensus.cc:740] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:30.946619  1727 consensus_queue.cc:260] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.947899  1727 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.951149  1800 raft_consensus.cc:2468] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 2.
I20260709 12:20:30.952455  1617 leader_election.cc:304] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:30.953274  1894 raft_consensus.cc:2804] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:30.953787  1894 raft_consensus.cc:697] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Becoming Leader. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:30.954646  1894 consensus_queue.cc:237] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:30.956144  1727 raft_consensus.cc:2468] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 2.
I20260709 12:20:30.963543  1547 catalog_manager.cc:5697] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 reported cstate change: term changed from 1 to 2, leader changed from 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66) to b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "b074c91c6ea54bea8aae038429fc5a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.986958  1882 raft_consensus.cc:993] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c: : Instructing follower b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:30.987483  1884 raft_consensus.cc:1081] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Signalling peer b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:30.989241  1651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:52560
I20260709 12:20:30.989929  1651 raft_consensus.cc:493] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:30.990317  1651 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:30.997989  1651 raft_consensus.cc:515] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:31.000414  1651 leader_election.cc:290] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 election: Requested vote from peers d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:31.001608  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:31.002328  1800 raft_consensus.cc:3055] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:31.002677  1800 raft_consensus.cc:740] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:31.003201  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:31.003825  1725 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:31.003693  1800 consensus_queue.cc:260] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:31.005107  1800 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:31.015941  1800 raft_consensus.cc:2468] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 2.
I20260709 12:20:31.016080  1725 raft_consensus.cc:2468] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 2.
I20260709 12:20:31.017314  1617 leader_election.cc:304] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:31.018132  1894 raft_consensus.cc:2804] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:31.018673  1894 raft_consensus.cc:697] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Becoming Leader. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:31.019563  1894 consensus_queue.cc:237] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:31.026978  1547 catalog_manager.cc:5697] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 reported cstate change: term changed from 1 to 2, leader changed from d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67) to b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "b074c91c6ea54bea8aae038429fc5a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.040771  1850 raft_consensus.cc:993] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c: : Instructing follower b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:31.041337  1882 raft_consensus.cc:1081] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Signalling peer b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:31.043313  1651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:52560
I20260709 12:20:31.043982  1651 raft_consensus.cc:493] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:31.044417  1651 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:31.050738  1651 raft_consensus.cc:515] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:31.052639  1651 leader_election.cc:290] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 election: Requested vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:31.053973  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:31.053901  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:31.054847  1725 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:31.055439  1800 raft_consensus.cc:3055] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:31.055800  1800 raft_consensus.cc:740] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:31.056686  1800 consensus_queue.cc:260] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:31.058044  1800 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:31.062456  1727 raft_consensus.cc:1275] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:20:31.062427  1725 raft_consensus.cc:2468] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 2.
I20260709 12:20:31.063037  1727 pending_rounds.cc:85] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3: Aborting all ops after (but not including) 0
I20260709 12:20:31.063434  1727 pending_rounds.cc:107] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:20:31.063777  1727 raft_consensus.cc:2889] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:31.063838  1618 leader_election.cc:304] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:31.064785  1894 raft_consensus.cc:2804] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:31.065308  1894 raft_consensus.cc:697] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Becoming Leader. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:31.065778  1882 consensus_queue.cc:1048] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:31.066251  1894 consensus_queue.cc:237] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:31.073002  1850 raft_consensus.cc:993] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c: : Instructing follower 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:31.073490  1850 raft_consensus.cc:1081] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 LEADER]: Signalling peer 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:31.077476  1727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:40484
I20260709 12:20:31.076599  1548 catalog_manager.cc:5697] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 reported cstate change: term changed from 1 to 2, leader changed from d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67) to b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "b074c91c6ea54bea8aae038429fc5a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.078137  1727 raft_consensus.cc:493] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:31.078557  1727 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:31.082407  1651 raft_consensus.cc:1275] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:20:31.068645  1800 raft_consensus.cc:2468] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 2.
I20260709 12:20:31.089109  1882 consensus_queue.cc:1048] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.091253  1727 raft_consensus.cc:515] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:31.092360  1725 raft_consensus.cc:1240] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Rejecting Update request from peer d6c3f73078c047c7a7b8a53cd63d674c for earlier term 2. Current term is 3. Ops: []
I20260709 12:20:31.093616  1727 leader_election.cc:290] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 3 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:31.095166  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:31.095359  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:31.097231  1882 consensus_queue.cc:1059] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.099476  1800 raft_consensus.cc:3055] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:31.103045  1800 raft_consensus.cc:740] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:31.104130  1800 consensus_queue.cc:260] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:31.105508  1800 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:31.114557  1800 raft_consensus.cc:2468] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 3.
W20260709 12:20:31.115281  1928 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:20:31.117264  1692 leader_election.cc:304] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c; no voters: b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:31.118201  1883 raft_consensus.cc:2804] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:31.118710  1883 raft_consensus.cc:697] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:31.119593  1883 consensus_queue.cc:237] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:31.128077  1549 catalog_manager.cc:5697] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 2 to 3, leader changed from d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67) to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.200510  1727 raft_consensus.cc:1275] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.202051  1894 consensus_queue.cc:1048] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.227871  1800 raft_consensus.cc:1275] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.229588  1894 consensus_queue.cc:1048] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.232978  1851 raft_consensus.cc:993] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38: : Instructing follower 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:31.233459  1851 raft_consensus.cc:1081] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 1 LEADER]: Signalling peer 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:31.236013  1727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:40490
I20260709 12:20:31.236608  1727 raft_consensus.cc:493] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:31.237170  1727 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:31.239507  1801 raft_consensus.cc:3060] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:31.245333  1727 raft_consensus.cc:515] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:31.247431  1801 raft_consensus.cc:1275] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.248124  1727 leader_election.cc:290] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 2 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:31.248436  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:31.249095  1651 raft_consensus.cc:3055] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:31.249313  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:31.249500  1651 raft_consensus.cc:740] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:31.249979  1799 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:31.250409  1651 consensus_queue.cc:260] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:31.251768  1651 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:31.255110  1883 consensus_queue.cc:1048] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.263506  1799 raft_consensus.cc:2468] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 2.
I20260709 12:20:31.264798  1727 raft_consensus.cc:1275] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.265790  1652 raft_consensus.cc:1275] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.269840  1692 leader_election.cc:304] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:31.267706  1651 raft_consensus.cc:2468] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 2.
I20260709 12:20:31.271132  1867 raft_consensus.cc:2804] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:31.271595  1867 raft_consensus.cc:697] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 2 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:31.272221  1883 consensus_queue.cc:1048] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.272411  1867 consensus_queue.cc:237] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:31.274726  1928 consensus_queue.cc:1048] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.285895  1545 catalog_manager.cc:5697] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 1 to 2, leader changed from b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65) to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.302577  1651 raft_consensus.cc:1275] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.304203  1884 consensus_queue.cc:1048] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.381734  1800 raft_consensus.cc:1275] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.383366  1894 consensus_queue.cc:1048] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.396529  1727 raft_consensus.cc:1275] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.398343  1930 consensus_queue.cc:1048] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:31.476996  1800 raft_consensus.cc:1275] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.478703  1930 consensus_queue.cc:1048] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.488511  1727 raft_consensus.cc:1275] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.490198  1894 consensus_queue.cc:1048] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:31.523452  1727 raft_consensus.cc:1275] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.525138  1894 consensus_queue.cc:1048] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.536124  1800 raft_consensus.cc:1275] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.538113  1930 consensus_queue.cc:1048] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.545353  1651 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:31.555673  1651 raft_consensus.cc:1275] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:31.558189  1867 consensus_queue.cc:1048] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:31.568810  1801 raft_consensus.cc:1275] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:31.570788  1931 consensus_queue.cc:1048] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.658952  1801 raft_consensus.cc:1275] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.660738  1867 consensus_queue.cc:1048] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:31.676831  1651 raft_consensus.cc:1275] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:31.678733  1852 consensus_queue.cc:1048] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:33.762392  1517 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:20:33.763242  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:33.786782  1652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
mode: GRACEFUL
new_leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
 from {username='slave'} at 127.0.0.1:52574
I20260709 12:20:33.787424  1652 raft_consensus.cc:606] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Received request to transfer leadership to TS d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:33.800587  1727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
mode: GRACEFUL
new_leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
 from {username='slave'} at 127.0.0.1:40502
I20260709 12:20:33.801203  1727 raft_consensus.cc:606] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Received request to transfer leadership to TS d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:33.802507  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:33.803035  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:34.302904  1948 raft_consensus.cc:993] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3: : Instructing follower d6c3f73078c047c7a7b8a53cd63d674c to start an election
I20260709 12:20:34.303452  1943 raft_consensus.cc:1081] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Signalling peer d6c3f73078c047c7a7b8a53cd63d674c to start an election
I20260709 12:20:34.305253  1801 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0"
dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
 from {username='slave'} at 127.0.0.1:53210
I20260709 12:20:34.305941  1801 raft_consensus.cc:493] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:34.306351  1801 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:34.311550  1801 raft_consensus.cc:515] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:34.313290  1801 leader_election.cc:290] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 4 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:34.314540  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:34.314703  1727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:34.315285  1652 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:34.315310  1727 raft_consensus.cc:3055] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:34.315857  1727 raft_consensus.cc:740] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:34.316751  1727 consensus_queue.cc:260] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:34.318095  1727 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:34.323822  1652 raft_consensus.cc:2468] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 4.
I20260709 12:20:34.325243  1767 leader_election.cc:304] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:34.326069  1727 raft_consensus.cc:2468] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 4.
I20260709 12:20:34.326146  1947 raft_consensus.cc:2804] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:34.327486  1947 raft_consensus.cc:697] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 LEADER]: Becoming Leader. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:34.328372  1947 consensus_queue.cc:237] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:34.338768  1547 catalog_manager.cc:5697] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c reported cstate change: term changed from 3 to 4, leader changed from 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66) to d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67). New cstate: current_term: 4 leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:34.383244  1892 raft_consensus.cc:993] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38: : Instructing follower d6c3f73078c047c7a7b8a53cd63d674c to start an election
I20260709 12:20:34.383736  1892 raft_consensus.cc:1081] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Signalling peer d6c3f73078c047c7a7b8a53cd63d674c to start an election
I20260709 12:20:34.385373  1801 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a"
dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
 from {username='slave'} at 127.0.0.1:53214
I20260709 12:20:34.385973  1801 raft_consensus.cc:493] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:34.386302  1801 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:34.391610  1801 raft_consensus.cc:515] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:34.393641  1801 leader_election.cc:290] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 3 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:34.394694  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:34.394992  1727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:34.395269  1652 raft_consensus.cc:3055] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:34.395566  1727 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:34.395601  1652 raft_consensus.cc:740] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:34.396453  1652 consensus_queue.cc:260] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:34.397478  1652 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:34.401276  1727 raft_consensus.cc:2468] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 3.
I20260709 12:20:34.402917  1652 raft_consensus.cc:2468] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 3.
I20260709 12:20:34.402806  1767 leader_election.cc:304] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:34.404033  1947 raft_consensus.cc:2804] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:34.404536  1947 raft_consensus.cc:697] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 3 LEADER]: Becoming Leader. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:34.405339  1947 consensus_queue.cc:237] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:34.413107  1547 catalog_manager.cc:5697] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c reported cstate change: term changed from 2 to 3, leader changed from b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65) to d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67). New cstate: current_term: 3 leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:34.804242  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:34.809460  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:34.810045  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:34.824640  1652 raft_consensus.cc:1275] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:20:34.826241  1947 consensus_queue.cc:1048] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:34.841375  1727 raft_consensus.cc:1275] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:20:34.845013  1959 consensus_queue.cc:1048] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:34.849720  1652 raft_consensus.cc:1275] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:34.851647  1959 consensus_queue.cc:1048] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:34.863801  1727 raft_consensus.cc:1275] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:34.867502  1947 consensus_queue.cc:1048] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:35.811270  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:35.816596  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:35.817131  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:36.818845  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:36.825575  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:36.826133  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:37.827546  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:37.833886  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:37.834381  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:39.544884  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:39.555775  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:39.556826  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:39.644335  1973 raft_consensus.cc:493] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d6c3f73078c047c7a7b8a53cd63d674c)
I20260709 12:20:39.645496  1973 raft_consensus.cc:515] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:39.681674  1975 raft_consensus.cc:493] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 265a51ff1cc7461e82200790ac4907c3)
I20260709 12:20:39.682655  1975 raft_consensus.cc:515] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:39.687081  1975 leader_election.cc:290] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:39.834818  1975 raft_consensus.cc:493] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 265a51ff1cc7461e82200790ac4907c3)
I20260709 12:20:39.835872  1975 raft_consensus.cc:515] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:39.841347  1975 leader_election.cc:290] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:39.649190  1973 leader_election.cc:290] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:39.877775  1975 raft_consensus.cc:493] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 265a51ff1cc7461e82200790ac4907c3)
I20260709 12:20:39.878851  1975 raft_consensus.cc:515] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:39.883744  1975 leader_election.cc:290] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:40.008319  1978 raft_consensus.cc:493] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b074c91c6ea54bea8aae038429fc5a38)
I20260709 12:20:40.009171  1978 raft_consensus.cc:515] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.009847  1977 raft_consensus.cc:493] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 265a51ff1cc7461e82200790ac4907c3)
I20260709 12:20:40.010943  1977 raft_consensus.cc:515] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.012419  1978 leader_election.cc:290] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:40.013167  1977 leader_election.cc:290] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:40.026811  1797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:40.027177  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:40.027655  1797 raft_consensus.cc:2468] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 2.
I20260709 12:20:40.027964  1979 raft_consensus.cc:493] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b074c91c6ea54bea8aae038429fc5a38)
I20260709 12:20:40.028627  1979 raft_consensus.cc:515] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.029964  1617 leader_election.cc:304] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:40.020136  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:40.031340  1979 leader_election.cc:290] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:40.029958  1796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:40.021385  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:40.031483  1968 raft_consensus.cc:2804] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:20:40.032618  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677bc6e8dade496dad5352691517e43f" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:40.032686  1968 raft_consensus.cc:493] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 265a51ff1cc7461e82200790ac4907c3)
I20260709 12:20:40.013582  1978 raft_consensus.cc:493] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 265a51ff1cc7461e82200790ac4907c3)
I20260709 12:20:40.033281  1968 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:40.026895  1980 raft_consensus.cc:493] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b074c91c6ea54bea8aae038429fc5a38)
I20260709 12:20:40.034153  1980 raft_consensus.cc:515] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.033622  1978 raft_consensus.cc:515] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.034933  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:40.035722  1798 raft_consensus.cc:2468] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 1.
I20260709 12:20:40.037801  1978 leader_election.cc:290] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:40.037854  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:40.041676  1726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:40.044871  1968 raft_consensus.cc:515] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.046396  1650 raft_consensus.cc:2393] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d6c3f73078c047c7a7b8a53cd63d674c in current term 3: Already voted for candidate b074c91c6ea54bea8aae038429fc5a38 in this term.
I20260709 12:20:40.047161  1968 leader_election.cc:290] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 election: Requested vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:40.049985  1652 raft_consensus.cc:1240] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Rejecting Update request from peer 265a51ff1cc7461e82200790ac4907c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:20:40.053691  1973 consensus_queue.cc:1059] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:40.054303  1980 leader_election.cc:290] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:40.062031  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:40.062922  1799 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:40.066406  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:40.067940  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:40.069347  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:40.071084  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:40.077876  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:40.083459  1799 raft_consensus.cc:2468] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 3.
I20260709 12:20:40.084241  1798 raft_consensus.cc:1240] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Rejecting Update request from peer 265a51ff1cc7461e82200790ac4907c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:20:40.084787  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:40.085096  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:40.085870  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b074c91c6ea54bea8aae038429fc5a38" is_pre_election: true
I20260709 12:20:40.086755  1618 leader_election.cc:304] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38; no voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:40.087322  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677bc6e8dade496dad5352691517e43f" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:40.091435  1982 raft_consensus.cc:2749] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:20:40.091068  1767 leader_election.cc:304] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6c3f73078c047c7a7b8a53cd63d674c; no voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:40.090371  1617 leader_election.cc:304] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:40.093299  1972 raft_consensus.cc:2749] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:20:40.093886  1982 raft_consensus.cc:2804] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:40.093288  1767 leader_election.cc:304] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6c3f73078c047c7a7b8a53cd63d674c; no voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:40.094296  1982 raft_consensus.cc:493] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 265a51ff1cc7461e82200790ac4907c3)
I20260709 12:20:40.095695  1980 raft_consensus.cc:2749] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:20:40.094693  1982 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:40.096830  1767 leader_election.cc:304] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6c3f73078c047c7a7b8a53cd63d674c; no voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:40.097592  1980 raft_consensus.cc:2749] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:20:40.098883  1767 leader_election.cc:304] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6c3f73078c047c7a7b8a53cd63d674c; no voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:40.099985  1980 raft_consensus.cc:2749] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:20:40.099460  1617 leader_election.cc:304] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:40.096412  1973 raft_consensus.cc:3055] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:40.101765  1973 raft_consensus.cc:740] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:40.101778  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:40.102044  1968 raft_consensus.cc:2804] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:40.103387  1968 raft_consensus.cc:697] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Becoming Leader. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:40.104432  1968 consensus_queue.cc:237] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.107031  1692 leader_election.cc:304] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3; no voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:40.108399  1767 leader_election.cc:304] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6c3f73078c047c7a7b8a53cd63d674c; no voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:40.108294  1986 raft_consensus.cc:2749] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:20:40.109794  1980 raft_consensus.cc:2749] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:40.112006  1985 consensus_queue.cc:1059] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:20:40.116863  1973 consensus_queue.cc:260] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.118431  1973 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:40.121188  1549 catalog_manager.cc:5697] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 reported cstate change: term changed from 2 to 3, leader changed from 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66) to b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "b074c91c6ea54bea8aae038429fc5a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:40.122489  1982 raft_consensus.cc:515] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.126627  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:40.126854  1982 leader_election.cc:290] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 election: Requested vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:40.127102  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:40.128738  1617 leader_election.cc:304] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38; no voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:40.129612  1982 raft_consensus.cc:2749] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:20:40.315689  1748 debug-util.cc:398] Leaking SignalData structure 0x7b080012a060 after lost signal to thread 1520
W20260709 12:20:40.316861  1748 debug-util.cc:398] Leaking SignalData structure 0x7b080012a740 after lost signal to thread 1583
W20260709 12:20:40.317863  1748 debug-util.cc:398] Leaking SignalData structure 0x7b0800175080 after lost signal to thread 1676
W20260709 12:20:40.318688  1748 debug-util.cc:398] Leaking SignalData structure 0x7b08001e3680 after lost signal to thread 1751
W20260709 12:20:40.319593  1748 debug-util.cc:398] Leaking SignalData structure 0x7b08001be3c0 after lost signal to thread 1825
I20260709 12:20:40.405736  1650 raft_consensus.cc:1240] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Rejecting Update request from peer 265a51ff1cc7461e82200790ac4907c3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:20:40.407459  1973 consensus_queue.cc:1059] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:40.408917  1985 raft_consensus.cc:3055] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:40.409317  1985 raft_consensus.cc:740] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:40.410277  1985 consensus_queue.cc:260] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.411489  1985 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:40.558400  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:40.568316  1650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
mode: GRACEFUL
new_leader_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:52574
I20260709 12:20:40.569314  1650 raft_consensus.cc:606] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Received request to transfer leadership to TS 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:40.571046  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:40.571553  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:40.681869  1798 raft_consensus.cc:1275] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:40.684775  1982 consensus_queue.cc:1048] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:40.689992  1992 raft_consensus.cc:993] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38: : Instructing follower 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:40.690706  1992 raft_consensus.cc:1081] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Signalling peer 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:40.692433  1725 raft_consensus.cc:1275] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:40.695053  1725 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:40490
I20260709 12:20:40.696137  1725 raft_consensus.cc:493] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:40.696765  1725 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:40.699661  1994 consensus_queue.cc:1048] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:40.717074  1725 raft_consensus.cc:515] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.732800  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:40.733649  1650 raft_consensus.cc:3055] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:40.734040  1650 raft_consensus.cc:740] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:40.734983  1650 consensus_queue.cc:260] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.736331  1650 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:40.745661  1725 leader_election.cc:290] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 3 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:40.750025  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:40.749917  1650 raft_consensus.cc:2468] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 3.
I20260709 12:20:40.750737  1798 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:40.751943  1693 leader_election.cc:304] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:40.752947  1985 raft_consensus.cc:2804] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:40.753464  1985 raft_consensus.cc:697] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:40.754446  1985 consensus_queue.cc:237] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:40.758879  1798 raft_consensus.cc:2468] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 3.
I20260709 12:20:40.764439  1547 catalog_manager.cc:5697] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 2 to 3, leader changed from b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65) to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:41.185984  1650 raft_consensus.cc:1275] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:41.187490  1985 consensus_queue.cc:1048] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:41.198560  1798 raft_consensus.cc:1275] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:41.200549  1985 consensus_queue.cc:1048] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:41.572566  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:41.578469  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:41.579036  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:41.816202  1992 raft_consensus.cc:493] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.816932  1992 raft_consensus.cc:515] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:41.819512  1992 leader_election.cc:290] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:41.820595  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3" is_pre_election: true
I20260709 12:20:41.820611  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" is_pre_election: true
I20260709 12:20:41.821364  1724 raft_consensus.cc:2468] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 2.
I20260709 12:20:41.821405  1798 raft_consensus.cc:2468] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 1.
I20260709 12:20:41.822860  1617 leader_election.cc:304] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:41.823733  1992 raft_consensus.cc:2804] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:20:41.824216  1992 raft_consensus.cc:493] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:41.824677  1992 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:41.833670  1992 raft_consensus.cc:515] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:41.842705  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:41.843663  1724 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:41.843930  1992 leader_election.cc:290] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 election: Requested vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:41.847703  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:41.849942  1798 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:20:41.851996  1724 raft_consensus.cc:2468] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 3.
I20260709 12:20:41.853297  1618 leader_election.cc:304] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:41.856904  2009 raft_consensus.cc:2804] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:41.857275  1798 raft_consensus.cc:2468] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 3.
I20260709 12:20:41.857465  2009 raft_consensus.cc:697] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Becoming Leader. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:41.859215  2009 consensus_queue.cc:237] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:41.868970  1545 catalog_manager.cc:5697] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 reported cstate change: term changed from 1 to 3, leader changed from 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66) to b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "b074c91c6ea54bea8aae038429fc5a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.380142  1798 raft_consensus.cc:1275] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:42.381814  2009 consensus_queue.cc:1048] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:42.410658  1724 raft_consensus.cc:1275] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:42.414101  2009 consensus_queue.cc:1048] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:42.580256  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:42.589879  1650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
mode: GRACEFUL
new_leader_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:52574
I20260709 12:20:42.590517  1650 raft_consensus.cc:606] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Received request to transfer leadership to TS 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:42.591882  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:42.592348  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:42.924798  2009 raft_consensus.cc:993] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38: : Instructing follower 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:42.925432  2017 raft_consensus.cc:1081] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Signalling peer 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:42.927135  1725 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:40490
I20260709 12:20:42.927757  1725 raft_consensus.cc:493] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:42.928097  1725 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:42.933709  1725 raft_consensus.cc:515] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:42.935344  1725 leader_election.cc:290] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 4 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:42.936398  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:42.936393  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31f514965854ca89d1b161aa8e5c802" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:42.937263  1650 raft_consensus.cc:3055] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:42.937263  1798 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:42.937829  1650 raft_consensus.cc:740] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:42.938694  1650 consensus_queue.cc:260] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:42.939967  1650 raft_consensus.cc:3060] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:42.945417  1798 raft_consensus.cc:2468] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 4.
I20260709 12:20:42.946892  1692 leader_election.cc:304] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:42.947157  1650 raft_consensus.cc:2468] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 4.
I20260709 12:20:42.947672  2008 raft_consensus.cc:2804] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:42.948930  2008 raft_consensus.cc:697] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 4 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:42.949798  2008 consensus_queue.cc:237] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:42.957166  1545 catalog_manager.cc:5697] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 3 to 4, leader changed from b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65) to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 4 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.329375  1650 raft_consensus.cc:1275] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:43.330632  2023 consensus_queue.cc:1048] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:43.345139  1798 raft_consensus.cc:1275] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:43.346971  2008 consensus_queue.cc:1048] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:43.687170  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:43.692337  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:43.692723  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:20:44.511723  1822 debug-util.cc:398] Leaking SignalData structure 0x7b080016d060 after lost signal to thread 1520
W20260709 12:20:44.512730  1822 debug-util.cc:398] Leaking SignalData structure 0x7b080016d740 after lost signal to thread 1583
W20260709 12:20:44.513727  1822 debug-util.cc:398] Leaking SignalData structure 0x7b0800171920 after lost signal to thread 1676
W20260709 12:20:44.514707  1822 debug-util.cc:398] Leaking SignalData structure 0x7b0800171980 after lost signal to thread 1751
W20260709 12:20:44.515524  1822 debug-util.cc:398] Leaking SignalData structure 0x7b08001bdc00 after lost signal to thread 1825
I20260709 12:20:44.730334  1517 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:20:44.736127  1798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a"
dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
mode: GRACEFUL
new_leader_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:53222
I20260709 12:20:44.736742  1798 raft_consensus.cc:606] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 3 LEADER]: Received request to transfer leadership to TS 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:44.742882  1650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
mode: GRACEFUL
new_leader_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:52568
I20260709 12:20:44.743625  1650 raft_consensus.cc:606] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Received request to transfer leadership to TS 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:44.750110  1650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
mode: GRACEFUL
new_leader_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:52568
I20260709 12:20:44.750826  1650 raft_consensus.cc:606] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Received request to transfer leadership to TS 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:44.759338  1798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0"
dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
mode: GRACEFUL
new_leader_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:53222
I20260709 12:20:44.759994  1798 raft_consensus.cc:606] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 LEADER]: Received request to transfer leadership to TS 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:44.765332  1650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
mode: GRACEFUL
new_leader_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:52568
I20260709 12:20:44.766073  1650 raft_consensus.cc:606] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Received request to transfer leadership to TS 265a51ff1cc7461e82200790ac4907c3
I20260709 12:20:44.851078  2036 raft_consensus.cc:993] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38: : Instructing follower 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:44.851702  2038 raft_consensus.cc:1081] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Signalling peer 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:44.853403  1724 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:40490
I20260709 12:20:44.854002  1724 raft_consensus.cc:493] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:44.854339  1724 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:44.859752  1724 raft_consensus.cc:515] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:44.862112  1724 leader_election.cc:290] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 3 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:44.863113  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:44.863232  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:44.863669  1650 raft_consensus.cc:3055] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:44.863859  1798 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:44.864060  1650 raft_consensus.cc:740] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:44.864948  1650 consensus_queue.cc:260] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:44.866029  1650 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:44.871315  1650 raft_consensus.cc:2468] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 3.
I20260709 12:20:44.872480  1798 raft_consensus.cc:2468] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 3.
I20260709 12:20:44.872511  1693 leader_election.cc:304] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:44.873522  2040 raft_consensus.cc:2804] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:44.874063  2040 raft_consensus.cc:697] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:44.874893  2040 consensus_queue.cc:237] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:44.880312  2038 raft_consensus.cc:993] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38: : Instructing follower 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:44.880872  2038 raft_consensus.cc:1081] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Signalling peer 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:44.882453  1724 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:40490
I20260709 12:20:44.882984  1724 raft_consensus.cc:493] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:44.883486  1724 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:44.886255  1549 catalog_manager.cc:5697] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 2 to 3, leader changed from b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65) to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:44.892904  1724 raft_consensus.cc:515] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:44.896770  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:44.896771  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:44.897559  1798 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:44.897799  1650 raft_consensus.cc:3055] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:44.898274  1650 raft_consensus.cc:740] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:44.899387  1650 consensus_queue.cc:260] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:44.901430  1650 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:44.906873  1798 raft_consensus.cc:2468] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 3.
I20260709 12:20:44.908244  1692 leader_election.cc:304] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:44.909263  2040 raft_consensus.cc:2804] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:44.909795  2040 raft_consensus.cc:697] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:44.911090  2038 raft_consensus.cc:993] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38: : Instructing follower 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:44.910751  2040 consensus_queue.cc:237] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:44.911646  2038 raft_consensus.cc:1081] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Signalling peer 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:44.915202  1724 leader_election.cc:290] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 3 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:44.916342  1723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:40490
I20260709 12:20:44.917168  1723 raft_consensus.cc:493] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:44.917690  1723 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:44.930686  1723 raft_consensus.cc:515] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:44.934612  1650 raft_consensus.cc:2468] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 3.
I20260709 12:20:44.934635  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:44.935352  1652 raft_consensus.cc:3055] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:44.935659  1652 raft_consensus.cc:740] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:44.936457  1652 consensus_queue.cc:260] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:44.937575  1652 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:44.937808  1723 leader_election.cc:290] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 4 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:44.939481  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:44.940107  1798 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:44.946892  1652 raft_consensus.cc:2468] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 4.
I20260709 12:20:44.948760  1693 leader_election.cc:304] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:44.949823  2040 raft_consensus.cc:2804] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:44.950387  2040 raft_consensus.cc:697] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 4 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:44.951318  2040 consensus_queue.cc:237] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:44.949332  1798 raft_consensus.cc:2468] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 4.
I20260709 12:20:44.961673  1549 catalog_manager.cc:5697] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 2 to 3, leader changed from b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65) to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:44.962911  1549 catalog_manager.cc:5697] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 3 to 4, leader changed from b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65) to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 4 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.168776  2048 raft_consensus.cc:993] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c: : Instructing follower 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:45.169313  2035 raft_consensus.cc:1081] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 3 LEADER]: Signalling peer 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:45.171331  1723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:40484
I20260709 12:20:45.172045  1723 raft_consensus.cc:493] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:45.172472  1723 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:45.181205  1723 raft_consensus.cc:515] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:45.183795  1723 leader_election.cc:290] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 4 election: Requested vote from peers d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083), b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227)
I20260709 12:20:45.184895  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:45.184947  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012b99b449bc4f02a31b9c18fa87475a" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:45.185722  1652 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:45.185722  1798 raft_consensus.cc:3055] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:45.186218  1798 raft_consensus.cc:740] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:45.187151  1798 consensus_queue.cc:260] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:45.188532  1798 raft_consensus.cc:3060] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:45.193516  1652 raft_consensus.cc:2468] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 4.
I20260709 12:20:45.194810  1693 leader_election.cc:304] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, b074c91c6ea54bea8aae038429fc5a38; no voters: 
I20260709 12:20:45.195798  2040 raft_consensus.cc:2804] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:45.195992  1798 raft_consensus.cc:2468] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 4.
I20260709 12:20:45.196345  2040 raft_consensus.cc:697] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 4 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:45.197247  2040 consensus_queue.cc:237] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } }
I20260709 12:20:45.206638  1549 catalog_manager.cc:5697] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 3 to 4, leader changed from d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67) to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 4 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:45.234622  2048 raft_consensus.cc:993] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c: : Instructing follower 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:45.235116  2035 raft_consensus.cc:1081] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 LEADER]: Signalling peer 265a51ff1cc7461e82200790ac4907c3 to start an election
I20260709 12:20:45.236928  1723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
 from {username='slave'} at 127.0.0.1:40484
I20260709 12:20:45.237497  1723 raft_consensus.cc:493] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:45.237937  1723 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:45.243160  1723 raft_consensus.cc:515] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:45.244966  1723 leader_election.cc:290] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 5 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:45.246029  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:45.246021  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a99c2cf2385417a86196fc38ccfbab0" candidate_uuid: "265a51ff1cc7461e82200790ac4907c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:45.246766  1798 raft_consensus.cc:3055] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:20:45.246814  1652 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:45.247234  1798 raft_consensus.cc:740] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:45.248072  1798 consensus_queue.cc:260] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:45.249250  1798 raft_consensus.cc:3060] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:45.255014  1798 raft_consensus.cc:2468] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 5.
I20260709 12:20:45.255007  1652 raft_consensus.cc:2468] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 265a51ff1cc7461e82200790ac4907c3 in term 5.
I20260709 12:20:45.256295  1692 leader_election.cc:304] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265a51ff1cc7461e82200790ac4907c3, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:45.257023  2040 raft_consensus.cc:2804] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:20:45.257508  2040 raft_consensus.cc:697] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 5 LEADER]: Becoming Leader. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:45.258440  2040 consensus_queue.cc:237] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:45.267035  1549 catalog_manager.cc:5697] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 reported cstate change: term changed from 4 to 5, leader changed from d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67) to 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66). New cstate: current_term: 5 leader_uuid: "265a51ff1cc7461e82200790ac4907c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.291932  1652 raft_consensus.cc:1275] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:45.293473  2040 consensus_queue.cc:1048] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:45.306193  1798 raft_consensus.cc:1275] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:45.309021  2040 consensus_queue.cc:1048] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:45.370101  1652 raft_consensus.cc:1275] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:45.371428  2008 consensus_queue.cc:1048] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:45.382895  1798 raft_consensus.cc:1275] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:45.384351  2008 consensus_queue.cc:1048] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:45.385914  1799 raft_consensus.cc:1275] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:20:45.389322  2008 consensus_queue.cc:1048] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:45.399999  1652 raft_consensus.cc:1275] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:20:45.403507  2008 consensus_queue.cc:1048] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:45.615579  1799 raft_consensus.cc:1275] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:45.616885  2053 consensus_queue.cc:1048] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:45.632159  1652 raft_consensus.cc:1275] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:45.633812  2058 consensus_queue.cc:1048] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:45.666858  1799 raft_consensus.cc:1275] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 5 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:20:45.668278  2058 consensus_queue.cc:1048] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:45.677022  1652 raft_consensus.cc:1275] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 5 FOLLOWER]: Refusing update from remote peer 265a51ff1cc7461e82200790ac4907c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:20:45.678457  2008 consensus_queue.cc:1048] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:47.807267  1517 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:20:47.808163  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:47.815657  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
mode: GRACEFUL
new_leader_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:40502
I20260709 12:20:47.816696  1723 raft_consensus.cc:606] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 2 LEADER]: Received request to transfer leadership to TS b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:47.820441  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
mode: GRACEFUL
new_leader_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:40502
I20260709 12:20:47.821043  1723 raft_consensus.cc:606] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 4 LEADER]: Received request to transfer leadership to TS b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:47.823911  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
mode: GRACEFUL
new_leader_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:40502
I20260709 12:20:47.824409  1723 raft_consensus.cc:606] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Received request to transfer leadership to TS b074c91c6ea54bea8aae038429fc5a38
I20260709 12:20:47.826978  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
mode: GRACEFUL
new_leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
 from {username='slave'} at 127.0.0.1:40502
I20260709 12:20:47.827483  1723 raft_consensus.cc:606] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Received request to transfer leadership to TS d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:47.829808  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3"
dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
mode: GRACEFUL
new_leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
 from {username='slave'} at 127.0.0.1:40502
I20260709 12:20:47.830214  1723 raft_consensus.cc:606] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Received request to transfer leadership to TS d6c3f73078c047c7a7b8a53cd63d674c
I20260709 12:20:47.831283  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:47.831735  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:47.904031  2058 raft_consensus.cc:993] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3: : Instructing follower d6c3f73078c047c7a7b8a53cd63d674c to start an election
I20260709 12:20:47.904699  2069 raft_consensus.cc:1081] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Signalling peer d6c3f73078c047c7a7b8a53cd63d674c to start an election
I20260709 12:20:47.906577  1799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3"
dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
 from {username='slave'} at 127.0.0.1:53210
I20260709 12:20:47.907133  1799 raft_consensus.cc:493] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:47.907486  1799 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:47.913756  1799 raft_consensus.cc:515] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:47.916378  1799 leader_election.cc:290] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 4 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:47.917514  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:47.917657  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9c09f11fdd44984937a807c10c5d5d3" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:47.918392  1652 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:47.918509  1723 raft_consensus.cc:3055] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:47.918876  1723 raft_consensus.cc:740] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:47.919813  1723 consensus_queue.cc:260] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:47.920984  1723 raft_consensus.cc:3060] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:47.924584  1652 raft_consensus.cc:2468] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 4.
I20260709 12:20:47.926004  1767 leader_election.cc:304] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:47.926887  2068 raft_consensus.cc:2804] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:47.927266  1723 raft_consensus.cc:2468] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 4.
I20260709 12:20:47.928248  2068 raft_consensus.cc:697] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 LEADER]: Becoming Leader. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:47.929458  2068 consensus_queue.cc:237] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:47.935510  2058 raft_consensus.cc:993] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3: : Instructing follower b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:47.935945  2069 raft_consensus.cc:1081] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Signalling peer b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:47.937701  1652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:52556
I20260709 12:20:47.938235  1652 raft_consensus.cc:493] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:47.938553  1652 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:47.938066  1545 catalog_manager.cc:5697] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c reported cstate change: term changed from 3 to 4, leader changed from 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66) to d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67). New cstate: current_term: 4 leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:47.944849  1652 raft_consensus.cc:515] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:47.947434  1652 leader_election.cc:290] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 4 election: Requested vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:47.948724  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:47.948760  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3cba7d3b0e14ea68eba21b6298dbfa7" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:47.949492  1723 raft_consensus.cc:3055] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:47.949841  1799 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:47.950008  1723 raft_consensus.cc:740] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:47.951015  1723 consensus_queue.cc:260] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:47.952178  1723 raft_consensus.cc:3060] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:47.957160  1799 raft_consensus.cc:2468] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 4.
I20260709 12:20:47.958923  1617 leader_election.cc:304] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:47.960950  1723 raft_consensus.cc:2468] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 4.
I20260709 12:20:47.961176  2076 raft_consensus.cc:2804] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:47.962790  2076 raft_consensus.cc:697] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 4 LEADER]: Becoming Leader. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:47.963721  2076 consensus_queue.cc:237] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:47.971002  1545 catalog_manager.cc:5697] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 reported cstate change: term changed from 3 to 4, leader changed from 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66) to b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65). New cstate: current_term: 4 leader_uuid: "b074c91c6ea54bea8aae038429fc5a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.102960  2058 raft_consensus.cc:993] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3: : Instructing follower b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:48.103557  2069 raft_consensus.cc:1081] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 4 LEADER]: Signalling peer b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:48.105394  1652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:52556
I20260709 12:20:48.105937  1652 raft_consensus.cc:493] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:48.106245  1652 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:48.107441  2069 raft_consensus.cc:993] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3: : Instructing follower d6c3f73078c047c7a7b8a53cd63d674c to start an election
I20260709 12:20:48.107959  2058 raft_consensus.cc:1081] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Signalling peer d6c3f73078c047c7a7b8a53cd63d674c to start an election
I20260709 12:20:48.109705  1799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605"
dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
 from {username='slave'} at 127.0.0.1:53210
I20260709 12:20:48.110299  1799 raft_consensus.cc:493] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:48.110682  1799 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:48.112886  1652 raft_consensus.cc:515] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:48.114709  1652 leader_election.cc:290] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 5 election: Requested vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:48.115732  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:48.115957  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d944dcaa2834f528cbe940a7cf041c9" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:48.116369  1723 raft_consensus.cc:3055] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:20:48.116537  1798 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:48.116757  1723 raft_consensus.cc:740] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:48.117134  1799 raft_consensus.cc:515] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:48.117761  1723 consensus_queue.cc:260] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:48.119115  1723 raft_consensus.cc:3060] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:48.119410  1799 leader_election.cc:290] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 4 election: Requested vote from peers b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227), 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587)
I20260709 12:20:48.122007  1798 raft_consensus.cc:2468] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 5.
I20260709 12:20:48.123155  1617 leader_election.cc:304] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:48.123725  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
I20260709 12:20:48.124300  2076 raft_consensus.cc:2804] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:20:48.124382  1652 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:48.124675  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ededca7aa44f16b1e09004b3c06605" candidate_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:48.125285  1724 raft_consensus.cc:3055] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:48.125487  1723 raft_consensus.cc:2468] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 5.
I20260709 12:20:48.125619  1724 raft_consensus.cc:740] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:48.125819  2076 raft_consensus.cc:697] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 5 LEADER]: Becoming Leader. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:48.126581  1724 consensus_queue.cc:260] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:48.127967  1724 raft_consensus.cc:3060] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:48.127681  2076 consensus_queue.cc:237] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:48.131114  1652 raft_consensus.cc:2468] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 4.
I20260709 12:20:48.133831  1767 leader_election.cc:304] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:48.134568  2068 raft_consensus.cc:2804] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:48.137018  2068 raft_consensus.cc:697] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 LEADER]: Becoming Leader. State: Replica: d6c3f73078c047c7a7b8a53cd63d674c, State: Running, Role: LEADER
I20260709 12:20:48.138257  2068 consensus_queue.cc:237] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:48.139299  1724 raft_consensus.cc:2468] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c3f73078c047c7a7b8a53cd63d674c in term 4.
I20260709 12:20:48.142491  1545 catalog_manager.cc:5697] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 reported cstate change: term changed from 4 to 5, leader changed from 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66) to b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65). New cstate: current_term: 5 leader_uuid: "b074c91c6ea54bea8aae038429fc5a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.147091  1547 catalog_manager.cc:5697] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c reported cstate change: term changed from 3 to 4, leader changed from 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66) to d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67). New cstate: current_term: 4 leader_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:48.212709  2069 raft_consensus.cc:993] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3: : Instructing follower b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:48.213200  2058 raft_consensus.cc:1081] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 2 LEADER]: Signalling peer b074c91c6ea54bea8aae038429fc5a38 to start an election
I20260709 12:20:48.214676  1652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e"
dest_uuid: "b074c91c6ea54bea8aae038429fc5a38"
 from {username='slave'} at 127.0.0.1:52556
I20260709 12:20:48.215121  1652 raft_consensus.cc:493] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:48.215360  1652 raft_consensus.cc:3060] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.220171  1652 raft_consensus.cc:515] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:48.222198  1652 leader_election.cc:290] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 election: Requested vote from peers 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587), d6c3f73078c047c7a7b8a53cd63d674c (127.1.123.67:40083)
I20260709 12:20:48.223069  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "265a51ff1cc7461e82200790ac4907c3"
I20260709 12:20:48.223372  1798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1842932f5ba24f0ca6c62e1acfeb017e" candidate_uuid: "b074c91c6ea54bea8aae038429fc5a38" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d6c3f73078c047c7a7b8a53cd63d674c"
I20260709 12:20:48.223644  1724 raft_consensus.cc:3055] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:48.223968  1798 raft_consensus.cc:3060] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.224009  1724 raft_consensus.cc:740] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 265a51ff1cc7461e82200790ac4907c3, State: Running, Role: LEADER
I20260709 12:20:48.224867  1724 consensus_queue.cc:260] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:48.225944  1724 raft_consensus.cc:3060] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.228929  1798 raft_consensus.cc:2468] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 3.
I20260709 12:20:48.229947  1617 leader_election.cc:304] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b074c91c6ea54bea8aae038429fc5a38, d6c3f73078c047c7a7b8a53cd63d674c; no voters: 
I20260709 12:20:48.230700  2076 raft_consensus.cc:2804] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:48.231208  2076 raft_consensus.cc:697] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Becoming Leader. State: Replica: b074c91c6ea54bea8aae038429fc5a38, State: Running, Role: LEADER
I20260709 12:20:48.231302  1724 raft_consensus.cc:2468] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b074c91c6ea54bea8aae038429fc5a38 in term 3.
I20260709 12:20:48.232221  2076 consensus_queue.cc:237] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } }
I20260709 12:20:48.240437  1545 catalog_manager.cc:5697] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 reported cstate change: term changed from 2 to 3, leader changed from 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66) to b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "b074c91c6ea54bea8aae038429fc5a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.446652  1798 raft_consensus.cc:1275] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:48.448114  2076 consensus_queue.cc:1048] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:48.460330  1724 raft_consensus.cc:1275] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:48.463124  1723 raft_consensus.cc:1275] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:48.464638  2068 consensus_queue.cc:1048] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:48.464046  2077 consensus_queue.cc:1048] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:20:48.477774  1652 raft_consensus.cc:1275] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:48.481205  2070 consensus_queue.cc:1048] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:48.521785  1724 raft_consensus.cc:1275] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 5 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:20:48.523183  2077 consensus_queue.cc:1048] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:48.533402  1799 raft_consensus.cc:1275] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 5 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:20:48.535733  2091 consensus_queue.cc:1048] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:48.646380  1724 raft_consensus.cc:1275] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:48.647723  2070 consensus_queue.cc:1048] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:48.660218  1652 raft_consensus.cc:1275] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Refusing update from remote peer d6c3f73078c047c7a7b8a53cd63d674c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:48.661703  2070 consensus_queue.cc:1048] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b074c91c6ea54bea8aae038429fc5a38" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:48.685320  1799 raft_consensus.cc:1275] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:48.686707  2091 consensus_queue.cc:1048] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c3f73078c047c7a7b8a53cd63d674c" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:48.697153  1724 raft_consensus.cc:1275] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Refusing update from remote peer b074c91c6ea54bea8aae038429fc5a38: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:48.698679  2091 consensus_queue.cc:1048] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265a51ff1cc7461e82200790ac4907c3" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:48.833339  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:48.838505  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:48.838997  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:49.840236  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:49.846221  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:49.846746  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:50.847883  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:50.853820  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:50.854326  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:51.855559  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:51.861932  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:51.862448  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:52.863765  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:52.869309  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:52.869807  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:53.871160  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:53.877807  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:53.878264  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:54.879436  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:54.884727  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:54.885255  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:55.886543  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:55.891872  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:55.892421  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:56.894109  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:56.901439  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:56.902055  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:57.933900  1517 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:20:57.934628  1517 tablet_server.cc:179] TabletServer@127.1.123.65:0 shutting down...
W20260709 12:20:57.971983  1767 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:57.975103  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:57.976502  1517 tablet_replica.cc:333] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38: stopping tablet replica
W20260709 12:20:57.977942  1767 consensus_peers.cc:597] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c -> Peer b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227): Couldn't send request to peer b074c91c6ea54bea8aae038429fc5a38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.977942  1517 raft_consensus.cc:2243] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.979537  1517 raft_consensus.cc:2272] T 1842932f5ba24f0ca6c62e1acfeb017e P b074c91c6ea54bea8aae038429fc5a38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.983498  1517 tablet_replica.cc:333] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38: stopping tablet replica
I20260709 12:20:57.984377  1517 raft_consensus.cc:2243] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.985705  1517 raft_consensus.cc:2272] T a3cba7d3b0e14ea68eba21b6298dbfa7 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.988677  1517 tablet_replica.cc:333] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38: stopping tablet replica
W20260709 12:20:57.988979  1693 consensus_peers.cc:597] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 -> Peer b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227): Couldn't send request to peer b074c91c6ea54bea8aae038429fc5a38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.989607  1517 raft_consensus.cc:2243] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.990180  1517 raft_consensus.cc:2272] T 677bc6e8dade496dad5352691517e43f P b074c91c6ea54bea8aae038429fc5a38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.992038  1517 tablet_replica.cc:333] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38: stopping tablet replica
I20260709 12:20:57.992558  1517 raft_consensus.cc:2243] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.992947  1517 raft_consensus.cc:2272] T c31f514965854ca89d1b161aa8e5c802 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.995076  1517 tablet_replica.cc:333] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38: stopping tablet replica
I20260709 12:20:57.995569  1517 raft_consensus.cc:2243] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.996357  1517 raft_consensus.cc:2272] T 8d944dcaa2834f528cbe940a7cf041c9 P b074c91c6ea54bea8aae038429fc5a38 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.998206  1517 tablet_replica.cc:333] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38: stopping tablet replica
I20260709 12:20:57.998724  1517 raft_consensus.cc:2243] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.999127  1517 raft_consensus.cc:2272] T 7a99c2cf2385417a86196fc38ccfbab0 P b074c91c6ea54bea8aae038429fc5a38 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.001041  1517 tablet_replica.cc:333] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38: stopping tablet replica
I20260709 12:20:58.001891  1517 raft_consensus.cc:2243] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.002424  1517 raft_consensus.cc:2272] T c9c09f11fdd44984937a807c10c5d5d3 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.004349  1517 tablet_replica.cc:333] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38: stopping tablet replica
I20260709 12:20:58.004808  1517 raft_consensus.cc:2243] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.005249  1517 raft_consensus.cc:2272] T b0ededca7aa44f16b1e09004b3c06605 P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.007086  1517 tablet_replica.cc:333] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38: stopping tablet replica
I20260709 12:20:58.007503  1517 raft_consensus.cc:2243] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.007921  1517 raft_consensus.cc:2272] T 012b99b449bc4f02a31b9c18fa87475a P b074c91c6ea54bea8aae038429fc5a38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.117336  1517 tablet_server.cc:196] TabletServer@127.1.123.65:0 shutdown complete.
W20260709 12:20:58.120591  1693 consensus_peers.cc:597] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 -> Peer b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227): Couldn't send request to peer b074c91c6ea54bea8aae038429fc5a38. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:46227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:58.121622  1767 consensus_peers.cc:597] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c -> Peer b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227): Couldn't send request to peer b074c91c6ea54bea8aae038429fc5a38. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:46227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:58.140422  1693 consensus_peers.cc:597] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 -> Peer b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227): Couldn't send request to peer b074c91c6ea54bea8aae038429fc5a38. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:46227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.148375  1517 tablet_server.cc:179] TabletServer@127.1.123.66:0 shutting down...
I20260709 12:20:58.180366  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:58.181294  1517 tablet_replica.cc:333] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3: stopping tablet replica
I20260709 12:20:58.182183  1517 raft_consensus.cc:2243] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.182840  1517 raft_consensus.cc:2272] T 1842932f5ba24f0ca6c62e1acfeb017e P 265a51ff1cc7461e82200790ac4907c3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.185334  1517 tablet_replica.cc:333] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3: stopping tablet replica
I20260709 12:20:58.186019  1517 raft_consensus.cc:2243] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.186548  1517 raft_consensus.cc:2272] T 677bc6e8dade496dad5352691517e43f P 265a51ff1cc7461e82200790ac4907c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.188432  1517 tablet_replica.cc:333] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3: stopping tablet replica
I20260709 12:20:58.189013  1517 raft_consensus.cc:2243] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.189980  1517 raft_consensus.cc:2272] T c31f514965854ca89d1b161aa8e5c802 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.191915  1517 tablet_replica.cc:333] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3: stopping tablet replica
I20260709 12:20:58.192512  1517 raft_consensus.cc:2243] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.192991  1517 raft_consensus.cc:2272] T a3cba7d3b0e14ea68eba21b6298dbfa7 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.195003  1517 tablet_replica.cc:333] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3: stopping tablet replica
I20260709 12:20:58.195544  1517 raft_consensus.cc:2243] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.196422  1517 raft_consensus.cc:2272] T 7a99c2cf2385417a86196fc38ccfbab0 P 265a51ff1cc7461e82200790ac4907c3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.198441  1517 tablet_replica.cc:333] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3: stopping tablet replica
I20260709 12:20:58.199002  1517 raft_consensus.cc:2243] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.199477  1517 raft_consensus.cc:2272] T 8d944dcaa2834f528cbe940a7cf041c9 P 265a51ff1cc7461e82200790ac4907c3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.201571  1517 tablet_replica.cc:333] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3: stopping tablet replica
I20260709 12:20:58.202111  1517 raft_consensus.cc:2243] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.202592  1517 raft_consensus.cc:2272] T c9c09f11fdd44984937a807c10c5d5d3 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.204572  1517 tablet_replica.cc:333] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3: stopping tablet replica
I20260709 12:20:58.205061  1517 raft_consensus.cc:2243] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.205591  1517 raft_consensus.cc:2272] T b0ededca7aa44f16b1e09004b3c06605 P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.207691  1517 tablet_replica.cc:333] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3: stopping tablet replica
I20260709 12:20:58.208312  1517 raft_consensus.cc:2243] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.209360  1517 raft_consensus.cc:2272] T 012b99b449bc4f02a31b9c18fa87475a P 265a51ff1cc7461e82200790ac4907c3 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:58.210362  1767 consensus_peers.cc:597] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c -> Peer 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587): Couldn't send request to peer 265a51ff1cc7461e82200790ac4907c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:58.230369  1767 consensus_peers.cc:597] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c -> Peer 265a51ff1cc7461e82200790ac4907c3 (127.1.123.66:38587): Couldn't send request to peer 265a51ff1cc7461e82200790ac4907c3. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:38587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.257416  1517 tablet_server.cc:196] TabletServer@127.1.123.66:0 shutdown complete.
I20260709 12:20:58.279886  1517 tablet_server.cc:179] TabletServer@127.1.123.67:0 shutting down...
W20260709 12:20:58.287474  1767 consensus_peers.cc:597] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c -> Peer b074c91c6ea54bea8aae038429fc5a38 (127.1.123.65:46227): Couldn't send request to peer b074c91c6ea54bea8aae038429fc5a38. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:46227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.318720  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:58.319567  1517 tablet_replica.cc:333] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c: stopping tablet replica
I20260709 12:20:58.320600  1517 raft_consensus.cc:2243] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.321329  1517 raft_consensus.cc:2272] T 1842932f5ba24f0ca6c62e1acfeb017e P d6c3f73078c047c7a7b8a53cd63d674c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.323474  1517 tablet_replica.cc:333] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c: stopping tablet replica
I20260709 12:20:58.324023  1517 raft_consensus.cc:2243] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.324487  1517 raft_consensus.cc:2272] T 8d944dcaa2834f528cbe940a7cf041c9 P d6c3f73078c047c7a7b8a53cd63d674c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.326306  1517 tablet_replica.cc:333] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c: stopping tablet replica
I20260709 12:20:58.326857  1517 raft_consensus.cc:2243] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.327301  1517 raft_consensus.cc:2272] T 7a99c2cf2385417a86196fc38ccfbab0 P d6c3f73078c047c7a7b8a53cd63d674c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.329088  1517 tablet_replica.cc:333] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c: stopping tablet replica
I20260709 12:20:58.329687  1517 raft_consensus.cc:2243] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.330125  1517 raft_consensus.cc:2272] T a3cba7d3b0e14ea68eba21b6298dbfa7 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.331797  1517 tablet_replica.cc:333] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c: stopping tablet replica
I20260709 12:20:58.332304  1517 raft_consensus.cc:2243] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.333156  1517 raft_consensus.cc:2272] T 677bc6e8dade496dad5352691517e43f P d6c3f73078c047c7a7b8a53cd63d674c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.334867  1517 tablet_replica.cc:333] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c: stopping tablet replica
I20260709 12:20:58.335371  1517 raft_consensus.cc:2243] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.335778  1517 raft_consensus.cc:2272] T c31f514965854ca89d1b161aa8e5c802 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.337625  1517 tablet_replica.cc:333] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c: stopping tablet replica
I20260709 12:20:58.338128  1517 raft_consensus.cc:2243] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.338946  1517 raft_consensus.cc:2272] T c9c09f11fdd44984937a807c10c5d5d3 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.340651  1517 tablet_replica.cc:333] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c: stopping tablet replica
I20260709 12:20:58.341140  1517 raft_consensus.cc:2243] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.342005  1517 raft_consensus.cc:2272] T b0ededca7aa44f16b1e09004b3c06605 P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.343871  1517 tablet_replica.cc:333] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c: stopping tablet replica
I20260709 12:20:58.344321  1517 raft_consensus.cc:2243] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.344755  1517 raft_consensus.cc:2272] T 012b99b449bc4f02a31b9c18fa87475a P d6c3f73078c047c7a7b8a53cd63d674c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.392946  1517 tablet_server.cc:196] TabletServer@127.1.123.67:0 shutdown complete.
I20260709 12:20:58.415730  1517 master.cc:562] Master@127.1.123.126:32847 shutting down...
I20260709 12:20:58.642621  1517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.643450  1517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.643982  1517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6190226b862b4c9abdc05e358228ec56: stopping tablet replica
I20260709 12:20:58.666182  1517 master.cc:584] Master@127.1.123.126:32847 shutdown complete.
265a51ff1cc7461e82200790ac4907c3  4
b074c91c6ea54bea8aae038429fc5a38  4
d6c3f73078c047c7a7b8a53cd63d674c  1
265a51ff1cc7461e82200790ac4907c3  3
b074c91c6ea54bea8aae038429fc5a38  3
d6c3f73078c047c7a7b8a53cd63d674c  3
265a51ff1cc7461e82200790ac4907c3  8
b074c91c6ea54bea8aae038429fc5a38  0
d6c3f73078c047c7a7b8a53cd63d674c  1
265a51ff1cc7461e82200790ac4907c3  3
b074c91c6ea54bea8aae038429fc5a38  3
d6c3f73078c047c7a7b8a53cd63d674c  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (32095 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:20:58.715483  1517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.123.126:34823
I20260709 12:20:58.716908  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:58.723363  2114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:58.723593  2115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:58.724949  2117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:58.725778  1517 server_base.cc:1061] running on GCE node
I20260709 12:20:58.726840  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:58.727043  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:58.727164  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599658727152 us; error 0 us; skew 500 ppm
I20260709 12:20:58.727659  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:58.730427  1517 webserver.cc:533] Webserver started at http://127.1.123.126:42007/ using document root <none> and password file <none>
I20260709 12:20:58.730928  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:58.731091  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:58.731366  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:58.732657  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/master-0-root/instance:
uuid: "c20511a19e7245f3b8d4440a3dc7e335"
format_stamp: "Formatted at 2026-07-09 12:20:58 on dist-test-slave-08l4"
I20260709 12:20:58.737646  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:20:58.741444  2122 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:58.742492  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:20:58.742856  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/master-0-root
uuid: "c20511a19e7245f3b8d4440a3dc7e335"
format_stamp: "Formatted at 2026-07-09 12:20:58 on dist-test-slave-08l4"
I20260709 12:20:58.743185  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:58.755196  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:58.756727  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:58.801427  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.126:34823
I20260709 12:20:58.801617  2173 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.126:34823 every 8 connection(s)
I20260709 12:20:58.806074  2174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:58.817339  2174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335: Bootstrap starting.
I20260709 12:20:58.823239  2174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:58.828224  2174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335: No bootstrap required, opened a new log
I20260709 12:20:58.830806  2174 raft_consensus.cc:359] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20511a19e7245f3b8d4440a3dc7e335" member_type: VOTER }
I20260709 12:20:58.831307  2174 raft_consensus.cc:385] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:58.831514  2174 raft_consensus.cc:740] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20511a19e7245f3b8d4440a3dc7e335, State: Initialized, Role: FOLLOWER
I20260709 12:20:58.832068  2174 consensus_queue.cc:260] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20511a19e7245f3b8d4440a3dc7e335" member_type: VOTER }
I20260709 12:20:58.832536  2174 raft_consensus.cc:399] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:20:58.832746  2174 raft_consensus.cc:493] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:20:58.832967  2174 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:58.837615  2174 raft_consensus.cc:515] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20511a19e7245f3b8d4440a3dc7e335" member_type: VOTER }
I20260709 12:20:58.838279  2174 leader_election.cc:304] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c20511a19e7245f3b8d4440a3dc7e335; no voters: 
I20260709 12:20:58.839605  2174 leader_election.cc:290] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:20:58.840107  2177 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:58.841774  2177 raft_consensus.cc:697] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [term 1 LEADER]: Becoming Leader. State: Replica: c20511a19e7245f3b8d4440a3dc7e335, State: Running, Role: LEADER
I20260709 12:20:58.842614  2177 consensus_queue.cc:237] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20511a19e7245f3b8d4440a3dc7e335" member_type: VOTER }
I20260709 12:20:58.843008  2174 sys_catalog.cc:565] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:20:58.847936  2179 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c20511a19e7245f3b8d4440a3dc7e335. Latest consensus state: current_term: 1 leader_uuid: "c20511a19e7245f3b8d4440a3dc7e335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20511a19e7245f3b8d4440a3dc7e335" member_type: VOTER } }
I20260709 12:20:58.847936  2178 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c20511a19e7245f3b8d4440a3dc7e335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20511a19e7245f3b8d4440a3dc7e335" member_type: VOTER } }
I20260709 12:20:58.848745  2179 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:58.848820  2178 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:58.853824  2184 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:20:58.860549  2184 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:20:58.861699  1517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:20:58.870786  2184 catalog_manager.cc:1383] Generated new cluster ID: d99500cbba6a46659e47b0dc2c4c08ce
I20260709 12:20:58.871160  2184 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:20:58.885473  2184 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:20:58.887049  2184 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:20:58.908964  2184 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335: Generated new TSK 0
I20260709 12:20:58.909714  2184 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:20:58.928752  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:58.936770  2196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:58.936030  2195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:58.938577  2198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:58.940245  1517 server_base.cc:1061] running on GCE node
I20260709 12:20:58.941426  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:58.941752  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:58.941922  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599658941909 us; error 0 us; skew 500 ppm
I20260709 12:20:58.942548  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:58.945456  1517 webserver.cc:533] Webserver started at http://127.1.123.65:38287/ using document root <none> and password file <none>
I20260709 12:20:58.946105  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:58.946302  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:58.946650  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:58.947937  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-0-root/instance:
uuid: "efe356271a3d43b7867710d1e5962dff"
format_stamp: "Formatted at 2026-07-09 12:20:58 on dist-test-slave-08l4"
I20260709 12:20:58.952785  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:20:58.956481  2203 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:58.957299  1517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:20:58.957648  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-0-root
uuid: "efe356271a3d43b7867710d1e5962dff"
format_stamp: "Formatted at 2026-07-09 12:20:58 on dist-test-slave-08l4"
I20260709 12:20:58.957967  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:58.981894  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:58.983425  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:58.985143  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:58.987979  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:58.988193  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:58.988528  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:58.988705  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.036262  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.65:40239
I20260709 12:20:59.036370  2265 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.65:40239 every 8 connection(s)
I20260709 12:20:59.041963  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:59.050186  2270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:59.052657  2271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:59.056443  2273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.056494  1517 server_base.cc:1061] running on GCE node
I20260709 12:20:59.056918  2266 heartbeater.cc:344] Connected to a master server at 127.1.123.126:34823
I20260709 12:20:59.057399  2266 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:59.058130  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:20:59.058233  2266 heartbeater.cc:507] Master 127.1.123.126:34823 requested a full tablet report, sending...
W20260709 12:20:59.058378  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:59.058634  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599659058610 us; error 0 us; skew 500 ppm
I20260709 12:20:59.059311  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:59.060926  2139 ts_manager.cc:194] Registered new tserver with Master: efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239)
I20260709 12:20:59.062565  1517 webserver.cc:533] Webserver started at http://127.1.123.66:42159/ using document root <none> and password file <none>
I20260709 12:20:59.063243  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:59.063473  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:59.063634  2139 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45702
I20260709 12:20:59.063822  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:59.065578  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-1-root/instance:
uuid: "915a285f0ece44d0b4aa1d695a3a408c"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-08l4"
I20260709 12:20:59.070453  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:20:59.073972  2278 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.074925  1517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:20:59.075184  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-1-root
uuid: "915a285f0ece44d0b4aa1d695a3a408c"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-08l4"
I20260709 12:20:59.075435  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:59.096755  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:59.098196  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:59.099761  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:59.102757  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:59.102969  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.103309  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:59.103498  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.150823  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.66:39587
I20260709 12:20:59.150935  2340 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.66:39587 every 8 connection(s)
I20260709 12:20:59.156064  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:59.164517  2344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:59.166244  2345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.168357  1517 server_base.cc:1061] running on GCE node
I20260709 12:20:59.170671  2341 heartbeater.cc:344] Connected to a master server at 127.1.123.126:34823
W20260709 12:20:59.170802  2347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.171351  2341 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:59.172107  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:59.172380  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:59.172287  2341 heartbeater.cc:507] Master 127.1.123.126:34823 requested a full tablet report, sending...
I20260709 12:20:59.172696  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599659172680 us; error 0 us; skew 500 ppm
I20260709 12:20:59.173424  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:59.175071  2139 ts_manager.cc:194] Registered new tserver with Master: 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:20:59.176409  1517 webserver.cc:533] Webserver started at http://127.1.123.67:36635/ using document root <none> and password file <none>
I20260709 12:20:59.177100  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:59.177124  2139 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45706
I20260709 12:20:59.177472  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:59.177866  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:59.179548  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-2-root/instance:
uuid: "8837957e7c2c4384bdc425d1f86deb95"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-08l4"
I20260709 12:20:59.184613  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:20:59.188422  2352 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.189283  1517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:20:59.189646  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-2-root
uuid: "8837957e7c2c4384bdc425d1f86deb95"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-08l4"
I20260709 12:20:59.189957  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599626433458-1517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:59.201264  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:59.202694  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:59.204523  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:59.208480  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:59.208796  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.209129  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:59.209306  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.256552  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.67:39493
I20260709 12:20:59.256670  2414 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.67:39493 every 8 connection(s)
I20260709 12:20:59.272869  2415 heartbeater.cc:344] Connected to a master server at 127.1.123.126:34823
I20260709 12:20:59.273310  2415 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:59.274175  2415 heartbeater.cc:507] Master 127.1.123.126:34823 requested a full tablet report, sending...
I20260709 12:20:59.276521  2139 ts_manager.cc:194] Registered new tserver with Master: 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:20:59.277587  1517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017282982s
I20260709 12:20:59.278190  2139 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45710
I20260709 12:20:59.305123  2139 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45722:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:20:59.308391  2139 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:20:59.384757  2231 tablet_service.cc:1511] Processing CreateTablet for tablet ad2032208e6a41d6bfddab63d9e28eea (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:59.395231  2231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2032208e6a41d6bfddab63d9e28eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.396939  2380 tablet_service.cc:1511] Processing CreateTablet for tablet ad2032208e6a41d6bfddab63d9e28eea (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:59.398644  2380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2032208e6a41d6bfddab63d9e28eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.386705  2230 tablet_service.cc:1511] Processing CreateTablet for tablet 2a3380e528b3434caefdaf08b8a80a04 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:59.402519  2230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a3380e528b3434caefdaf08b8a80a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.403733  2379 tablet_service.cc:1511] Processing CreateTablet for tablet 2a3380e528b3434caefdaf08b8a80a04 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:59.405296  2379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a3380e528b3434caefdaf08b8a80a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.388682  2229 tablet_service.cc:1511] Processing CreateTablet for tablet a41d9df904094b35b6d61a167f8359d2 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:59.406453  2229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a41d9df904094b35b6d61a167f8359d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.391609  2228 tablet_service.cc:1511] Processing CreateTablet for tablet e118c00462e141328b65f49b71fc2b09 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:59.410557  2228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e118c00462e141328b65f49b71fc2b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.393527  2227 tablet_service.cc:1511] Processing CreateTablet for tablet 14763d4c61bd457faccb12c0376a7282 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:59.412895  2378 tablet_service.cc:1511] Processing CreateTablet for tablet a41d9df904094b35b6d61a167f8359d2 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:59.414471  2227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14763d4c61bd457faccb12c0376a7282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.414736  2378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a41d9df904094b35b6d61a167f8359d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.419647  2226 tablet_service.cc:1511] Processing CreateTablet for tablet b3f751eb23354b4ebadb0dec235106f2 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:59.421334  2226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3f751eb23354b4ebadb0dec235106f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.421111  2305 tablet_service.cc:1511] Processing CreateTablet for tablet 2a3380e528b3434caefdaf08b8a80a04 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:59.424556  2304 tablet_service.cc:1511] Processing CreateTablet for tablet a41d9df904094b35b6d61a167f8359d2 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:59.424719  2375 tablet_service.cc:1511] Processing CreateTablet for tablet b3f751eb23354b4ebadb0dec235106f2 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:59.425441  2305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a3380e528b3434caefdaf08b8a80a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.426594  2375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3f751eb23354b4ebadb0dec235106f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.427393  2304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a41d9df904094b35b6d61a167f8359d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.419497  2377 tablet_service.cc:1511] Processing CreateTablet for tablet e118c00462e141328b65f49b71fc2b09 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:59.428908  2377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e118c00462e141328b65f49b71fc2b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.422993  2306 tablet_service.cc:1511] Processing CreateTablet for tablet ad2032208e6a41d6bfddab63d9e28eea (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:59.430773  2306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2032208e6a41d6bfddab63d9e28eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.422149  2376 tablet_service.cc:1511] Processing CreateTablet for tablet 14763d4c61bd457faccb12c0376a7282 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:59.433089  2376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14763d4c61bd457faccb12c0376a7282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.434193  2303 tablet_service.cc:1511] Processing CreateTablet for tablet e118c00462e141328b65f49b71fc2b09 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:59.435796  2303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e118c00462e141328b65f49b71fc2b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.439715  2373 tablet_service.cc:1511] Processing CreateTablet for tablet 747f34920d614b61852845d2ab96deb3 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:59.441301  2373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 747f34920d614b61852845d2ab96deb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.436995  2374 tablet_service.cc:1511] Processing CreateTablet for tablet c1a4c73011a7449bb1e0852f1e3056d1 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:59.445899  2374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a4c73011a7449bb1e0852f1e3056d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.446734  2302 tablet_service.cc:1511] Processing CreateTablet for tablet 14763d4c61bd457faccb12c0376a7282 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:59.448300  2302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14763d4c61bd457faccb12c0376a7282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.461234  2225 tablet_service.cc:1511] Processing CreateTablet for tablet c1a4c73011a7449bb1e0852f1e3056d1 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:59.461644  2223 tablet_service.cc:1511] Processing CreateTablet for tablet 2c237ebe3cce4553aa71f1dabf1abbea (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:59.461298  2301 tablet_service.cc:1511] Processing CreateTablet for tablet b3f751eb23354b4ebadb0dec235106f2 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:59.466621  2301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3f751eb23354b4ebadb0dec235106f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.463290  2372 tablet_service.cc:1511] Processing CreateTablet for tablet 2c237ebe3cce4553aa71f1dabf1abbea (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:59.464938  2224 tablet_service.cc:1511] Processing CreateTablet for tablet 747f34920d614b61852845d2ab96deb3 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:59.469731  2299 tablet_service.cc:1511] Processing CreateTablet for tablet 747f34920d614b61852845d2ab96deb3 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:59.463302  2300 tablet_service.cc:1511] Processing CreateTablet for tablet c1a4c73011a7449bb1e0852f1e3056d1 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:59.470294  2298 tablet_service.cc:1511] Processing CreateTablet for tablet 2c237ebe3cce4553aa71f1dabf1abbea (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:59.471165  2299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 747f34920d614b61852845d2ab96deb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.471634  2225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a4c73011a7449bb1e0852f1e3056d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.474329  2224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 747f34920d614b61852845d2ab96deb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.477766  2223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c237ebe3cce4553aa71f1dabf1abbea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.464955  2222 tablet_service.cc:1511] Processing CreateTablet for tablet f4200fe852224b1c8f21a0bb88a0fc47 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:59.479085  2222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4200fe852224b1c8f21a0bb88a0fc47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.480826  2371 tablet_service.cc:1511] Processing CreateTablet for tablet f4200fe852224b1c8f21a0bb88a0fc47 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:59.483081  2300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a4c73011a7449bb1e0852f1e3056d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.467702  2372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c237ebe3cce4553aa71f1dabf1abbea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.492871  2298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c237ebe3cce4553aa71f1dabf1abbea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.499233  2297 tablet_service.cc:1511] Processing CreateTablet for tablet f4200fe852224b1c8f21a0bb88a0fc47 (DEFAULT_TABLE table=test-workload [id=db5bdf9998734657945bdcf2d2bb3f55]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:59.500831  2297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4200fe852224b1c8f21a0bb88a0fc47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.514091  2371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4200fe852224b1c8f21a0bb88a0fc47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.583846  2430 tablet_bootstrap.cc:492] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95: Bootstrap starting.
I20260709 12:20:59.594478  2430 tablet_bootstrap.cc:654] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.606204  2429 tablet_bootstrap.cc:492] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c: Bootstrap starting.
I20260709 12:20:59.636723  2431 tablet_bootstrap.cc:492] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff: Bootstrap starting.
I20260709 12:20:59.639227  2430 tablet_bootstrap.cc:492] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95: No bootstrap required, opened a new log
I20260709 12:20:59.641561  2430 ts_tablet_manager.cc:1403] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.025s
I20260709 12:20:59.644079  2430 raft_consensus.cc:359] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:20:59.644975  2430 raft_consensus.cc:385] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.645434  2430 raft_consensus.cc:740] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.646303  2430 consensus_queue.cc:260] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:20:59.653386  2415 heartbeater.cc:499] Master 127.1.123.126:34823 was elected leader, sending a full tablet report...
I20260709 12:20:59.653946  2431 tablet_bootstrap.cc:654] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.654006  2430 ts_tablet_manager.cc:1434] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.003s
I20260709 12:20:59.656543  2430 tablet_bootstrap.cc:492] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95: Bootstrap starting.
I20260709 12:20:59.656513  2429 tablet_bootstrap.cc:654] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.666105  2434 raft_consensus.cc:493] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.666934  2434 raft_consensus.cc:515] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:20:59.676522  2434 leader_election.cc:290] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:20:59.677322  2430 tablet_bootstrap.cc:654] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.686744  2429 tablet_bootstrap.cc:492] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c: No bootstrap required, opened a new log
I20260709 12:20:59.687328  2429 ts_tablet_manager.cc:1403] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.024s
I20260709 12:20:59.688702  2430 tablet_bootstrap.cc:492] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95: No bootstrap required, opened a new log
I20260709 12:20:59.689251  2430 ts_tablet_manager.cc:1403] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.000s
I20260709 12:20:59.690753  2429 raft_consensus.cc:359] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.691545  2429 raft_consensus.cc:385] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.691839  2429 raft_consensus.cc:740] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.692535  2430 raft_consensus.cc:359] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.692775  2429 consensus_queue.cc:260] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.693322  2430 raft_consensus.cc:385] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.693663  2430 raft_consensus.cc:740] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.694733  2430 consensus_queue.cc:260] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.697903  2430 ts_tablet_manager.cc:1434] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:20:59.699277  2430 tablet_bootstrap.cc:492] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95: Bootstrap starting.
I20260709 12:20:59.706411  2241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:20:59.708549  2430 tablet_bootstrap.cc:654] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:59.708938  2354 leader_election.cc:343] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.715654  2431 tablet_bootstrap.cc:492] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff: No bootstrap required, opened a new log
I20260709 12:20:59.722698  2431 ts_tablet_manager.cc:1403] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff: Time spent bootstrapping tablet: real 0.086s	user 0.024s	sys 0.042s
I20260709 12:20:59.728552  2430 tablet_bootstrap.cc:492] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95: No bootstrap required, opened a new log
I20260709 12:20:59.729329  2430 ts_tablet_manager.cc:1403] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.013s
I20260709 12:20:59.729104  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c" is_pre_election: true
I20260709 12:20:59.728430  2431 raft_consensus.cc:359] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:20:59.730080  2431 raft_consensus.cc:385] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.730593  2431 raft_consensus.cc:740] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Initialized, Role: FOLLOWER
W20260709 12:20:59.731364  2353 leader_election.cc:343] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.731928  2353 leader_election.cc:304] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95; no voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff
I20260709 12:20:59.732004  2431 consensus_queue.cc:260] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:20:59.733081  2433 raft_consensus.cc:2749] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:59.732962  2430 raft_consensus.cc:359] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.733946  2430 raft_consensus.cc:385] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.734295  2430 raft_consensus.cc:740] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.735167  2430 consensus_queue.cc:260] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.737773  2430 ts_tablet_manager.cc:1434] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:20:59.739154  2430 tablet_bootstrap.cc:492] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95: Bootstrap starting.
I20260709 12:20:59.743850  2431 ts_tablet_manager.cc:1434] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff: Time spent starting tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 12:20:59.745210  2266 heartbeater.cc:499] Master 127.1.123.126:34823 was elected leader, sending a full tablet report...
I20260709 12:20:59.747380  2429 ts_tablet_manager.cc:1434] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent starting tablet: real 0.060s	user 0.022s	sys 0.032s
I20260709 12:20:59.748253  2341 heartbeater.cc:499] Master 127.1.123.126:34823 was elected leader, sending a full tablet report...
I20260709 12:20:59.750754  2429 tablet_bootstrap.cc:492] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c: Bootstrap starting.
I20260709 12:20:59.751379  2430 tablet_bootstrap.cc:654] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.758612  2429 tablet_bootstrap.cc:654] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.760061  2431 tablet_bootstrap.cc:492] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff: Bootstrap starting.
W20260709 12:20:59.763293  2416 tablet.cc:2406] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:20:59.767664  2430 tablet_bootstrap.cc:492] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95: No bootstrap required, opened a new log
I20260709 12:20:59.768245  2430 ts_tablet_manager.cc:1403] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.009s
I20260709 12:20:59.770222  2431 tablet_bootstrap.cc:654] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.771500  2430 raft_consensus.cc:359] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.772397  2430 raft_consensus.cc:385] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.772773  2430 raft_consensus.cc:740] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.774008  2430 consensus_queue.cc:260] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.776878  2429 tablet_bootstrap.cc:492] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c: No bootstrap required, opened a new log
I20260709 12:20:59.777215  2430 ts_tablet_manager.cc:1434] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.002s
I20260709 12:20:59.777689  2429 ts_tablet_manager.cc:1403] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.007s
I20260709 12:20:59.778522  2430 tablet_bootstrap.cc:492] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95: Bootstrap starting.
I20260709 12:20:59.782362  2429 raft_consensus.cc:359] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.783528  2429 raft_consensus.cc:385] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.783913  2429 raft_consensus.cc:740] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.785923  2429 consensus_queue.cc:260] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.787581  2430 tablet_bootstrap.cc:654] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.787902  2431 tablet_bootstrap.cc:492] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff: No bootstrap required, opened a new log
I20260709 12:20:59.788436  2431 ts_tablet_manager.cc:1403] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.008s
I20260709 12:20:59.788604  2429 ts_tablet_manager.cc:1434] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 12:20:59.789680  2429 tablet_bootstrap.cc:492] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c: Bootstrap starting.
I20260709 12:20:59.791770  2431 raft_consensus.cc:359] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.792558  2431 raft_consensus.cc:385] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.792867  2431 raft_consensus.cc:740] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.793838  2431 consensus_queue.cc:260] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.797379  2431 ts_tablet_manager.cc:1434] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.007s
I20260709 12:20:59.798480  2431 tablet_bootstrap.cc:492] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff: Bootstrap starting.
I20260709 12:20:59.798904  2429 tablet_bootstrap.cc:654] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.805102  2433 raft_consensus.cc:493] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.805748  2433 raft_consensus.cc:515] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.810262  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c" is_pre_election: true
I20260709 12:20:59.811633  2241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
W20260709 12:20:59.813158  2353 leader_election.cc:343] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.813807  2433 leader_election.cc:290] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
W20260709 12:20:59.813844  2354 leader_election.cc:343] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.814525  2354 leader_election.cc:304] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95; no voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff
I20260709 12:20:59.815673  2433 raft_consensus.cc:2749] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:59.817265  2431 tablet_bootstrap.cc:654] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.824810  2430 tablet_bootstrap.cc:492] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95: No bootstrap required, opened a new log
I20260709 12:20:59.825073  2429 tablet_bootstrap.cc:492] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c: No bootstrap required, opened a new log
I20260709 12:20:59.825382  2430 ts_tablet_manager.cc:1403] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95: Time spent bootstrapping tablet: real 0.047s	user 0.017s	sys 0.026s
I20260709 12:20:59.825655  2429 ts_tablet_manager.cc:1403] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.018s
I20260709 12:20:59.826253  2431 tablet_bootstrap.cc:492] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff: No bootstrap required, opened a new log
I20260709 12:20:59.826746  2431 ts_tablet_manager.cc:1403] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.000s
I20260709 12:20:59.828148  2430 raft_consensus.cc:359] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.828774  2430 raft_consensus.cc:385] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.829070  2430 raft_consensus.cc:740] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.828835  2429 raft_consensus.cc:359] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.829646  2429 raft_consensus.cc:385] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.829734  2431 raft_consensus.cc:359] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.830039  2429 raft_consensus.cc:740] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.829985  2430 consensus_queue.cc:260] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.830585  2431 raft_consensus.cc:385] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.830973  2431 raft_consensus.cc:740] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.831235  2429 consensus_queue.cc:260] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.831827  2431 consensus_queue.cc:260] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.832463  2430 ts_tablet_manager.cc:1434] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:20:59.833577  2430 tablet_bootstrap.cc:492] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95: Bootstrap starting.
I20260709 12:20:59.833877  2429 ts_tablet_manager.cc:1434] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:20:59.834405  2431 ts_tablet_manager.cc:1434] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:59.834959  2429 tablet_bootstrap.cc:492] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c: Bootstrap starting.
I20260709 12:20:59.835412  2431 tablet_bootstrap.cc:492] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff: Bootstrap starting.
I20260709 12:20:59.837852  2439 raft_consensus.cc:493] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.838376  2439 raft_consensus.cc:515] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.842197  2430 tablet_bootstrap.cc:654] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.842332  2429 tablet_bootstrap.cc:654] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.845664  2439 leader_election.cc:290] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
W20260709 12:20:59.856324  2187 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:59.861301  2431 tablet_bootstrap.cc:654] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.878304  2241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:20:59.879281  2241 raft_consensus.cc:2468] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 0.
I20260709 12:20:59.881009  2280 leader_election.cc:304] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:20:59.882359  2439 raft_consensus.cc:2804] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:59.882782  2439 raft_consensus.cc:493] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:59.883137  2439 raft_consensus.cc:3060] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.891125  2439 raft_consensus.cc:515] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.894768  2241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:20:59.895406  2241 raft_consensus.cc:3060] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.901360  2433 raft_consensus.cc:493] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.902021  2433 raft_consensus.cc:515] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.904708  2241 raft_consensus.cc:2468] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 1.
I20260709 12:20:59.910179  2439 leader_election.cc:290] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:20:59.910303  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a41d9df904094b35b6d61a167f8359d2" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c" is_pre_election: true
I20260709 12:20:59.910904  2439 raft_consensus.cc:493] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.911701  2434 raft_consensus.cc:493] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:59.912112  2353 leader_election.cc:343] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.912205  2434 raft_consensus.cc:515] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.914032  2430 tablet_bootstrap.cc:492] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95: No bootstrap required, opened a new log
I20260709 12:20:59.911383  2439 raft_consensus.cc:515] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.906352  2433 leader_election.cc:290] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:20:59.914598  2430 ts_tablet_manager.cc:1403] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95: Time spent bootstrapping tablet: real 0.081s	user 0.019s	sys 0.053s
I20260709 12:20:59.919085  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:20:59.919555  2429 tablet_bootstrap.cc:492] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c: No bootstrap required, opened a new log
I20260709 12:20:59.919801  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:20:59.920245  2430 raft_consensus.cc:359] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.920560  2388 raft_consensus.cc:2468] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 0.
I20260709 12:20:59.920629  2390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:20:59.920030  2429 ts_tablet_manager.cc:1403] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent bootstrapping tablet: real 0.085s	user 0.038s	sys 0.032s
I20260709 12:20:59.921947  2439 leader_election.cc:290] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
W20260709 12:20:59.922559  2279 leader_election.cc:343] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.923069  2279 leader_election.cc:304] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 8837957e7c2c4384bdc425d1f86deb95
I20260709 12:20:59.923269  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c" is_pre_election: true
I20260709 12:20:59.923974  2316 raft_consensus.cc:2468] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 0.
I20260709 12:20:59.923879  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:20:59.925563  2239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:20:59.926282  2353 leader_election.cc:304] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: 
I20260709 12:20:59.926569  2453 raft_consensus.cc:2804] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:59.927120  2453 raft_consensus.cc:697] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Leader. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:20:59.915844  2434 leader_election.cc:290] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
W20260709 12:20:59.927351  2280 leader_election.cc:343] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.925282  2429 raft_consensus.cc:359] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.928282  2429 raft_consensus.cc:385] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.928381  2241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a41d9df904094b35b6d61a167f8359d2" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
W20260709 12:20:59.928501  2354 leader_election.cc:343] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.928068  2453 consensus_queue.cc:237] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.927312  2433 raft_consensus.cc:2804] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:59.921052  2430 raft_consensus.cc:385] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.929167  2279 leader_election.cc:304] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: efe356271a3d43b7867710d1e5962dff
I20260709 12:20:59.930760  2433 raft_consensus.cc:493] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:59.931175  2354 leader_election.cc:343] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.931593  2433 raft_consensus.cc:3060] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.931768  2354 leader_election.cc:304] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95; no voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff
I20260709 12:20:59.932937  2431 tablet_bootstrap.cc:492] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff: No bootstrap required, opened a new log
I20260709 12:20:59.933043  2434 raft_consensus.cc:2749] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:59.928659  2429 raft_consensus.cc:740] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.931193  2430 raft_consensus.cc:740] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.933463  2431 ts_tablet_manager.cc:1403] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff: Time spent bootstrapping tablet: real 0.098s	user 0.007s	sys 0.007s
I20260709 12:20:59.934430  2429 consensus_queue.cc:260] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.934814  2430 consensus_queue.cc:260] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.937099  2429 ts_tablet_manager.cc:1434] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.005s
I20260709 12:20:59.937465  2431 raft_consensus.cc:359] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.938262  2431 raft_consensus.cc:385] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.938601  2431 raft_consensus.cc:740] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.938771  2429 tablet_bootstrap.cc:492] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c: Bootstrap starting.
I20260709 12:20:59.939690  2430 ts_tablet_manager.cc:1434] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95: Time spent starting tablet: real 0.023s	user 0.001s	sys 0.005s
I20260709 12:20:59.940815  2430 tablet_bootstrap.cc:492] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95: Bootstrap starting.
I20260709 12:20:59.941823  2431 consensus_queue.cc:260] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.947252  2429 tablet_bootstrap.cc:654] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.947819  2430 tablet_bootstrap.cc:654] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.949298  2439 raft_consensus.cc:2804] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:59.949774  2439 raft_consensus.cc:493] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:59.950153  2439 raft_consensus.cc:3060] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:59.954399  2279 leader_election.cc:343] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.958117  2441 raft_consensus.cc:493] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.958802  2441 raft_consensus.cc:515] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.961144  2430 tablet_bootstrap.cc:492] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95: No bootstrap required, opened a new log
I20260709 12:20:59.961294  2429 tablet_bootstrap.cc:492] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c: No bootstrap required, opened a new log
I20260709 12:20:59.961763  2430 ts_tablet_manager.cc:1403] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 12:20:59.962065  2429 ts_tablet_manager.cc:1403] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.013s
I20260709 12:20:59.964831  2439 raft_consensus.cc:515] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.965078  2441 leader_election.cc:290] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:20:59.965123  2430 raft_consensus.cc:359] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.966264  2429 raft_consensus.cc:359] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.968379  2430 raft_consensus.cc:385] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.968492  2429 raft_consensus.cc:385] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.968801  2430 raft_consensus.cc:740] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.969007  2429 raft_consensus.cc:740] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.967429  2433 raft_consensus.cc:515] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.970054  2441 raft_consensus.cc:493] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.970204  2429 consensus_queue.cc:260] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.970024  2430 consensus_queue.cc:260] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:20:59.970744  2441 raft_consensus.cc:515] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:20:59.973377  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:20:59.974370  2316 raft_consensus.cc:2393] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8837957e7c2c4384bdc425d1f86deb95 in current term 1: Already voted for candidate 915a285f0ece44d0b4aa1d695a3a408c in this term.
I20260709 12:20:59.973167  2441 leader_election.cc:290] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:20:59.974049  2241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:20:59.975719  2241 raft_consensus.cc:3060] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.976459  2430 ts_tablet_manager.cc:1434] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:20:59.977669  2430 tablet_bootstrap.cc:492] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95: Bootstrap starting.
I20260709 12:20:59.980207  2429 ts_tablet_manager.cc:1434] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.008s
I20260709 12:20:59.981374  2429 tablet_bootstrap.cc:492] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c: Bootstrap starting.
I20260709 12:20:59.982316  2433 leader_election.cc:290] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:20:59.984962  2430 tablet_bootstrap.cc:654] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.987025  2390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:20:59.987965  2390 raft_consensus.cc:2393] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 915a285f0ece44d0b4aa1d695a3a408c in current term 1: Already voted for candidate 8837957e7c2c4384bdc425d1f86deb95 in this term.
I20260709 12:20:59.988786  2429 tablet_bootstrap.cc:654] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.989853  2439 leader_election.cc:290] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:20:59.997157  2430 tablet_bootstrap.cc:492] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95: No bootstrap required, opened a new log
I20260709 12:20:59.997915  2430 ts_tablet_manager.cc:1403] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.013s
I20260709 12:20:59.997938  2431 ts_tablet_manager.cc:1434] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.000s
I20260709 12:20:59.997087  2241 raft_consensus.cc:2468] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 1.
I20260709 12:21:00.002486  2431 tablet_bootstrap.cc:492] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff: Bootstrap starting.
I20260709 12:21:00.001276  2430 raft_consensus.cc:359] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.003013  2430 raft_consensus.cc:385] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.003397  2430 raft_consensus.cc:740] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.004334  2430 consensus_queue.cc:260] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.004647  2280 leader_election.cc:304] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 8837957e7c2c4384bdc425d1f86deb95
I20260709 12:21:00.006157  2429 tablet_bootstrap.cc:492] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c: No bootstrap required, opened a new log
I20260709 12:21:00.006749  2430 ts_tablet_manager.cc:1434] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:21:00.006783  2429 ts_tablet_manager.cc:1403] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent bootstrapping tablet: real 0.026s	user 0.004s	sys 0.015s
I20260709 12:21:00.000367  2137 catalog_manager.cc:5697] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c reported cstate change: term changed from 0 to 1, leader changed from <none> to 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66). New cstate: current_term: 1 leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.008152  2430 tablet_bootstrap.cc:492] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95: Bootstrap starting.
I20260709 12:21:00.011206  2453 raft_consensus.cc:2804] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.011264  2429 raft_consensus.cc:359] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.012329  2429 raft_consensus.cc:385] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.012694  2429 raft_consensus.cc:740] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.013646  2429 consensus_queue.cc:260] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.012329  2453 raft_consensus.cc:697] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Leader. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:00.015645  2453 consensus_queue.cc:237] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.019407  2429 ts_tablet_manager.cc:1434] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 12:21:00.020602  2429 tablet_bootstrap.cc:492] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c: Bootstrap starting.
I20260709 12:21:00.022568  2430 tablet_bootstrap.cc:654] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.036780  2430 tablet_bootstrap.cc:492] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95: No bootstrap required, opened a new log
I20260709 12:21:00.037374  2241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:00.037724  2430 ts_tablet_manager.cc:1403] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95: Time spent bootstrapping tablet: real 0.030s	user 0.021s	sys 0.000s
I20260709 12:21:00.038460  2241 raft_consensus.cc:2393] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8837957e7c2c4384bdc425d1f86deb95 in current term 1: Already voted for candidate 915a285f0ece44d0b4aa1d695a3a408c in this term.
I20260709 12:21:00.042050  2430 raft_consensus.cc:359] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.043409  2430 raft_consensus.cc:385] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.044008  2430 raft_consensus.cc:740] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.045032  2354 leader_election.cc:304] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95; no voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.045154  2430 consensus_queue.cc:260] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.046180  2433 raft_consensus.cc:2749] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.047351  2429 tablet_bootstrap.cc:654] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.048077  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c237ebe3cce4553aa71f1dabf1abbea" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c" is_pre_election: true
I20260709 12:21:00.049830  2390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c237ebe3cce4553aa71f1dabf1abbea" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:21:00.050261  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:21:00.050611  2390 raft_consensus.cc:2468] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 0.
I20260709 12:21:00.051126  2388 raft_consensus.cc:2468] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 0.
I20260709 12:21:00.049772  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c" is_pre_election: true
W20260709 12:21:00.052825  2204 leader_election.cc:343] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.055310  2430 ts_tablet_manager.cc:1434] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:21:00.056886  2430 tablet_bootstrap.cc:492] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95: Bootstrap starting.
I20260709 12:21:00.055331  2204 leader_election.cc:304] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff; no voters: 915a285f0ece44d0b4aa1d695a3a408c
I20260709 12:21:00.061780  2429 tablet_bootstrap.cc:492] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c: No bootstrap required, opened a new log
I20260709 12:21:00.062374  2429 ts_tablet_manager.cc:1403] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.000s
I20260709 12:21:00.062872  2430 tablet_bootstrap.cc:654] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.062908  2441 raft_consensus.cc:2804] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.063465  2441 raft_consensus.cc:493] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.063827  2441 raft_consensus.cc:3060] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.065833  2429 raft_consensus.cc:359] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.066659  2429 raft_consensus.cc:385] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.066983  2429 raft_consensus.cc:740] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.067831  2429 consensus_queue.cc:260] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.070827  2429 ts_tablet_manager.cc:1434] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.006s
I20260709 12:21:00.071969  2429 tablet_bootstrap.cc:492] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c: Bootstrap starting.
I20260709 12:21:00.072290  2441 raft_consensus.cc:515] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.074632  2441 leader_election.cc:290] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 election: Requested vote from peers 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.075966  2204 leader_election.cc:304] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:00.078589  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c237ebe3cce4553aa71f1dabf1abbea" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:00.079736  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c237ebe3cce4553aa71f1dabf1abbea" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:00.080377  2388 raft_consensus.cc:3060] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.080607  2429 tablet_bootstrap.cc:654] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.081470  2434 raft_consensus.cc:493] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:00.082338  2204 leader_election.cc:343] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.082242  2434 raft_consensus.cc:515] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
W20260709 12:21:00.083417  2204 leader_election.cc:343] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.079327  2441 raft_consensus.cc:2804] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.084255  2441 raft_consensus.cc:493] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.084650  2441 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.085634  2434 leader_election.cc:290] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:21:00.085937  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c" is_pre_election: true
I20260709 12:21:00.086573  2315 raft_consensus.cc:2468] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 0.
I20260709 12:21:00.087800  2353 leader_election.cc:304] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: 
I20260709 12:21:00.088773  2434 raft_consensus.cc:2804] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.089185  2434 raft_consensus.cc:493] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.089617  2434 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.098137  2433 raft_consensus.cc:493] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.098829  2434 raft_consensus.cc:515] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.098948  2433 raft_consensus.cc:515] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.101243  2430 tablet_bootstrap.cc:492] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95: No bootstrap required, opened a new log
I20260709 12:21:00.099236  2388 raft_consensus.cc:2468] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 1.
I20260709 12:21:00.101847  2430 ts_tablet_manager.cc:1403] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95: Time spent bootstrapping tablet: real 0.045s	user 0.017s	sys 0.005s
I20260709 12:21:00.101963  2433 leader_election.cc:290] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:21:00.103474  2434 leader_election.cc:290] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:21:00.104677  2431 tablet_bootstrap.cc:654] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.105317  2430 raft_consensus.cc:359] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.106220  2430 raft_consensus.cc:385] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.106552  2430 raft_consensus.cc:740] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.107388  2430 consensus_queue.cc:260] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.109460  2204 leader_election.cc:304] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff; no voters: 915a285f0ece44d0b4aa1d695a3a408c
I20260709 12:21:00.109768  2430 ts_tablet_manager.cc:1434] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:00.112839  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c" is_pre_election: true
I20260709 12:21:00.114732  2471 raft_consensus.cc:2804] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.115034  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:00.115942  2316 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.116156  2441 raft_consensus.cc:515] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:00.118808  2441 leader_election.cc:290] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 election: Requested vote from peers 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:21:00.120579  2241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:21:00.121943  2239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:00.122165  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:00.121861  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:21:00.123618  2240 raft_consensus.cc:2393] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8837957e7c2c4384bdc425d1f86deb95 in current term 1: Already voted for candidate 915a285f0ece44d0b4aa1d695a3a408c in this term.
W20260709 12:21:00.124053  2354 leader_election.cc:343] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.124418  2388 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:00.125494  2354 leader_election.cc:343] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.125908  2453 raft_consensus.cc:493] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.126590  2453 raft_consensus.cc:515] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.127434  2431 tablet_bootstrap.cc:492] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff: No bootstrap required, opened a new log
I20260709 12:21:00.128099  2431 ts_tablet_manager.cc:1403] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff: Time spent bootstrapping tablet: real 0.126s	user 0.012s	sys 0.001s
I20260709 12:21:00.131608  2431 raft_consensus.cc:359] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.132154  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e118c00462e141328b65f49b71fc2b09" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:21:00.132900  2453 leader_election.cc:290] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.134398  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
W20260709 12:21:00.140702  2204 leader_election.cc:343] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:00.140977  2280 leader_election.cc:343] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.136121  2390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e118c00462e141328b65f49b71fc2b09" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:21:00.142148  2390 raft_consensus.cc:2468] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 0.
I20260709 12:21:00.142923  2137 catalog_manager.cc:5697] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c reported cstate change: term changed from 0 to 1, leader changed from <none> to 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66). New cstate: current_term: 1 leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.144645  2279 leader_election.cc:304] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.133179  2388 raft_consensus.cc:2468] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 1.
I20260709 12:21:00.146318  2439 raft_consensus.cc:2804] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.146869  2439 raft_consensus.cc:493] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.146983  2204 leader_election.cc:304] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff; no voters: 915a285f0ece44d0b4aa1d695a3a408c
I20260709 12:21:00.147426  2439 raft_consensus.cc:3060] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.147902  2441 raft_consensus.cc:2804] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.138521  2431 raft_consensus.cc:385] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.162649  2471 raft_consensus.cc:697] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 1 LEADER]: Becoming Leader. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:00.148401  2441 raft_consensus.cc:697] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 1 LEADER]: Becoming Leader. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:00.169973  2441 consensus_queue.cc:237] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:00.171944  2316 raft_consensus.cc:2468] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 1.
I20260709 12:21:00.174189  2434 raft_consensus.cc:493] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.174849  2434 raft_consensus.cc:515] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.177228  2434 leader_election.cc:290] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:21:00.178891  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4200fe852224b1c8f21a0bb88a0fc47" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:21:00.179101  2439 raft_consensus.cc:515] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
W20260709 12:21:00.181154  2354 leader_election.cc:343] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.137132  2353 leader_election.cc:304] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95; no voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.182578  2434 raft_consensus.cc:3060] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:00.182678  2353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:39587, user_credentials={real_user=slave}} blocked reactor thread for 45697.7us
I20260709 12:21:00.184499  2429 tablet_bootstrap.cc:492] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c: No bootstrap required, opened a new log
I20260709 12:21:00.185575  2429 ts_tablet_manager.cc:1403] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c: Time spent bootstrapping tablet: real 0.114s	user 0.033s	sys 0.059s
I20260709 12:21:00.165789  2471 consensus_queue.cc:237] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.187288  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e118c00462e141328b65f49b71fc2b09" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:00.184551  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4200fe852224b1c8f21a0bb88a0fc47" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c" is_pre_election: true
I20260709 12:21:00.162559  2431 raft_consensus.cc:740] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.189041  2453 raft_consensus.cc:493] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.197477  2431 consensus_queue.cc:260] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.198544  2453 raft_consensus.cc:515] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.200729  2431 ts_tablet_manager.cc:1434] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff: Time spent starting tablet: real 0.072s	user 0.004s	sys 0.004s
I20260709 12:21:00.200891  2453 leader_election.cc:290] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
W20260709 12:21:00.201483  2280 leader_election.cc:343] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.202234  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:21:00.203020  2439 leader_election.cc:290] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.202965  2388 raft_consensus.cc:2468] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 0.
I20260709 12:21:00.204314  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:21:00.203861  2353 leader_election.cc:304] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: efe356271a3d43b7867710d1e5962dff
W20260709 12:21:00.205617  2280 leader_election.cc:343] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.206632  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e118c00462e141328b65f49b71fc2b09" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:00.207386  2388 raft_consensus.cc:3060] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.208865  2429 raft_consensus.cc:359] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.209756  2433 raft_consensus.cc:2804] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.206611  2279 leader_election.cc:304] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.209818  2429 raft_consensus.cc:385] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.211223  2429 raft_consensus.cc:740] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Initialized, Role: FOLLOWER
W20260709 12:21:00.214382  2353 leader_election.cc:343] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.215716  2353 leader_election.cc:304] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95; no voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.217700  2431 tablet_bootstrap.cc:492] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff: Bootstrap starting.
I20260709 12:21:00.219235  2429 consensus_queue.cc:260] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.221583  2137 catalog_manager.cc:5697] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff reported cstate change: term changed from 0 to 1, leader changed from <none> to efe356271a3d43b7867710d1e5962dff (127.1.123.65). New cstate: current_term: 1 leader_uuid: "efe356271a3d43b7867710d1e5962dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.223042  2434 raft_consensus.cc:2749] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.219352  2433 raft_consensus.cc:697] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 LEADER]: Becoming Leader. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Running, Role: LEADER
I20260709 12:21:00.218345  2439 raft_consensus.cc:2804] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.229035  2434 raft_consensus.cc:2749] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.236178  2429 ts_tablet_manager.cc:1434] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.000s
I20260709 12:21:00.237247  2429 tablet_bootstrap.cc:492] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c: Bootstrap starting.
I20260709 12:21:00.238304  2433 consensus_queue.cc:237] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.241122  2453 raft_consensus.cc:493] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.241680  2453 raft_consensus.cc:515] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.243578  2429 tablet_bootstrap.cc:654] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.243959  2453 leader_election.cc:290] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.245504  2390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c237ebe3cce4553aa71f1dabf1abbea" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:21:00.228410  2439 raft_consensus.cc:493] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.247275  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c237ebe3cce4553aa71f1dabf1abbea" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:21:00.247802  2390 raft_consensus.cc:2393] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 915a285f0ece44d0b4aa1d695a3a408c in current term 1: Already voted for candidate efe356271a3d43b7867710d1e5962dff in this term.
I20260709 12:21:00.251791  2439 raft_consensus.cc:3060] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.256237  2279 leader_election.cc:304] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c; no voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.257638  2453 raft_consensus.cc:3060] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.259238  2138 catalog_manager.cc:5697] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff reported cstate change: term changed from 0 to 1, leader changed from <none> to efe356271a3d43b7867710d1e5962dff (127.1.123.65). New cstate: current_term: 1 leader_uuid: "efe356271a3d43b7867710d1e5962dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.281589  2439 raft_consensus.cc:515] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.281649  2138 catalog_manager.cc:5697] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "8837957e7c2c4384bdc425d1f86deb95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.284422  2439 leader_election.cc:290] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.285317  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:00.286007  2453 raft_consensus.cc:2749] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:00.287002  2280 leader_election.cc:343] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.289031  2429 tablet_bootstrap.cc:492] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c: No bootstrap required, opened a new log
I20260709 12:21:00.288909  2390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:00.289558  2429 ts_tablet_manager.cc:1403] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.015s
I20260709 12:21:00.289664  2390 raft_consensus.cc:3060] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.289808  2431 tablet_bootstrap.cc:654] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.292778  2429 raft_consensus.cc:359] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.293478  2429 raft_consensus.cc:385] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.294059  2429 raft_consensus.cc:740] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.295449  2429 consensus_queue.cc:260] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.296144  2388 raft_consensus.cc:2468] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 1.
I20260709 12:21:00.298051  2429 ts_tablet_manager.cc:1434] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:00.299316  2429 tablet_bootstrap.cc:492] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c: Bootstrap starting.
I20260709 12:21:00.305655  2279 leader_election.cc:304] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.307224  2429 tablet_bootstrap.cc:654] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.307325  2439 raft_consensus.cc:2804] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.308095  2439 raft_consensus.cc:697] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Leader. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:00.308975  2453 raft_consensus.cc:493] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.309504  2453 raft_consensus.cc:515] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.312141  2390 raft_consensus.cc:2468] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 1.
I20260709 12:21:00.312307  2453 leader_election.cc:290] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.314131  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a4c73011a7449bb1e0852f1e3056d1" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:21:00.315956  2240 raft_consensus.cc:2468] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 0.
I20260709 12:21:00.314828  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a4c73011a7449bb1e0852f1e3056d1" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:21:00.316686  2279 leader_election.cc:304] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.317094  2388 raft_consensus.cc:2468] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 0.
I20260709 12:21:00.318815  2280 leader_election.cc:304] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:00.319342  2429 tablet_bootstrap.cc:492] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c: No bootstrap required, opened a new log
I20260709 12:21:00.319852  2429 ts_tablet_manager.cc:1403] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 12:21:00.319984  2458 raft_consensus.cc:2804] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.320552  2458 raft_consensus.cc:493] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.320917  2458 raft_consensus.cc:3060] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.319715  2453 raft_consensus.cc:2804] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.309510  2439 consensus_queue.cc:237] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.324177  2431 tablet_bootstrap.cc:492] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff: No bootstrap required, opened a new log
I20260709 12:21:00.324295  2429 raft_consensus.cc:359] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:00.324985  2431 ts_tablet_manager.cc:1403] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff: Time spent bootstrapping tablet: real 0.108s	user 0.008s	sys 0.018s
I20260709 12:21:00.325047  2429 raft_consensus.cc:385] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.325469  2429 raft_consensus.cc:740] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.326222  2429 consensus_queue.cc:260] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:00.327713  2453 raft_consensus.cc:697] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Leader. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:00.328701  2431 raft_consensus.cc:359] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.329559  2431 raft_consensus.cc:385] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.328812  2453 consensus_queue.cc:237] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.330000  2431 raft_consensus.cc:740] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.330299  2458 raft_consensus.cc:515] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.330884  2431 consensus_queue.cc:260] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.333871  2431 ts_tablet_manager.cc:1434] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:21:00.334052  2429 ts_tablet_manager.cc:1434] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:21:00.335310  2431 tablet_bootstrap.cc:492] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff: Bootstrap starting.
I20260709 12:21:00.341887  2458 leader_election.cc:290] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.342573  2458 raft_consensus.cc:493] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.342847  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a4c73011a7449bb1e0852f1e3056d1" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:00.343047  2458 raft_consensus.cc:515] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.343780  2240 raft_consensus.cc:3060] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.344060  2431 tablet_bootstrap.cc:654] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.346169  2458 leader_election.cc:290] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.347038  2239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4200fe852224b1c8f21a0bb88a0fc47" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:21:00.347949  2239 raft_consensus.cc:2468] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 0.
I20260709 12:21:00.354585  2280 leader_election.cc:304] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:00.355959  2458 raft_consensus.cc:2804] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.356472  2458 raft_consensus.cc:493] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.356920  2458 raft_consensus.cc:3060] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.358345  2431 tablet_bootstrap.cc:492] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff: No bootstrap required, opened a new log
I20260709 12:21:00.358924  2431 ts_tablet_manager.cc:1403] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.004s
I20260709 12:21:00.362397  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a4c73011a7449bb1e0852f1e3056d1" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:00.362421  2431 raft_consensus.cc:359] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.363456  2388 raft_consensus.cc:3060] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.363598  2431 raft_consensus.cc:385] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.364108  2431 raft_consensus.cc:740] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.364943  2431 consensus_queue.cc:260] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.367377  2240 raft_consensus.cc:2468] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 1.
I20260709 12:21:00.367910  2431 ts_tablet_manager.cc:1434] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:21:00.368541  2390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4200fe852224b1c8f21a0bb88a0fc47" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:21:00.369064  2431 tablet_bootstrap.cc:492] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff: Bootstrap starting.
I20260709 12:21:00.371667  2280 leader_election.cc:304] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:00.372335  2483 consensus_queue.cc:1048] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.369490  2390 raft_consensus.cc:2468] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 0.
I20260709 12:21:00.378049  2431 tablet_bootstrap.cc:654] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.393258  2481 raft_consensus.cc:2804] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.399994  2388 raft_consensus.cc:2468] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 1.
I20260709 12:21:00.407574  2481 raft_consensus.cc:697] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Leader. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:00.411383  2481 consensus_queue.cc:237] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.406126  2458 raft_consensus.cc:515] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.421478  2458 leader_election.cc:290] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.427485  2458 raft_consensus.cc:493] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.427968  2458 raft_consensus.cc:515] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.422979  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4200fe852224b1c8f21a0bb88a0fc47" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:00.436830  2240 raft_consensus.cc:3060] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.442764  2483 consensus_queue.cc:1048] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.443207  2239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a41d9df904094b35b6d61a167f8359d2" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:21:00.444152  2239 raft_consensus.cc:2468] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 0.
I20260709 12:21:00.445793  2280 leader_election.cc:304] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:00.445858  2458 leader_election.cc:290] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.447446  2488 raft_consensus.cc:2804] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.450732  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a41d9df904094b35b6d61a167f8359d2" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:21:00.451783  2488 raft_consensus.cc:493] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.451669  2389 raft_consensus.cc:2468] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 0.
I20260709 12:21:00.434851  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4200fe852224b1c8f21a0bb88a0fc47" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:00.454799  2388 raft_consensus.cc:3060] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.457224  2431 tablet_bootstrap.cc:492] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff: No bootstrap required, opened a new log
I20260709 12:21:00.457937  2431 ts_tablet_manager.cc:1403] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff: Time spent bootstrapping tablet: real 0.089s	user 0.011s	sys 0.046s
I20260709 12:21:00.461184  2488 raft_consensus.cc:3060] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.472563  2240 raft_consensus.cc:2468] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 1.
I20260709 12:21:00.474203  2280 leader_election.cc:304] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:00.475003  2481 raft_consensus.cc:2804] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.475674  2481 raft_consensus.cc:697] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Leader. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:00.476855  2481 consensus_queue.cc:237] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.484956  2431 raft_consensus.cc:359] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.487922  2431 raft_consensus.cc:385] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.488241  2431 raft_consensus.cc:740] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.492998  2431 consensus_queue.cc:260] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.500072  2431 ts_tablet_manager.cc:1434] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 12:21:00.499944  2388 raft_consensus.cc:2468] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 1.
I20260709 12:21:00.501374  2431 tablet_bootstrap.cc:492] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff: Bootstrap starting.
I20260709 12:21:00.485960  2136 catalog_manager.cc:5697] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c reported cstate change: term changed from 0 to 1, leader changed from <none> to 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66). New cstate: current_term: 1 leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.509771  2488 raft_consensus.cc:515] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.514132  2239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a41d9df904094b35b6d61a167f8359d2" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:00.514863  2239 raft_consensus.cc:3060] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.518121  2488 leader_election.cc:290] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.519488  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a41d9df904094b35b6d61a167f8359d2" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:00.520197  2389 raft_consensus.cc:3060] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.521311  2431 tablet_bootstrap.cc:654] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.527160  2431 tablet_bootstrap.cc:492] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff: No bootstrap required, opened a new log
I20260709 12:21:00.527719  2431 ts_tablet_manager.cc:1403] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.000s
I20260709 12:21:00.529102  2389 raft_consensus.cc:2468] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 1.
I20260709 12:21:00.530748  2431 raft_consensus.cc:359] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.531543  2431 raft_consensus.cc:385] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.531833  2431 raft_consensus.cc:740] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.532605  2431 consensus_queue.cc:260] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.534369  2279 leader_election.cc:304] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: 
I20260709 12:21:00.535131  2239 raft_consensus.cc:2468] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 1.
I20260709 12:21:00.536114  2431 ts_tablet_manager.cc:1434] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:21:00.537505  2431 tablet_bootstrap.cc:492] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff: Bootstrap starting.
I20260709 12:21:00.537990  2488 raft_consensus.cc:2804] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.538477  2488 raft_consensus.cc:697] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Leader. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:00.539250  2488 consensus_queue.cc:237] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.544026  2431 tablet_bootstrap.cc:654] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.545938  2137 catalog_manager.cc:5697] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c reported cstate change: term changed from 0 to 1, leader changed from <none> to 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66). New cstate: current_term: 1 leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.548120  2137 catalog_manager.cc:5697] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c reported cstate change: term changed from 0 to 1, leader changed from <none> to 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66). New cstate: current_term: 1 leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.549821  2431 tablet_bootstrap.cc:492] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff: No bootstrap required, opened a new log
I20260709 12:21:00.550387  2431 ts_tablet_manager.cc:1403] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.003s
I20260709 12:21:00.550113  2137 catalog_manager.cc:5697] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c reported cstate change: term changed from 0 to 1, leader changed from <none> to 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66). New cstate: current_term: 1 leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.553640  2431 raft_consensus.cc:359] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.554379  2431 raft_consensus.cc:385] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.554670  2431 raft_consensus.cc:740] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.555330  2431 consensus_queue.cc:260] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.556258  2488 raft_consensus.cc:493] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.556874  2431 ts_tablet_manager.cc:1434] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:21:00.556746  2488 raft_consensus.cc:515] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:00.559158  2488 leader_election.cc:290] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.561475  2239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe356271a3d43b7867710d1e5962dff" is_pre_election: true
I20260709 12:21:00.561475  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:21:00.563246  2389 raft_consensus.cc:2393] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 915a285f0ece44d0b4aa1d695a3a408c in current term 1: Already voted for candidate efe356271a3d43b7867710d1e5962dff in this term.
I20260709 12:21:00.564385  2279 leader_election.cc:304] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c; no voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.565194  2488 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.569962  2481 consensus_queue.cc:1048] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.572204  2138 catalog_manager.cc:5697] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c reported cstate change: term changed from 0 to 1, leader changed from <none> to 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66). New cstate: current_term: 1 leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.575992  2488 raft_consensus.cc:2749] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.582415  2471 consensus_queue.cc:1048] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.590118  2488 consensus_queue.cc:1048] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.610442  2472 consensus_queue.cc:1048] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.636861  2472 raft_consensus.cc:493] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.637413  2472 raft_consensus.cc:515] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.639724  2472 leader_election.cc:290] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.640897  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c" is_pre_election: true
I20260709 12:21:00.641158  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8837957e7c2c4384bdc425d1f86deb95" is_pre_election: true
I20260709 12:21:00.642302  2389 raft_consensus.cc:2393] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efe356271a3d43b7867710d1e5962dff in current term 1: Already voted for candidate 915a285f0ece44d0b4aa1d695a3a408c in this term.
I20260709 12:21:00.643453  2204 leader_election.cc:304] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efe356271a3d43b7867710d1e5962dff; no voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c
I20260709 12:21:00.644291  2472 raft_consensus.cc:3060] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.650913  2472 raft_consensus.cc:2749] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.679946  2472 consensus_queue.cc:1048] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.692320  2472 consensus_queue.cc:1048] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.720214  2488 consensus_queue.cc:1048] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.733098  2458 consensus_queue.cc:1048] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.770987  2239 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.772492  2458 consensus_queue.cc:1048] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.781661  2433 consensus_queue.cc:1048] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.790378  2240 raft_consensus.cc:3060] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.794629  2389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04"
dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
mode: GRACEFUL
new_leader_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:59642
I20260709 12:21:00.795398  2389 raft_consensus.cc:606] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 LEADER]: Received request to transfer leadership to TS efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.809489  2458 consensus_queue.cc:1048] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.822383  2316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a41d9df904094b35b6d61a167f8359d2"
dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
mode: GRACEFUL
new_leader_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:50046
I20260709 12:21:00.823285  2316 raft_consensus.cc:606] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Received request to transfer leadership to TS efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.822894  2511 raft_consensus.cc:993] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95: : Instructing follower efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:00.831702  2458 consensus_queue.cc:1048] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.834676  2316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e118c00462e141328b65f49b71fc2b09"
dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
mode: GRACEFUL
new_leader_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:50046
I20260709 12:21:00.835389  2316 raft_consensus.cc:606] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Received request to transfer leadership to TS efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.846014  2316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282"
dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
mode: GRACEFUL
new_leader_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:50046
I20260709 12:21:00.846925  2316 raft_consensus.cc:606] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Received request to transfer leadership to TS efe356271a3d43b7867710d1e5962dff
I20260709 12:21:00.851945  2433 consensus_queue.cc:1048] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.845937  2513 raft_consensus.cc:1081] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 LEADER]: Signalling peer efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:00.854926  2240 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:39100
I20260709 12:21:00.855633  2240 raft_consensus.cc:493] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:00.856285  2240 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:00.871600  2488 consensus_queue.cc:1048] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.872268  2240 raft_consensus.cc:515] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
W20260709 12:21:00.878093  2353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:39587, user_credentials={real_user=slave}} blocked reactor thread for 44238us
I20260709 12:21:00.881618  2240 leader_election.cc:290] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Requested vote from peers 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:00.882190  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:00.905184  2316 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:00.884006  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:00.909919  2388 raft_consensus.cc:3055] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:00.910351  2388 raft_consensus.cc:740] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Running, Role: LEADER
I20260709 12:21:00.911284  2388 consensus_queue.cc:260] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.912158  2316 raft_consensus.cc:2468] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:00.912609  2388 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:00.913373  2204 leader_election.cc:304] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:00.914428  2441 raft_consensus.cc:2804] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:00.914932  2441 raft_consensus.cc:697] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Becoming Leader. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:00.915772  2441 consensus_queue.cc:237] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.915926  2240 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c237ebe3cce4553aa71f1dabf1abbea"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
mode: GRACEFUL
new_leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
 from {username='slave'} at 127.0.0.1:39120
I20260709 12:21:00.916855  2240 raft_consensus.cc:606] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 1 LEADER]: Received request to transfer leadership to TS 915a285f0ece44d0b4aa1d695a3a408c
I20260709 12:21:00.919607  2388 raft_consensus.cc:2468] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:00.924651  2138 catalog_manager.cc:5697] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff reported cstate change: term changed from 1 to 2, leader changed from 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67) to efe356271a3d43b7867710d1e5962dff (127.1.123.65). New cstate: current_term: 2 leader_uuid: "efe356271a3d43b7867710d1e5962dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.925576  2316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4200fe852224b1c8f21a0bb88a0fc47"
dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
mode: GRACEFUL
new_leader_uuid: "8837957e7c2c4384bdc425d1f86deb95"
 from {username='slave'} at 127.0.0.1:50046
I20260709 12:21:00.926204  2316 raft_consensus.cc:606] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Received request to transfer leadership to TS 8837957e7c2c4384bdc425d1f86deb95
I20260709 12:21:00.957131  2488 consensus_queue.cc:1048] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.969558  2439 consensus_queue.cc:1048] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.979017  2488 raft_consensus.cc:993] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c: : Instructing follower 8837957e7c2c4384bdc425d1f86deb95 to start an election
I20260709 12:21:00.979570  2439 raft_consensus.cc:1081] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Signalling peer 8837957e7c2c4384bdc425d1f86deb95 to start an election
I20260709 12:21:00.981320  2388 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4200fe852224b1c8f21a0bb88a0fc47"
dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
 from {username='slave'} at 127.0.0.1:59632
I20260709 12:21:00.982020  2388 raft_consensus.cc:493] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:00.982457  2388 raft_consensus.cc:3060] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:00.988490  2388 raft_consensus.cc:515] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.990480  2388 leader_election.cc:290] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 2 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:21:00.991736  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4200fe852224b1c8f21a0bb88a0fc47" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:00.992062  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4200fe852224b1c8f21a0bb88a0fc47" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:00.992434  2316 raft_consensus.cc:3055] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:00.992821  2240 raft_consensus.cc:3060] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:00.992856  2316 raft_consensus.cc:740] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:00.993991  2316 consensus_queue.cc:260] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:00.995559  2316 raft_consensus.cc:3060] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.001500  2240 raft_consensus.cc:2468] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 2.
I20260709 12:21:01.003094  2354 leader_election.cc:304] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:01.003539  2316 raft_consensus.cc:2468] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 2.
I20260709 12:21:01.004000  2515 raft_consensus.cc:2804] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:01.004558  2515 raft_consensus.cc:697] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 LEADER]: Becoming Leader. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Running, Role: LEADER
I20260709 12:21:01.005355  2439 raft_consensus.cc:993] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c: : Instructing follower efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:01.005971  2481 raft_consensus.cc:1081] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Signalling peer efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:01.005523  2515 consensus_queue.cc:237] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.008111  2239 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:39104
I20260709 12:21:01.008805  2239 raft_consensus.cc:493] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:01.009213  2239 raft_consensus.cc:3060] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.014220  2136 catalog_manager.cc:5697] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 reported cstate change: term changed from 1 to 2, leader changed from 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66) to 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "8837957e7c2c4384bdc425d1f86deb95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:01.018900  2239 raft_consensus.cc:515] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.022423  2316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:01.022758  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:01.023190  2316 raft_consensus.cc:3055] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:01.023334  2388 raft_consensus.cc:3060] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.023643  2316 raft_consensus.cc:740] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:01.024626  2316 consensus_queue.cc:260] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.025979  2316 raft_consensus.cc:3060] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.029039  2481 consensus_queue.cc:1048] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:01.031127  2388 raft_consensus.cc:2468] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:01.033990  2204 leader_election.cc:304] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:01.035013  2441 raft_consensus.cc:2804] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:01.035567  2441 raft_consensus.cc:697] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Becoming Leader. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:01.021745  2239 leader_election.cc:290] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Requested vote from peers 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:01.036549  2441 consensus_queue.cc:237] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.037497  2316 raft_consensus.cc:2468] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:01.045737  2458 consensus_queue.cc:1048] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:01.046962  2138 catalog_manager.cc:5697] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff reported cstate change: term changed from 1 to 2, leader changed from 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66) to efe356271a3d43b7867710d1e5962dff (127.1.123.65). New cstate: current_term: 2 leader_uuid: "efe356271a3d43b7867710d1e5962dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:01.075687  2488 raft_consensus.cc:993] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c: : Instructing follower efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:01.076246  2488 raft_consensus.cc:1081] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Signalling peer efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:01.078361  2471 raft_consensus.cc:993] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff: : Instructing follower 915a285f0ece44d0b4aa1d695a3a408c to start an election
I20260709 12:21:01.078842  2441 raft_consensus.cc:1081] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 1 LEADER]: Signalling peer 915a285f0ece44d0b4aa1d695a3a408c to start an election
I20260709 12:21:01.078701  2239 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a41d9df904094b35b6d61a167f8359d2"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:39104
I20260709 12:21:01.079361  2239 raft_consensus.cc:493] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:01.079772  2239 raft_consensus.cc:3060] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.080531  2316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c237ebe3cce4553aa71f1dabf1abbea"
dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
 from {username='slave'} at 127.0.0.1:50038
I20260709 12:21:01.081037  2316 raft_consensus.cc:493] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:01.081346  2316 raft_consensus.cc:3060] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.086315  2239 raft_consensus.cc:515] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.086683  2316 raft_consensus.cc:515] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.088660  2239 leader_election.cc:290] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Requested vote from peers 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:01.089125  2316 leader_election.cc:290] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 2 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:01.089710  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a41d9df904094b35b6d61a167f8359d2" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:01.089985  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c237ebe3cce4553aa71f1dabf1abbea" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:01.090451  2315 raft_consensus.cc:3055] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:01.090619  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c237ebe3cce4553aa71f1dabf1abbea" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:01.090588  2240 raft_consensus.cc:3055] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:01.090870  2315 raft_consensus.cc:740] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:01.091228  2240 raft_consensus.cc:740] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:01.091974  2315 consensus_queue.cc:260] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.092271  2240 consensus_queue.cc:260] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.091276  2389 raft_consensus.cc:3060] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.093367  2388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a41d9df904094b35b6d61a167f8359d2" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:01.093724  2240 raft_consensus.cc:3060] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.093645  2315 raft_consensus.cc:3060] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.094065  2388 raft_consensus.cc:3060] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.102133  2388 raft_consensus.cc:2468] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:01.102133  2389 raft_consensus.cc:2468] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 2.
I20260709 12:21:01.102808  2240 raft_consensus.cc:2468] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 2.
I20260709 12:21:01.102813  2315 raft_consensus.cc:2468] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:01.103988  2279 leader_election.cc:304] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: 
I20260709 12:21:01.104722  2204 leader_election.cc:304] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:01.104935  2458 raft_consensus.cc:2804] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:01.105574  2458 raft_consensus.cc:697] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 2 LEADER]: Becoming Leader. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:01.105726  2441 raft_consensus.cc:2804] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:01.106245  2441 raft_consensus.cc:697] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Becoming Leader. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:01.106633  2458 consensus_queue.cc:237] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.107200  2441 consensus_queue.cc:237] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.114539  2138 catalog_manager.cc:5697] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c reported cstate change: term changed from 1 to 2, leader changed from efe356271a3d43b7867710d1e5962dff (127.1.123.65) to 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66). New cstate: current_term: 2 leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:01.114964  2137 catalog_manager.cc:5697] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff reported cstate change: term changed from 1 to 2, leader changed from 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66) to efe356271a3d43b7867710d1e5962dff (127.1.123.65). New cstate: current_term: 2 leader_uuid: "efe356271a3d43b7867710d1e5962dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:01.285110  2439 raft_consensus.cc:993] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c: : Instructing follower efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:01.285676  2458 raft_consensus.cc:1081] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Signalling peer efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:01.287467  2240 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e118c00462e141328b65f49b71fc2b09"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:39104
I20260709 12:21:01.288017  2240 raft_consensus.cc:493] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:01.288363  2240 raft_consensus.cc:3060] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.294691  2240 raft_consensus.cc:515] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.296645  2240 leader_election.cc:290] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Requested vote from peers 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:01.297680  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e118c00462e141328b65f49b71fc2b09" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:01.298170  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e118c00462e141328b65f49b71fc2b09" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:01.298262  2315 raft_consensus.cc:3055] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:01.298743  2315 raft_consensus.cc:740] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:01.298769  2389 raft_consensus.cc:3060] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.299625  2315 consensus_queue.cc:260] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.300631  2315 raft_consensus.cc:3060] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:01.304299  2389 raft_consensus.cc:2468] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:01.305763  2204 leader_election.cc:304] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:01.306694  2441 raft_consensus.cc:2804] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:01.306690  2315 raft_consensus.cc:2468] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:01.307436  2441 raft_consensus.cc:697] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Becoming Leader. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:01.308332  2441 consensus_queue.cc:237] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:01.315765  2137 catalog_manager.cc:5697] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff reported cstate change: term changed from 1 to 2, leader changed from 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66) to efe356271a3d43b7867710d1e5962dff (127.1.123.65). New cstate: current_term: 2 leader_uuid: "efe356271a3d43b7867710d1e5962dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:01.360277  2315 raft_consensus.cc:1275] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.361864  2441 consensus_queue.cc:1048] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:01.370784  2389 raft_consensus.cc:1275] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.372530  2441 consensus_queue.cc:1048] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:01.510609  2389 raft_consensus.cc:1275] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.512148  2441 consensus_queue.cc:1048] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:01.520936  2315 raft_consensus.cc:1275] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.522946  2472 consensus_queue.cc:1048] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:01.550805  2389 raft_consensus.cc:1275] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.552470  2472 consensus_queue.cc:1048] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:01.555905  2316 raft_consensus.cc:1275] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Refusing update from remote peer 8837957e7c2c4384bdc425d1f86deb95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.555876  2315 raft_consensus.cc:1275] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.557914  2515 consensus_queue.cc:1048] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:01.558277  2471 consensus_queue.cc:1048] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:01.568084  2240 raft_consensus.cc:1275] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Refusing update from remote peer 8837957e7c2c4384bdc425d1f86deb95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.571138  2515 consensus_queue.cc:1048] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:21:01.726786  2240 raft_consensus.cc:1275] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Refusing update from remote peer 915a285f0ece44d0b4aa1d695a3a408c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.728438  2458 consensus_queue.cc:1048] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:01.731669  2389 raft_consensus.cc:1275] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Refusing update from remote peer 915a285f0ece44d0b4aa1d695a3a408c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.733489  2481 consensus_queue.cc:1048] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:01.734076  2389 raft_consensus.cc:1275] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.735833  2472 consensus_queue.cc:1048] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:01.756978  2316 raft_consensus.cc:1275] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:01.758663  2472 consensus_queue.cc:1048] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:03.960057  1517 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:21:03.960824  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:03.982569  2240 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
mode: GRACEFUL
new_leader_uuid: "8837957e7c2c4384bdc425d1f86deb95"
 from {username='slave'} at 127.0.0.1:39124
I20260709 12:21:03.983261  2240 raft_consensus.cc:606] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Received request to transfer leadership to TS 8837957e7c2c4384bdc425d1f86deb95
I20260709 12:21:03.985975  2240 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
mode: GRACEFUL
new_leader_uuid: "8837957e7c2c4384bdc425d1f86deb95"
 from {username='slave'} at 127.0.0.1:39124
I20260709 12:21:03.986605  2240 raft_consensus.cc:606] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 1 LEADER]: Received request to transfer leadership to TS 8837957e7c2c4384bdc425d1f86deb95
I20260709 12:21:03.987928  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:21:03.988456  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:04.161522  2533 raft_consensus.cc:993] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff: : Instructing follower 8837957e7c2c4384bdc425d1f86deb95 to start an election
I20260709 12:21:04.162096  2471 raft_consensus.cc:1081] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 1 LEADER]: Signalling peer 8837957e7c2c4384bdc425d1f86deb95 to start an election
I20260709 12:21:04.163787  2389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea"
dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
 from {username='slave'} at 127.0.0.1:59636
I20260709 12:21:04.164415  2389 raft_consensus.cc:493] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:04.164752  2389 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:04.170083  2389 raft_consensus.cc:515] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:04.171816  2389 leader_election.cc:290] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 2 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:21:04.172991  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:04.173115  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:04.173804  2240 raft_consensus.cc:3055] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:04.173851  2315 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:04.174275  2240 raft_consensus.cc:740] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:04.175251  2240 consensus_queue.cc:260] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:04.176622  2240 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:04.181648  2315 raft_consensus.cc:2468] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 2.
I20260709 12:21:04.182904  2240 raft_consensus.cc:2468] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 2.
I20260709 12:21:04.183117  2353 leader_election.cc:304] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: 
I20260709 12:21:04.184051  2526 raft_consensus.cc:2804] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:04.185279  2526 raft_consensus.cc:697] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 LEADER]: Becoming Leader. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Running, Role: LEADER
I20260709 12:21:04.186141  2526 consensus_queue.cc:237] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:04.194147  2136 catalog_manager.cc:5697] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 reported cstate change: term changed from 1 to 2, leader changed from efe356271a3d43b7867710d1e5962dff (127.1.123.65) to 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "8837957e7c2c4384bdc425d1f86deb95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:04.307559  2533 raft_consensus.cc:993] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff: : Instructing follower 8837957e7c2c4384bdc425d1f86deb95 to start an election
I20260709 12:21:04.308095  2471 raft_consensus.cc:1081] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Signalling peer 8837957e7c2c4384bdc425d1f86deb95 to start an election
I20260709 12:21:04.309732  2389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04"
dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
 from {username='slave'} at 127.0.0.1:59636
I20260709 12:21:04.310251  2389 raft_consensus.cc:493] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:04.310590  2389 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:04.315729  2389 raft_consensus.cc:515] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:04.317453  2389 leader_election.cc:290] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 3 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:21:04.318544  2240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:04.318596  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:04.319336  2240 raft_consensus.cc:3055] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:04.319350  2315 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:04.319830  2240 raft_consensus.cc:740] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:04.320708  2240 consensus_queue.cc:260] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:04.322034  2240 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:04.327322  2315 raft_consensus.cc:2468] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 3.
I20260709 12:21:04.328644  2353 leader_election.cc:304] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: 
I20260709 12:21:04.329082  2240 raft_consensus.cc:2468] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 3.
I20260709 12:21:04.329505  2526 raft_consensus.cc:2804] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:04.330029  2526 raft_consensus.cc:697] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 LEADER]: Becoming Leader. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Running, Role: LEADER
I20260709 12:21:04.330817  2526 consensus_queue.cc:237] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:04.340909  2136 catalog_manager.cc:5697] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 reported cstate change: term changed from 2 to 3, leader changed from efe356271a3d43b7867710d1e5962dff (127.1.123.65) to 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67). New cstate: current_term: 3 leader_uuid: "8837957e7c2c4384bdc425d1f86deb95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:04.578877  2240 raft_consensus.cc:1275] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Refusing update from remote peer 8837957e7c2c4384bdc425d1f86deb95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:04.580276  2526 consensus_queue.cc:1048] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:04.592665  2315 raft_consensus.cc:1275] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Refusing update from remote peer 8837957e7c2c4384bdc425d1f86deb95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:04.594470  2526 consensus_queue.cc:1048] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:04.818616  2315 raft_consensus.cc:1275] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Refusing update from remote peer 8837957e7c2c4384bdc425d1f86deb95: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:04.820020  2526 consensus_queue.cc:1048] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:04.829659  2240 raft_consensus.cc:1275] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Refusing update from remote peer 8837957e7c2c4384bdc425d1f86deb95: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:04.831574  2549 consensus_queue.cc:1048] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:04.989701  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:04.995713  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:04.996258  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:05.997387  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:06.003279  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:06.003907  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:07.005291  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:07.011803  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:07.012341  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:08.013777  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:08.019847  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:08.020459  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:09.056983  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:09.063573  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:09.064198  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:21:09.918659  2170 debug-util.cc:398] Leaking SignalData structure 0x7b08000545a0 after lost signal to thread 1520
W20260709 12:21:09.919668  2170 debug-util.cc:398] Leaking SignalData structure 0x7b0800054c80 after lost signal to thread 2173
W20260709 12:21:09.920640  2170 debug-util.cc:398] Leaking SignalData structure 0x7b0800203240 after lost signal to thread 2265
W20260709 12:21:09.921435  2170 debug-util.cc:398] Leaking SignalData structure 0x7b0800203560 after lost signal to thread 2340
W20260709 12:21:09.922242  2170 debug-util.cc:398] Leaking SignalData structure 0x7b08001c9d40 after lost signal to thread 2414
I20260709 12:21:10.065656  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:10.075496  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:10.076212  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:11.077749  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:11.087433  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:11.088146  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:12.089516  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:12.099634  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:12.100378  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:13.101678  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:13.107420  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:13.107870  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:14.143435  1517 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:21:14.148490  2389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea"
dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
mode: GRACEFUL
new_leader_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:59642
I20260709 12:21:14.149312  2389 raft_consensus.cc:606] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 LEADER]: Received request to transfer leadership to TS efe356271a3d43b7867710d1e5962dff
I20260709 12:21:14.154620  2389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04"
dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
mode: GRACEFUL
new_leader_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:59642
I20260709 12:21:14.155114  2389 raft_consensus.cc:606] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 LEADER]: Received request to transfer leadership to TS efe356271a3d43b7867710d1e5962dff
I20260709 12:21:14.167980  2315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2"
dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
mode: GRACEFUL
new_leader_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:50046
I20260709 12:21:14.168510  2315 raft_consensus.cc:606] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Received request to transfer leadership to TS efe356271a3d43b7867710d1e5962dff
I20260709 12:21:14.173913  2315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1a4c73011a7449bb1e0852f1e3056d1"
dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
mode: GRACEFUL
new_leader_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:50046
I20260709 12:21:14.174536  2315 raft_consensus.cc:606] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Received request to transfer leadership to TS efe356271a3d43b7867710d1e5962dff
I20260709 12:21:14.179590  2315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "747f34920d614b61852845d2ab96deb3"
dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
mode: GRACEFUL
new_leader_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:50046
I20260709 12:21:14.180169  2315 raft_consensus.cc:606] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Received request to transfer leadership to TS efe356271a3d43b7867710d1e5962dff
I20260709 12:21:14.333412  2569 raft_consensus.cc:993] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c: : Instructing follower efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:14.333985  2569 raft_consensus.cc:1081] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Signalling peer efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:14.335728  2239 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:39104
I20260709 12:21:14.336314  2239 raft_consensus.cc:493] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.336715  2239 raft_consensus.cc:3060] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.342779  2239 raft_consensus.cc:515] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.344525  2239 leader_election.cc:290] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Requested vote from peers 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:14.346026  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:14.345847  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:14.346782  2389 raft_consensus.cc:3060] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.346704  2315 raft_consensus.cc:3055] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:14.347210  2315 raft_consensus.cc:740] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:14.348133  2315 consensus_queue.cc:260] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.349225  2315 raft_consensus.cc:3060] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.352228  2389 raft_consensus.cc:2468] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:14.353446  2204 leader_election.cc:304] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:14.354316  2551 raft_consensus.cc:2804] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:14.355136  2315 raft_consensus.cc:2468] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:14.356184  2551 raft_consensus.cc:697] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Becoming Leader. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:14.357635  2551 consensus_queue.cc:237] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.368810  2136 catalog_manager.cc:5697] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff reported cstate change: term changed from 1 to 2, leader changed from 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66) to efe356271a3d43b7867710d1e5962dff (127.1.123.65). New cstate: current_term: 2 leader_uuid: "efe356271a3d43b7867710d1e5962dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.404582  2566 raft_consensus.cc:993] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95: : Instructing follower efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:14.405387  2571 raft_consensus.cc:1081] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 LEADER]: Signalling peer efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:14.407272  2239 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:39100
I20260709 12:21:14.407783  2239 raft_consensus.cc:493] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.408094  2239 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:14.413123  2239 raft_consensus.cc:515] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:14.414876  2239 leader_election.cc:290] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 3 election: Requested vote from peers 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:21:14.415699  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:14.416041  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:14.416285  2389 raft_consensus.cc:3055] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:14.416592  2315 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:14.416651  2389 raft_consensus.cc:740] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Running, Role: LEADER
I20260709 12:21:14.417646  2389 consensus_queue.cc:260] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:14.418633  2389 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:14.422200  2315 raft_consensus.cc:2468] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 3.
I20260709 12:21:14.423563  2204 leader_election.cc:304] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:14.423813  2389 raft_consensus.cc:2468] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 3.
I20260709 12:21:14.424299  2551 raft_consensus.cc:2804] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:14.424850  2551 raft_consensus.cc:697] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 3 LEADER]: Becoming Leader. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:14.425729  2551 consensus_queue.cc:237] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:14.433434  2136 catalog_manager.cc:5697] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff reported cstate change: term changed from 2 to 3, leader changed from 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67) to efe356271a3d43b7867710d1e5962dff (127.1.123.65). New cstate: current_term: 3 leader_uuid: "efe356271a3d43b7867710d1e5962dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.630090  2572 raft_consensus.cc:993] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c: : Instructing follower efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:14.630586  2569 raft_consensus.cc:1081] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Signalling peer efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:14.632386  2239 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c1a4c73011a7449bb1e0852f1e3056d1"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:39104
I20260709 12:21:14.632925  2239 raft_consensus.cc:493] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.633224  2239 raft_consensus.cc:3060] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.638245  2239 raft_consensus.cc:515] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.639791  2239 leader_election.cc:290] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Requested vote from peers 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:14.640853  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a4c73011a7449bb1e0852f1e3056d1" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:14.641197  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a4c73011a7449bb1e0852f1e3056d1" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:14.641575  2315 raft_consensus.cc:3055] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:14.641825  2389 raft_consensus.cc:3060] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.641968  2315 raft_consensus.cc:740] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:14.642995  2315 consensus_queue.cc:260] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.644200  2315 raft_consensus.cc:3060] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.647744  2389 raft_consensus.cc:2468] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:14.649005  2204 leader_election.cc:304] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:14.649693  2315 raft_consensus.cc:2468] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:14.649832  2551 raft_consensus.cc:2804] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:14.651497  2551 raft_consensus.cc:697] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Becoming Leader. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:14.652273  2551 consensus_queue.cc:237] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.653882  2572 raft_consensus.cc:993] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c: : Instructing follower efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:14.654356  2569 raft_consensus.cc:1081] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Signalling peer efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:14.656306  2240 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "747f34920d614b61852845d2ab96deb3"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:39104
I20260709 12:21:14.656863  2240 raft_consensus.cc:493] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.657202  2240 raft_consensus.cc:3060] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.659492  2579 raft_consensus.cc:993] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95: : Instructing follower efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:14.660039  2571 raft_consensus.cc:1081] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 LEADER]: Signalling peer efe356271a3d43b7867710d1e5962dff to start an election
I20260709 12:21:14.661801  2239 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
 from {username='slave'} at 127.0.0.1:39100
I20260709 12:21:14.661343  2136 catalog_manager.cc:5697] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff reported cstate change: term changed from 1 to 2, leader changed from 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66) to efe356271a3d43b7867710d1e5962dff (127.1.123.65). New cstate: current_term: 2 leader_uuid: "efe356271a3d43b7867710d1e5962dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.662312  2239 raft_consensus.cc:493] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.662664  2239 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:14.663902  2240 raft_consensus.cc:515] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.665868  2240 leader_election.cc:290] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Requested vote from peers 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:14.668356  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:14.669034  2315 raft_consensus.cc:3055] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:14.668732  2239 raft_consensus.cc:515] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.669320  2315 raft_consensus.cc:740] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:14.670270  2315 consensus_queue.cc:260] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.670945  2239 leader_election.cc:290] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 4 election: Requested vote from peers 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:14.671537  2315 raft_consensus.cc:3060] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.671764  2314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:14.672210  2314 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:14.672133  2386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a3380e528b3434caefdaf08b8a80a04" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:14.672792  2386 raft_consensus.cc:3055] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:14.673095  2386 raft_consensus.cc:740] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Running, Role: LEADER
I20260709 12:21:14.668352  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "efe356271a3d43b7867710d1e5962dff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:14.673853  2389 raft_consensus.cc:3060] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.674031  2386 consensus_queue.cc:260] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.675247  2386 raft_consensus.cc:3060] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:14.679301  2315 raft_consensus.cc:2468] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:14.679693  2314 raft_consensus.cc:2468] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 4.
I20260709 12:21:14.680380  2204 leader_election.cc:304] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:14.681190  2389 raft_consensus.cc:2468] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 2.
I20260709 12:21:14.681355  2551 raft_consensus.cc:2804] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:14.682165  2551 raft_consensus.cc:697] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Becoming Leader. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:14.682366  2204 leader_election.cc:304] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915a285f0ece44d0b4aa1d695a3a408c, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:14.683357  2573 raft_consensus.cc:2804] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:14.683094  2551 consensus_queue.cc:237] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.685024  2573 raft_consensus.cc:697] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 4 LEADER]: Becoming Leader. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:14.686065  2573 consensus_queue.cc:237] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:14.690130  2386 raft_consensus.cc:2468] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe356271a3d43b7867710d1e5962dff in term 4.
I20260709 12:21:14.694572  2136 catalog_manager.cc:5697] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff reported cstate change: term changed from 1 to 2, leader changed from 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66) to efe356271a3d43b7867710d1e5962dff (127.1.123.65). New cstate: current_term: 2 leader_uuid: "efe356271a3d43b7867710d1e5962dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.707262  2136 catalog_manager.cc:5697] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff reported cstate change: term changed from 3 to 4, leader changed from 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67) to efe356271a3d43b7867710d1e5962dff (127.1.123.65). New cstate: current_term: 4 leader_uuid: "efe356271a3d43b7867710d1e5962dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.822321  2386 raft_consensus.cc:1275] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:14.823829  2573 consensus_queue.cc:1048] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:14.835206  2314 raft_consensus.cc:1275] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:14.836881  2551 consensus_queue.cc:1048] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:14.912379  2386 raft_consensus.cc:1275] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:14.914075  2551 consensus_queue.cc:1048] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:14.925225  2314 raft_consensus.cc:1275] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:14.927600  2551 consensus_queue.cc:1048] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:15.110526  2314 raft_consensus.cc:1275] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 4 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:15.112051  2591 consensus_queue.cc:1048] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:15.117409  2386 raft_consensus.cc:1275] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:15.119102  2591 consensus_queue.cc:1048] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:15.125020  2389 raft_consensus.cc:1275] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 4 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:15.127750  2573 consensus_queue.cc:1048] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:15.137452  2315 raft_consensus.cc:1275] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:15.137197  2314 raft_consensus.cc:1275] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:15.141825  2594 consensus_queue.cc:1048] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:15.142155  2582 consensus_queue.cc:1048] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:15.156291  2386 raft_consensus.cc:1275] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Refusing update from remote peer efe356271a3d43b7867710d1e5962dff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:15.158051  2582 consensus_queue.cc:1048] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:17.223927  1517 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:21:17.224823  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:17.232937  2239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
mode: GRACEFUL
new_leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
 from {username='slave'} at 127.0.0.1:39124
I20260709 12:21:17.233970  2239 raft_consensus.cc:606] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Received request to transfer leadership to TS 915a285f0ece44d0b4aa1d695a3a408c
I20260709 12:21:17.237998  2239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "747f34920d614b61852845d2ab96deb3"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
mode: GRACEFUL
new_leader_uuid: "8837957e7c2c4384bdc425d1f86deb95"
 from {username='slave'} at 127.0.0.1:39124
I20260709 12:21:17.238725  2239 raft_consensus.cc:606] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Received request to transfer leadership to TS 8837957e7c2c4384bdc425d1f86deb95
I20260709 12:21:17.241252  2239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
mode: GRACEFUL
new_leader_uuid: "8837957e7c2c4384bdc425d1f86deb95"
 from {username='slave'} at 127.0.0.1:39124
I20260709 12:21:17.241762  2239 raft_consensus.cc:606] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 3 LEADER]: Received request to transfer leadership to TS 8837957e7c2c4384bdc425d1f86deb95
I20260709 12:21:17.244266  2239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2"
dest_uuid: "efe356271a3d43b7867710d1e5962dff"
mode: GRACEFUL
new_leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
 from {username='slave'} at 127.0.0.1:39124
I20260709 12:21:17.244726  2239 raft_consensus.cc:606] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Received request to transfer leadership to TS 915a285f0ece44d0b4aa1d695a3a408c
I20260709 12:21:17.245997  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:21:17.246515  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:17.275755  2602 raft_consensus.cc:993] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff: : Instructing follower 915a285f0ece44d0b4aa1d695a3a408c to start an election
I20260709 12:21:17.276391  2582 raft_consensus.cc:1081] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Signalling peer 915a285f0ece44d0b4aa1d695a3a408c to start an election
I20260709 12:21:17.278113  2315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282"
dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
 from {username='slave'} at 127.0.0.1:50038
I20260709 12:21:17.278656  2315 raft_consensus.cc:493] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:17.278956  2315 raft_consensus.cc:3060] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:17.285450  2315 raft_consensus.cc:515] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:17.287746  2315 leader_election.cc:290] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 3 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:17.288823  2239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:17.288991  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14763d4c61bd457faccb12c0376a7282" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:17.289739  2389 raft_consensus.cc:3060] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:17.289863  2239 raft_consensus.cc:3055] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:17.290294  2239 raft_consensus.cc:740] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:17.291191  2239 consensus_queue.cc:260] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:17.292332  2239 raft_consensus.cc:3060] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:17.297590  2389 raft_consensus.cc:2468] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 3.
I20260709 12:21:17.297739  2602 raft_consensus.cc:993] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff: : Instructing follower 8837957e7c2c4384bdc425d1f86deb95 to start an election
I20260709 12:21:17.298403  2582 raft_consensus.cc:1081] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Signalling peer 8837957e7c2c4384bdc425d1f86deb95 to start an election
I20260709 12:21:17.299150  2279 leader_election.cc:304] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: 
I20260709 12:21:17.300098  2569 raft_consensus.cc:2804] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:17.300153  2389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "747f34920d614b61852845d2ab96deb3"
dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
 from {username='slave'} at 127.0.0.1:59636
I20260709 12:21:17.300799  2389 raft_consensus.cc:493] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:17.300997  2239 raft_consensus.cc:2468] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 3.
I20260709 12:21:17.301208  2389 raft_consensus.cc:3060] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:17.301340  2569 raft_consensus.cc:697] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 LEADER]: Becoming Leader. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:17.302603  2569 consensus_queue.cc:237] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:17.307998  2389 raft_consensus.cc:515] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:17.310812  2389 leader_election.cc:290] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 3 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:21:17.311013  2239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:17.311705  2239 raft_consensus.cc:3055] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:17.312034  2239 raft_consensus.cc:740] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:17.312000  2136 catalog_manager.cc:5697] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c reported cstate change: term changed from 2 to 3, leader changed from efe356271a3d43b7867710d1e5962dff (127.1.123.65) to 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66). New cstate: current_term: 3 leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:17.312939  2239 consensus_queue.cc:260] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:17.314016  2239 raft_consensus.cc:3060] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:17.316321  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747f34920d614b61852845d2ab96deb3" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:17.316963  2315 raft_consensus.cc:3060] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:17.322927  2239 raft_consensus.cc:2468] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 3.
I20260709 12:21:17.324205  2354 leader_election.cc:304] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, efe356271a3d43b7867710d1e5962dff; no voters: 
I20260709 12:21:17.324702  2315 raft_consensus.cc:2468] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 3.
I20260709 12:21:17.325438  2571 raft_consensus.cc:2804] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:17.327234  2571 raft_consensus.cc:697] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 LEADER]: Becoming Leader. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Running, Role: LEADER
I20260709 12:21:17.328293  2571 consensus_queue.cc:237] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:17.336954  2136 catalog_manager.cc:5697] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 reported cstate change: term changed from 2 to 3, leader changed from efe356271a3d43b7867710d1e5962dff (127.1.123.65) to 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67). New cstate: current_term: 3 leader_uuid: "8837957e7c2c4384bdc425d1f86deb95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:17.528641  2602 raft_consensus.cc:993] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff: : Instructing follower 915a285f0ece44d0b4aa1d695a3a408c to start an election
I20260709 12:21:17.529229  2582 raft_consensus.cc:1081] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Signalling peer 915a285f0ece44d0b4aa1d695a3a408c to start an election
I20260709 12:21:17.531020  2315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2"
dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
 from {username='slave'} at 127.0.0.1:50038
I20260709 12:21:17.531563  2315 raft_consensus.cc:493] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:17.531849  2315 raft_consensus.cc:3060] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:17.537261  2315 raft_consensus.cc:515] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:17.539183  2315 leader_election.cc:290] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 3 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67:39493)
I20260709 12:21:17.539443  2582 raft_consensus.cc:993] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff: : Instructing follower 8837957e7c2c4384bdc425d1f86deb95 to start an election
I20260709 12:21:17.540064  2582 raft_consensus.cc:1081] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 3 LEADER]: Signalling peer 8837957e7c2c4384bdc425d1f86deb95 to start an election
I20260709 12:21:17.540086  2239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:17.540481  2389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f751eb23354b4ebadb0dec235106f2" candidate_uuid: "915a285f0ece44d0b4aa1d695a3a408c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
I20260709 12:21:17.540787  2239 raft_consensus.cc:3055] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:17.540992  2389 raft_consensus.cc:3060] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:17.541124  2239 raft_consensus.cc:740] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:17.542102  2386 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea"
dest_uuid: "8837957e7c2c4384bdc425d1f86deb95"
 from {username='slave'} at 127.0.0.1:59636
I20260709 12:21:17.542068  2239 consensus_queue.cc:260] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:17.542713  2386 raft_consensus.cc:493] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:17.543145  2386 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:17.543423  2239 raft_consensus.cc:3060] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:17.548815  2389 raft_consensus.cc:2468] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 3.
I20260709 12:21:17.549500  2386 raft_consensus.cc:515] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:17.550203  2279 leader_election.cc:304] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: 
I20260709 12:21:17.550631  2239 raft_consensus.cc:2468] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915a285f0ece44d0b4aa1d695a3a408c in term 3.
I20260709 12:21:17.551162  2569 raft_consensus.cc:2804] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:17.551663  2569 raft_consensus.cc:697] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 LEADER]: Becoming Leader. State: Replica: 915a285f0ece44d0b4aa1d695a3a408c, State: Running, Role: LEADER
I20260709 12:21:17.552416  2386 leader_election.cc:290] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 4 election: Requested vote from peers efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239), 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587)
I20260709 12:21:17.552706  2569 consensus_queue.cc:237] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } }
I20260709 12:21:17.553195  2239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "efe356271a3d43b7867710d1e5962dff"
I20260709 12:21:17.553506  2315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2032208e6a41d6bfddab63d9e28eea" candidate_uuid: "8837957e7c2c4384bdc425d1f86deb95" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "915a285f0ece44d0b4aa1d695a3a408c"
I20260709 12:21:17.553790  2239 raft_consensus.cc:3055] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:17.554086  2315 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:17.554114  2239 raft_consensus.cc:740] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 3 LEADER]: Becoming Follower/Learner. State: Replica: efe356271a3d43b7867710d1e5962dff, State: Running, Role: LEADER
I20260709 12:21:17.555192  2239 consensus_queue.cc:260] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:17.556421  2239 raft_consensus.cc:3060] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:17.561679  2315 raft_consensus.cc:2468] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 4.
I20260709 12:21:17.561986  2136 catalog_manager.cc:5697] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c reported cstate change: term changed from 2 to 3, leader changed from efe356271a3d43b7867710d1e5962dff (127.1.123.65) to 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66). New cstate: current_term: 3 leader_uuid: "915a285f0ece44d0b4aa1d695a3a408c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:17.563153  2353 leader_election.cc:304] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8837957e7c2c4384bdc425d1f86deb95, 915a285f0ece44d0b4aa1d695a3a408c; no voters: 
I20260709 12:21:17.563727  2239 raft_consensus.cc:2468] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8837957e7c2c4384bdc425d1f86deb95 in term 4.
I20260709 12:21:17.564108  2571 raft_consensus.cc:2804] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:17.564695  2571 raft_consensus.cc:697] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 4 LEADER]: Becoming Leader. State: Replica: 8837957e7c2c4384bdc425d1f86deb95, State: Running, Role: LEADER
I20260709 12:21:17.565644  2571 consensus_queue.cc:237] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } }
I20260709 12:21:17.574980  2136 catalog_manager.cc:5697] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 reported cstate change: term changed from 3 to 4, leader changed from efe356271a3d43b7867710d1e5962dff (127.1.123.65) to 8837957e7c2c4384bdc425d1f86deb95 (127.1.123.67). New cstate: current_term: 4 leader_uuid: "8837957e7c2c4384bdc425d1f86deb95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:17.742632  2239 raft_consensus.cc:1275] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Refusing update from remote peer 8837957e7c2c4384bdc425d1f86deb95: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:17.743996  2571 consensus_queue.cc:1048] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:17.756912  2315 raft_consensus.cc:1275] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Refusing update from remote peer 8837957e7c2c4384bdc425d1f86deb95: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:17.759258  2607 consensus_queue.cc:1048] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:17.791249  2386 raft_consensus.cc:1275] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Refusing update from remote peer 915a285f0ece44d0b4aa1d695a3a408c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:17.792721  2569 consensus_queue.cc:1048] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:17.803246  2239 raft_consensus.cc:1275] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Refusing update from remote peer 915a285f0ece44d0b4aa1d695a3a408c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:17.805886  2603 consensus_queue.cc:1048] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:17.958261  2239 raft_consensus.cc:1275] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Refusing update from remote peer 915a285f0ece44d0b4aa1d695a3a408c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:17.959578  2603 consensus_queue.cc:1048] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:17.969802  2386 raft_consensus.cc:1275] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Refusing update from remote peer 915a285f0ece44d0b4aa1d695a3a408c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:17.971604  2622 consensus_queue.cc:1048] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8837957e7c2c4384bdc425d1f86deb95" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:18.029305  2239 raft_consensus.cc:1275] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 4 FOLLOWER]: Refusing update from remote peer 8837957e7c2c4384bdc425d1f86deb95: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:18.030690  2607 consensus_queue.cc:1048] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe356271a3d43b7867710d1e5962dff" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:18.041404  2315 raft_consensus.cc:1275] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 4 FOLLOWER]: Refusing update from remote peer 8837957e7c2c4384bdc425d1f86deb95: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:18.043889  2571 consensus_queue.cc:1048] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a285f0ece44d0b4aa1d695a3a408c" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:18.247601  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:18.253394  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:18.254024  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:19.255196  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:19.261263  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:19.261854  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:20.263018  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:20.268709  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:20.269284  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:21.270445  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:21.276633  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:21.277216  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:22.278381  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:22.284998  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:22.285605  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:23.286706  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:23.292928  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:23.293603  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:24.294713  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:24.300679  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:24.301235  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:25.302451  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:25.307960  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:25.308446  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:26.309453  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:26.315413  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:26.315948  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:27.350478  1517 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:21:27.351222  1517 tablet_server.cc:179] TabletServer@127.1.123.65:0 shutting down...
W20260709 12:21:27.365952  2280 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
W20260709 12:21:27.370651  2280 consensus_peers.cc:597] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c -> Peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Couldn't send request to peer efe356271a3d43b7867710d1e5962dff. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:27.385703  2280 consensus_peers.cc:597] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c -> Peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Couldn't send request to peer efe356271a3d43b7867710d1e5962dff. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:27.395773  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:27.396706  1517 tablet_replica.cc:333] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff: stopping tablet replica
I20260709 12:21:27.397938  1517 raft_consensus.cc:2243] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.398873  1517 raft_consensus.cc:2272] T 747f34920d614b61852845d2ab96deb3 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.401376  1517 tablet_replica.cc:333] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff: stopping tablet replica
I20260709 12:21:27.402115  1517 raft_consensus.cc:2243] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.403194  1517 raft_consensus.cc:2272] T e118c00462e141328b65f49b71fc2b09 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.405299  1517 tablet_replica.cc:333] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff: stopping tablet replica
I20260709 12:21:27.405901  1517 raft_consensus.cc:2243] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.406783  1517 raft_consensus.cc:2272] T a41d9df904094b35b6d61a167f8359d2 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.408764  1517 tablet_replica.cc:333] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff: stopping tablet replica
I20260709 12:21:27.409269  1517 raft_consensus.cc:2243] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.409768  1517 raft_consensus.cc:2272] T f4200fe852224b1c8f21a0bb88a0fc47 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.411635  1517 tablet_replica.cc:333] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff: stopping tablet replica
I20260709 12:21:27.412123  1517 raft_consensus.cc:2243] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.412598  1517 raft_consensus.cc:2272] T b3f751eb23354b4ebadb0dec235106f2 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.414495  1517 tablet_replica.cc:333] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff: stopping tablet replica
I20260709 12:21:27.415001  1517 raft_consensus.cc:2243] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.415866  1517 raft_consensus.cc:2272] T 2a3380e528b3434caefdaf08b8a80a04 P efe356271a3d43b7867710d1e5962dff [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.417757  1517 tablet_replica.cc:333] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff: stopping tablet replica
I20260709 12:21:27.418265  1517 raft_consensus.cc:2243] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.419101  1517 raft_consensus.cc:2272] T c1a4c73011a7449bb1e0852f1e3056d1 P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.421109  1517 tablet_replica.cc:333] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff: stopping tablet replica
I20260709 12:21:27.421693  1517 raft_consensus.cc:2243] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.422184  1517 raft_consensus.cc:2272] T ad2032208e6a41d6bfddab63d9e28eea P efe356271a3d43b7867710d1e5962dff [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.424211  1517 tablet_replica.cc:333] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff: stopping tablet replica
I20260709 12:21:27.424742  1517 raft_consensus.cc:2243] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.425263  1517 raft_consensus.cc:2272] T 2c237ebe3cce4553aa71f1dabf1abbea P efe356271a3d43b7867710d1e5962dff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.427253  1517 tablet_replica.cc:333] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff: stopping tablet replica
I20260709 12:21:27.427806  1517 raft_consensus.cc:2243] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.428341  1517 raft_consensus.cc:2272] T 14763d4c61bd457faccb12c0376a7282 P efe356271a3d43b7867710d1e5962dff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.461627  1517 tablet_server.cc:196] TabletServer@127.1.123.65:0 shutdown complete.
I20260709 12:21:27.488903  1517 tablet_server.cc:179] TabletServer@127.1.123.66:0 shutting down...
W20260709 12:21:27.511850  2354 consensus_peers.cc:597] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 -> Peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Couldn't send request to peer efe356271a3d43b7867710d1e5962dff. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:40239: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:27.530079  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:27.531026  1517 tablet_replica.cc:333] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c: stopping tablet replica
I20260709 12:21:27.532092  1517 raft_consensus.cc:2243] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.532894  1517 raft_consensus.cc:2272] T f4200fe852224b1c8f21a0bb88a0fc47 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.535522  1517 tablet_replica.cc:333] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c: stopping tablet replica
I20260709 12:21:27.536341  1517 raft_consensus.cc:2243] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 4 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:27.536887  2354 consensus_peers.cc:597] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 -> Peer efe356271a3d43b7867710d1e5962dff (127.1.123.65:40239): Couldn't send request to peer efe356271a3d43b7867710d1e5962dff. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:40239: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:27.536914  1517 raft_consensus.cc:2272] T ad2032208e6a41d6bfddab63d9e28eea P 915a285f0ece44d0b4aa1d695a3a408c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.539373  1517 tablet_replica.cc:333] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c: stopping tablet replica
I20260709 12:21:27.540004  1517 raft_consensus.cc:2243] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.540913  1517 raft_consensus.cc:2272] T 2c237ebe3cce4553aa71f1dabf1abbea P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.543005  1517 tablet_replica.cc:333] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c: stopping tablet replica
I20260709 12:21:27.543570  1517 raft_consensus.cc:2243] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.544039  1517 raft_consensus.cc:2272] T 747f34920d614b61852845d2ab96deb3 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.546602  1517 tablet_replica.cc:333] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c: stopping tablet replica
I20260709 12:21:27.547189  1517 raft_consensus.cc:2243] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.547643  1517 raft_consensus.cc:2272] T e118c00462e141328b65f49b71fc2b09 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.549670  1517 tablet_replica.cc:333] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c: stopping tablet replica
I20260709 12:21:27.550205  1517 raft_consensus.cc:2243] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.550688  1517 raft_consensus.cc:2272] T a41d9df904094b35b6d61a167f8359d2 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.552593  1517 tablet_replica.cc:333] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c: stopping tablet replica
I20260709 12:21:27.553126  1517 raft_consensus.cc:2243] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.554116  1517 raft_consensus.cc:2272] T b3f751eb23354b4ebadb0dec235106f2 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.556080  1517 tablet_replica.cc:333] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c: stopping tablet replica
I20260709 12:21:27.556609  1517 raft_consensus.cc:2243] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.557139  1517 raft_consensus.cc:2272] T 2a3380e528b3434caefdaf08b8a80a04 P 915a285f0ece44d0b4aa1d695a3a408c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.559222  1517 tablet_replica.cc:333] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c: stopping tablet replica
I20260709 12:21:27.559764  1517 raft_consensus.cc:2243] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.560222  1517 raft_consensus.cc:2272] T c1a4c73011a7449bb1e0852f1e3056d1 P 915a285f0ece44d0b4aa1d695a3a408c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.562142  1517 tablet_replica.cc:333] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c: stopping tablet replica
I20260709 12:21:27.562618  1517 raft_consensus.cc:2243] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.563527  1517 raft_consensus.cc:2272] T 14763d4c61bd457faccb12c0376a7282 P 915a285f0ece44d0b4aa1d695a3a408c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.596494  1517 tablet_server.cc:196] TabletServer@127.1.123.66:0 shutdown complete.
I20260709 12:21:27.621680  1517 tablet_server.cc:179] TabletServer@127.1.123.67:0 shutting down...
W20260709 12:21:27.650633  2353 consensus_peers.cc:597] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 -> Peer 915a285f0ece44d0b4aa1d695a3a408c (127.1.123.66:39587): Couldn't send request to peer 915a285f0ece44d0b4aa1d695a3a408c. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:39587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:27.657859  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:27.658871  1517 tablet_replica.cc:333] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95: stopping tablet replica
I20260709 12:21:27.659816  1517 raft_consensus.cc:2243] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.660889  1517 raft_consensus.cc:2272] T f4200fe852224b1c8f21a0bb88a0fc47 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.663161  1517 tablet_replica.cc:333] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95: stopping tablet replica
I20260709 12:21:27.663771  1517 raft_consensus.cc:2243] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.664304  1517 raft_consensus.cc:2272] T 14763d4c61bd457faccb12c0376a7282 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.666508  1517 tablet_replica.cc:333] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95: stopping tablet replica
I20260709 12:21:27.667140  1517 raft_consensus.cc:2243] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.667683  1517 raft_consensus.cc:2272] T c1a4c73011a7449bb1e0852f1e3056d1 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.669806  1517 tablet_replica.cc:333] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95: stopping tablet replica
I20260709 12:21:27.670362  1517 raft_consensus.cc:2243] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.670917  1517 raft_consensus.cc:2272] T 2a3380e528b3434caefdaf08b8a80a04 P 8837957e7c2c4384bdc425d1f86deb95 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.673224  1517 tablet_replica.cc:333] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95: stopping tablet replica
I20260709 12:21:27.674007  1517 raft_consensus.cc:2243] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.674741  1517 raft_consensus.cc:2272] T b3f751eb23354b4ebadb0dec235106f2 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.676927  1517 tablet_replica.cc:333] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95: stopping tablet replica
I20260709 12:21:27.677506  1517 raft_consensus.cc:2243] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.678102  1517 raft_consensus.cc:2272] T a41d9df904094b35b6d61a167f8359d2 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.680181  1517 tablet_replica.cc:333] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95: stopping tablet replica
I20260709 12:21:27.680776  1517 raft_consensus.cc:2243] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.681840  1517 raft_consensus.cc:2272] T 747f34920d614b61852845d2ab96deb3 P 8837957e7c2c4384bdc425d1f86deb95 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.683800  1517 tablet_replica.cc:333] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95: stopping tablet replica
I20260709 12:21:27.684345  1517 raft_consensus.cc:2243] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.684890  1517 raft_consensus.cc:2272] T e118c00462e141328b65f49b71fc2b09 P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.686856  1517 tablet_replica.cc:333] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95: stopping tablet replica
I20260709 12:21:27.687440  1517 raft_consensus.cc:2243] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.687927  1517 raft_consensus.cc:2272] T 2c237ebe3cce4553aa71f1dabf1abbea P 8837957e7c2c4384bdc425d1f86deb95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.689877  1517 tablet_replica.cc:333] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95: stopping tablet replica
I20260709 12:21:27.690390  1517 raft_consensus.cc:2243] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.691342  1517 raft_consensus.cc:2272] T ad2032208e6a41d6bfddab63d9e28eea P 8837957e7c2c4384bdc425d1f86deb95 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.723018  1517 tablet_server.cc:196] TabletServer@127.1.123.67:0 shutdown complete.
I20260709 12:21:27.746850  1517 master.cc:562] Master@127.1.123.126:34823 shutting down...
I20260709 12:21:27.774176  1517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.774969  1517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.775535  1517 tablet_replica.cc:333] T 00000000000000000000000000000000 P c20511a19e7245f3b8d4440a3dc7e335: stopping tablet replica
I20260709 12:21:27.814880  1517 master.cc:584] Master@127.1.123.126:34823 shutdown complete.
8837957e7c2c4384bdc425d1f86deb95  1
915a285f0ece44d0b4aa1d695a3a408c  4
efe356271a3d43b7867710d1e5962dff  5
8837957e7c2c4384bdc425d1f86deb95  3
915a285f0ece44d0b4aa1d695a3a408c  4
efe356271a3d43b7867710d1e5962dff  3
8837957e7c2c4384bdc425d1f86deb95  1
915a285f0ece44d0b4aa1d695a3a408c  1
efe356271a3d43b7867710d1e5962dff  8
8837957e7c2c4384bdc425d1f86deb95  3
915a285f0ece44d0b4aa1d695a3a408c  3
efe356271a3d43b7867710d1e5962dff  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (29147 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:21:27.863611  1517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.123.126:36731
I20260709 12:21:27.864995  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:27.871193  2642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:27.871379  2641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:27.872830  2644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:27.874238  1517 server_base.cc:1061] running on GCE node
I20260709 12:21:27.875383  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:27.875572  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:27.875694  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599687875681 us; error 0 us; skew 500 ppm
I20260709 12:21:27.876211  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:27.879462  1517 webserver.cc:533] Webserver started at http://127.1.123.126:38095/ using document root <none> and password file <none>
I20260709 12:21:27.880033  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:27.880218  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:27.880494  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:27.881961  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/master-0-root/instance:
uuid: "54239aeb1f214e00bccab640927a1e8f"
format_stamp: "Formatted at 2026-07-09 12:21:27 on dist-test-slave-08l4"
I20260709 12:21:27.886972  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:21:27.890921  2649 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:27.891780  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:21:27.892099  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/master-0-root
uuid: "54239aeb1f214e00bccab640927a1e8f"
format_stamp: "Formatted at 2026-07-09 12:21:27 on dist-test-slave-08l4"
I20260709 12:21:27.892458  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:27.911695  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:27.913166  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:27.958184  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.126:36731
I20260709 12:21:27.958281  2700 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.126:36731 every 8 connection(s)
I20260709 12:21:27.963030  2701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:27.975354  2701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f: Bootstrap starting.
I20260709 12:21:27.981025  2701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:27.986220  2701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f: No bootstrap required, opened a new log
I20260709 12:21:27.988771  2701 raft_consensus.cc:359] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54239aeb1f214e00bccab640927a1e8f" member_type: VOTER }
I20260709 12:21:27.989275  2701 raft_consensus.cc:385] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:27.989459  2701 raft_consensus.cc:740] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54239aeb1f214e00bccab640927a1e8f, State: Initialized, Role: FOLLOWER
I20260709 12:21:27.990063  2701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54239aeb1f214e00bccab640927a1e8f" member_type: VOTER }
I20260709 12:21:27.990571  2701 raft_consensus.cc:399] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:21:27.990847  2701 raft_consensus.cc:493] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:21:27.991091  2701 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:27.996371  2701 raft_consensus.cc:515] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54239aeb1f214e00bccab640927a1e8f" member_type: VOTER }
I20260709 12:21:27.996976  2701 leader_election.cc:304] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54239aeb1f214e00bccab640927a1e8f; no voters: 
I20260709 12:21:27.998234  2701 leader_election.cc:290] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:21:27.998752  2704 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:28.000218  2704 raft_consensus.cc:697] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [term 1 LEADER]: Becoming Leader. State: Replica: 54239aeb1f214e00bccab640927a1e8f, State: Running, Role: LEADER
I20260709 12:21:28.001037  2704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54239aeb1f214e00bccab640927a1e8f" member_type: VOTER }
I20260709 12:21:28.001722  2701 sys_catalog.cc:565] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:21:28.006583  2706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54239aeb1f214e00bccab640927a1e8f. Latest consensus state: current_term: 1 leader_uuid: "54239aeb1f214e00bccab640927a1e8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54239aeb1f214e00bccab640927a1e8f" member_type: VOTER } }
I20260709 12:21:28.007431  2706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:28.007992  2705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54239aeb1f214e00bccab640927a1e8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54239aeb1f214e00bccab640927a1e8f" member_type: VOTER } }
I20260709 12:21:28.008811  2705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:28.011165  2711 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:21:28.020093  2711 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:21:28.021369  1517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:21:28.030625  2711 catalog_manager.cc:1383] Generated new cluster ID: 7890913589534f2b92a194ead7ab906f
I20260709 12:21:28.031018  2711 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:21:28.082067  2711 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:21:28.083559  2711 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:21:28.097007  2711 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f: Generated new TSK 0
I20260709 12:21:28.097783  2711 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:21:28.152921  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:28.160254  2722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:28.161504  2723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:28.164013  2725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:28.164518  1517 server_base.cc:1061] running on GCE node
I20260709 12:21:28.165709  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:28.165904  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:28.166113  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599688166090 us; error 0 us; skew 500 ppm
I20260709 12:21:28.166915  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:28.169737  1517 webserver.cc:533] Webserver started at http://127.1.123.65:35213/ using document root <none> and password file <none>
I20260709 12:21:28.170286  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:28.170476  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:28.170815  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:28.172251  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/instance:
uuid: "442c92e5267046cd92f46e6f469570a5"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-08l4"
I20260709 12:21:28.177336  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:21:28.181180  2730 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.182286  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:21:28.182703  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root
uuid: "442c92e5267046cd92f46e6f469570a5"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-08l4"
I20260709 12:21:28.183128  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:28.197899  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:28.199365  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:28.200970  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:28.203749  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:28.204021  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.204278  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:28.204432  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.250579  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.65:35625
I20260709 12:21:28.250676  2792 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.65:35625 every 8 connection(s)
I20260709 12:21:28.256134  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:28.263962  2797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:28.265599  2798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:28.267287  2800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:28.269431  1517 server_base.cc:1061] running on GCE node
I20260709 12:21:28.270815  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:28.271092  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:28.271068  2793 heartbeater.cc:344] Connected to a master server at 127.1.123.126:36731
I20260709 12:21:28.271462  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599688271439 us; error 0 us; skew 500 ppm
I20260709 12:21:28.271687  2793 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:28.272172  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:28.272635  2793 heartbeater.cc:507] Master 127.1.123.126:36731 requested a full tablet report, sending...
I20260709 12:21:28.275272  2666 ts_manager.cc:194] Registered new tserver with Master: 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:21:28.275396  1517 webserver.cc:533] Webserver started at http://127.1.123.66:34499/ using document root <none> and password file <none>
I20260709 12:21:28.276027  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:28.276268  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:28.276554  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:28.277391  2666 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49702
I20260709 12:21:28.278015  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/instance:
uuid: "9a10672cd53140d59ff945f75345cc06"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-08l4"
I20260709 12:21:28.283176  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:21:28.287177  2805 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.287988  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 12:21:28.288296  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root
uuid: "9a10672cd53140d59ff945f75345cc06"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-08l4"
I20260709 12:21:28.288622  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:28.310441  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:28.311889  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:28.313632  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:28.316133  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:28.316363  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.316632  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:28.316819  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.361424  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.66:44215
I20260709 12:21:28.361738  2867 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.66:44215 every 8 connection(s)
I20260709 12:21:28.367000  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:28.375415  2871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:28.377210  2872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:28.380759  1517 server_base.cc:1061] running on GCE node
W20260709 12:21:28.380970  2874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:28.381807  2868 heartbeater.cc:344] Connected to a master server at 127.1.123.126:36731
I20260709 12:21:28.382215  2868 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:28.382234  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:28.382591  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:28.382799  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599688382776 us; error 0 us; skew 500 ppm
I20260709 12:21:28.383286  2868 heartbeater.cc:507] Master 127.1.123.126:36731 requested a full tablet report, sending...
I20260709 12:21:28.383612  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:28.385972  2666 ts_manager.cc:194] Registered new tserver with Master: 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:28.386703  1517 webserver.cc:533] Webserver started at http://127.1.123.67:36151/ using document root <none> and password file <none>
I20260709 12:21:28.387255  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:28.387477  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:28.387756  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:28.387705  2666 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49704
I20260709 12:21:28.389199  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/instance:
uuid: "7cc87046f1414b27928722690a5e388f"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-08l4"
I20260709 12:21:28.394248  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:21:28.398041  2879 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.399004  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:21:28.399544  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root
uuid: "7cc87046f1414b27928722690a5e388f"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-08l4"
I20260709 12:21:28.399892  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:28.420003  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:28.421442  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:28.423105  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:28.425828  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:28.426075  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.426335  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:28.426515  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.473112  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.67:37395
I20260709 12:21:28.473268  2941 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.67:37395 every 8 connection(s)
I20260709 12:21:28.488832  2942 heartbeater.cc:344] Connected to a master server at 127.1.123.126:36731
I20260709 12:21:28.489213  2942 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:28.490089  2942 heartbeater.cc:507] Master 127.1.123.126:36731 requested a full tablet report, sending...
I20260709 12:21:28.492357  2666 ts_manager.cc:194] Registered new tserver with Master: 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:28.492909  1517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01635316s
I20260709 12:21:28.494076  2666 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49708
I20260709 12:21:28.523582  2666 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49718:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:28.530992  2666 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:28.757762  2907 tablet_service.cc:1511] Processing CreateTablet for tablet f26ca468f5384641ad5bd05af362aef9 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:28.759539  2907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26ca468f5384641ad5bd05af362aef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.762773  2905 tablet_service.cc:1511] Processing CreateTablet for tablet bb1bafa30972440499c69fb1221eae27 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:28.764375  2905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1bafa30972440499c69fb1221eae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.764446  2758 tablet_service.cc:1511] Processing CreateTablet for tablet f26ca468f5384641ad5bd05af362aef9 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:28.765892  2758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26ca468f5384641ad5bd05af362aef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.766868  2906 tablet_service.cc:1511] Processing CreateTablet for tablet a2f8a42191e14f07a43032df18499a9c (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:28.767927  2904 tablet_service.cc:1511] Processing CreateTablet for tablet dc860efbfaf84207b7128fd001b59fe6 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:28.770577  2903 tablet_service.cc:1511] Processing CreateTablet for tablet 99bcb73e9233433a85aae0da2439d851 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:28.770819  2757 tablet_service.cc:1511] Processing CreateTablet for tablet a2f8a42191e14f07a43032df18499a9c (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:28.772148  2757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f8a42191e14f07a43032df18499a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.768378  2906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f8a42191e14f07a43032df18499a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.773861  2903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99bcb73e9233433a85aae0da2439d851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.774816  2756 tablet_service.cc:1511] Processing CreateTablet for tablet bb1bafa30972440499c69fb1221eae27 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:28.776265  2756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1bafa30972440499c69fb1221eae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.777956  2904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc860efbfaf84207b7128fd001b59fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.778815  2755 tablet_service.cc:1511] Processing CreateTablet for tablet dc860efbfaf84207b7128fd001b59fe6 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:28.781318  2902 tablet_service.cc:1511] Processing CreateTablet for tablet 2a6389f65afb4adabe1d70b6f09e99e7 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:28.783034  2902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a6389f65afb4adabe1d70b6f09e99e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.783933  2754 tablet_service.cc:1511] Processing CreateTablet for tablet 99bcb73e9233433a85aae0da2439d851 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:28.785305  2754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99bcb73e9233433a85aae0da2439d851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.788965  2752 tablet_service.cc:1511] Processing CreateTablet for tablet d500b7bb3bd042f09ffa7310e921aa24 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:28.790544  2901 tablet_service.cc:1511] Processing CreateTablet for tablet d500b7bb3bd042f09ffa7310e921aa24 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:28.791332  2833 tablet_service.cc:1511] Processing CreateTablet for tablet f26ca468f5384641ad5bd05af362aef9 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:28.791978  2901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d500b7bb3bd042f09ffa7310e921aa24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.792629  2833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26ca468f5384641ad5bd05af362aef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.794767  2753 tablet_service.cc:1511] Processing CreateTablet for tablet 2a6389f65afb4adabe1d70b6f09e99e7 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:28.795440  2900 tablet_service.cc:1511] Processing CreateTablet for tablet 3dbc0cbfcb67483f9f554976e140ba7f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:28.796818  2900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dbc0cbfcb67483f9f554976e140ba7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.797628  2752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d500b7bb3bd042f09ffa7310e921aa24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.799213  2753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a6389f65afb4adabe1d70b6f09e99e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.799453  2899 tablet_service.cc:1511] Processing CreateTablet for tablet 133424caefd34fa8b83015c9fe06d612 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:28.800869  2899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 133424caefd34fa8b83015c9fe06d612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.802788  2751 tablet_service.cc:1511] Processing CreateTablet for tablet 3dbc0cbfcb67483f9f554976e140ba7f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:28.804697  2751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dbc0cbfcb67483f9f554976e140ba7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.807353  2750 tablet_service.cc:1511] Processing CreateTablet for tablet 133424caefd34fa8b83015c9fe06d612 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:28.808835  2750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 133424caefd34fa8b83015c9fe06d612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.811023  2749 tablet_service.cc:1511] Processing CreateTablet for tablet c9f24d490dc840f1b2d87604abe412d8 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:28.812460  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9f24d490dc840f1b2d87604abe412d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.820705  2832 tablet_service.cc:1511] Processing CreateTablet for tablet a2f8a42191e14f07a43032df18499a9c (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:28.822294  2832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f8a42191e14f07a43032df18499a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.825001  2831 tablet_service.cc:1511] Processing CreateTablet for tablet bb1bafa30972440499c69fb1221eae27 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:28.826036  2755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc860efbfaf84207b7128fd001b59fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.826519  2831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1bafa30972440499c69fb1221eae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.831903  2830 tablet_service.cc:1511] Processing CreateTablet for tablet dc860efbfaf84207b7128fd001b59fe6 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:28.835441  2829 tablet_service.cc:1511] Processing CreateTablet for tablet 99bcb73e9233433a85aae0da2439d851 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:28.838790  2828 tablet_service.cc:1511] Processing CreateTablet for tablet 2a6389f65afb4adabe1d70b6f09e99e7 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:28.840744  2830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc860efbfaf84207b7128fd001b59fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.846094  2828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a6389f65afb4adabe1d70b6f09e99e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.850131  2829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99bcb73e9233433a85aae0da2439d851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.849727  2898 tablet_service.cc:1511] Processing CreateTablet for tablet c9f24d490dc840f1b2d87604abe412d8 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:28.851686  2898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9f24d490dc840f1b2d87604abe412d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.855208  2827 tablet_service.cc:1511] Processing CreateTablet for tablet d500b7bb3bd042f09ffa7310e921aa24 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:28.856693  2827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d500b7bb3bd042f09ffa7310e921aa24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.858762  2826 tablet_service.cc:1511] Processing CreateTablet for tablet 3dbc0cbfcb67483f9f554976e140ba7f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:28.860221  2826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dbc0cbfcb67483f9f554976e140ba7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.862811  2825 tablet_service.cc:1511] Processing CreateTablet for tablet 133424caefd34fa8b83015c9fe06d612 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:28.864786  2825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 133424caefd34fa8b83015c9fe06d612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.870857  2824 tablet_service.cc:1511] Processing CreateTablet for tablet c9f24d490dc840f1b2d87604abe412d8 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:28.872365  2824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9f24d490dc840f1b2d87604abe412d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.921653  2905 tablet_service.cc:1511] Processing CreateTablet for tablet c737810e34ba42deaa41e6478e84fe87 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:28.923107  2905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c737810e34ba42deaa41e6478e84fe87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.942293  2825 tablet_service.cc:1511] Processing CreateTablet for tablet c737810e34ba42deaa41e6478e84fe87 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:28.943889  2825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c737810e34ba42deaa41e6478e84fe87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.973519  2826 tablet_service.cc:1511] Processing CreateTablet for tablet 51910d09518b4757be25a91c35ee2ec2 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:28.976683  2957 tablet_bootstrap.cc:492] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:28.973374  2900 tablet_service.cc:1511] Processing CreateTablet for tablet 51910d09518b4757be25a91c35ee2ec2 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:28.987025  2900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51910d09518b4757be25a91c35ee2ec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.995304  2901 tablet_service.cc:1511] Processing CreateTablet for tablet d436a6cfd22640bea76c0256f1d01684 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:28.997937  2752 tablet_service.cc:1511] Processing CreateTablet for tablet c737810e34ba42deaa41e6478e84fe87 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:28.998770  2957 tablet_bootstrap.cc:654] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:28.999310  2752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c737810e34ba42deaa41e6478e84fe87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.992893  2903 tablet_service.cc:1511] Processing CreateTablet for tablet 62274aac902146f7b127a2afb55b8a12 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:28.996757  2901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d436a6cfd22640bea76c0256f1d01684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.002979  2898 tablet_service.cc:1511] Processing CreateTablet for tablet 532f0b78131648af8fd4930674a8bce0 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:29.004117  2903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62274aac902146f7b127a2afb55b8a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.004832  2758 tablet_service.cc:1511] Processing CreateTablet for tablet 51910d09518b4757be25a91c35ee2ec2 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
W20260709 12:21:29.020027  2714 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:29.028556  2833 tablet_service.cc:1511] Processing CreateTablet for tablet 62274aac902146f7b127a2afb55b8a12 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:29.034642  2904 tablet_service.cc:1511] Processing CreateTablet for tablet 021d84b2e9364749b9ae4e7fde0b14bc (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:29.036152  2904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 021d84b2e9364749b9ae4e7fde0b14bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.036823  2750 tablet_service.cc:1511] Processing CreateTablet for tablet 62274aac902146f7b127a2afb55b8a12 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:29.022080  2826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51910d09518b4757be25a91c35ee2ec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.047962  2898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 532f0b78131648af8fd4930674a8bce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.051829  2958 tablet_bootstrap.cc:492] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:29.056277  2907 tablet_service.cc:1511] Processing CreateTablet for tablet 41f5e1f4763f4d978619c2696e7aaa71 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:29.060207  2907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41f5e1f4763f4d978619c2696e7aaa71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.068090  2756 tablet_service.cc:1511] Processing CreateTablet for tablet d436a6cfd22640bea76c0256f1d01684 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:29.070027  2828 tablet_service.cc:1511] Processing CreateTablet for tablet 41f5e1f4763f4d978619c2696e7aaa71 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:29.071617  2754 tablet_service.cc:1511] Processing CreateTablet for tablet 532f0b78131648af8fd4930674a8bce0 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:29.061072  2831 tablet_service.cc:1511] Processing CreateTablet for tablet d436a6cfd22640bea76c0256f1d01684 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:29.081754  2758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51910d09518b4757be25a91c35ee2ec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.081511  2899 tablet_service.cc:1511] Processing CreateTablet for tablet 627b3359300e44fc9d2b83ed88272a9f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:29.082968  2899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 627b3359300e44fc9d2b83ed88272a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.060472  2832 tablet_service.cc:1511] Processing CreateTablet for tablet 532f0b78131648af8fd4930674a8bce0 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:29.063584  2833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62274aac902146f7b127a2afb55b8a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.064397  2824 tablet_service.cc:1511] Processing CreateTablet for tablet 021d84b2e9364749b9ae4e7fde0b14bc (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:29.091138  2753 tablet_service.cc:1511] Processing CreateTablet for tablet 41f5e1f4763f4d978619c2696e7aaa71 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:29.097060  2957 tablet_bootstrap.cc:492] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:29.097663  2957 ts_tablet_manager.cc:1403] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.121s	user 0.023s	sys 0.006s
I20260709 12:21:29.102406  2957 raft_consensus.cc:359] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.103334  2957 raft_consensus.cc:385] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.103694  2957 raft_consensus.cc:740] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.104250  2827 tablet_service.cc:1511] Processing CreateTablet for tablet 627b3359300e44fc9d2b83ed88272a9f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:29.104682  2957 consensus_queue.cc:260] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.105719  2750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62274aac902146f7b127a2afb55b8a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.106570  2756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d436a6cfd22640bea76c0256f1d01684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.109879  2754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 532f0b78131648af8fd4930674a8bce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.077353  2751 tablet_service.cc:1511] Processing CreateTablet for tablet 021d84b2e9364749b9ae4e7fde0b14bc (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:29.105716  2898 tablet_service.cc:1511] Processing CreateTablet for tablet 659c2c3a61024d07a2296bc83d87a52d (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:29.119045  2898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 659c2c3a61024d07a2296bc83d87a52d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.125856  2753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41f5e1f4763f4d978619c2696e7aaa71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.126236  2902 tablet_service.cc:1511] Processing CreateTablet for tablet 6855fa72e5f9486bb478855f721e77ce (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:29.127717  2902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6855fa72e5f9486bb478855f721e77ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.129081  2751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 021d84b2e9364749b9ae4e7fde0b14bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.132426  2829 tablet_service.cc:1511] Processing CreateTablet for tablet 6855fa72e5f9486bb478855f721e77ce (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:29.136164  2758 tablet_service.cc:1511] Processing CreateTablet for tablet 6855fa72e5f9486bb478855f721e77ce (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:29.142819  2906 tablet_service.cc:1511] Processing CreateTablet for tablet e3545f99312f483eb6dc8965c5bc418b (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:29.144443  2906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3545f99312f483eb6dc8965c5bc418b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.143463  2758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6855fa72e5f9486bb478855f721e77ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.148458  2905 tablet_service.cc:1511] Processing CreateTablet for tablet b46ad496aaf14cbc8cf01697dceb4cf5 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:29.150852  2827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 627b3359300e44fc9d2b83ed88272a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.154114  2959 tablet_bootstrap.cc:492] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:29.155735  2958 tablet_bootstrap.cc:654] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.153468  2828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41f5e1f4763f4d978619c2696e7aaa71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.158633  2942 heartbeater.cc:499] Master 127.1.123.126:36731 was elected leader, sending a full tablet report...
I20260709 12:21:29.162009  2824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 021d84b2e9364749b9ae4e7fde0b14bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.165866  2831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d436a6cfd22640bea76c0256f1d01684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.168322  2755 tablet_service.cc:1511] Processing CreateTablet for tablet 627b3359300e44fc9d2b83ed88272a9f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:29.168768  2749 tablet_service.cc:1511] Processing CreateTablet for tablet 659c2c3a61024d07a2296bc83d87a52d (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:29.169950  2829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6855fa72e5f9486bb478855f721e77ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.170410  2830 tablet_service.cc:1511] Processing CreateTablet for tablet 659c2c3a61024d07a2296bc83d87a52d (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:29.171752  2830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 659c2c3a61024d07a2296bc83d87a52d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.173950  2832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 532f0b78131648af8fd4930674a8bce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.177850  2755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 627b3359300e44fc9d2b83ed88272a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.181927  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 659c2c3a61024d07a2296bc83d87a52d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.182842  2957 ts_tablet_manager.cc:1434] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.085s	user 0.011s	sys 0.006s
I20260709 12:21:29.187572  2957 tablet_bootstrap.cc:492] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:29.190212  2905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46ad496aaf14cbc8cf01697dceb4cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.191633  2756 tablet_service.cc:1511] Processing CreateTablet for tablet e3545f99312f483eb6dc8965c5bc418b (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:29.197623  2756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3545f99312f483eb6dc8965c5bc418b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.211400  2901 tablet_service.cc:1511] Processing CreateTablet for tablet a66a027320bb4d04bb8735cd6fbffabf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:29.253132  2959 tablet_bootstrap.cc:654] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.265477  2957 tablet_bootstrap.cc:654] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.270221  2901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a66a027320bb4d04bb8735cd6fbffabf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.286466  2793 heartbeater.cc:499] Master 127.1.123.126:36731 was elected leader, sending a full tablet report...
I20260709 12:21:29.292273  2900 tablet_service.cc:1511] Processing CreateTablet for tablet 2594ea75a8d44503a64ea4e06916c4b3 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:29.308604  2961 raft_consensus.cc:493] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:29.309230  2961 raft_consensus.cc:515] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.312023  2961 leader_election.cc:290] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:29.319077  2957 tablet_bootstrap.cc:492] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:29.319784  2957 ts_tablet_manager.cc:1403] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.132s	user 0.010s	sys 0.021s
I20260709 12:21:29.323293  2958 tablet_bootstrap.cc:492] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:29.323848  2958 ts_tablet_manager.cc:1403] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.272s	user 0.026s	sys 0.028s
I20260709 12:21:29.323894  2957 raft_consensus.cc:359] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.324956  2957 raft_consensus.cc:385] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.325340  2957 raft_consensus.cc:740] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.326380  2957 consensus_queue.cc:260] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.332800  2957 ts_tablet_manager.cc:1434] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.007s
I20260709 12:21:29.351446  2757 tablet_service.cc:1511] Processing CreateTablet for tablet b46ad496aaf14cbc8cf01697dceb4cf5 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:29.352942  2757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46ad496aaf14cbc8cf01697dceb4cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.378453  2904 tablet_service.cc:1511] Processing CreateTablet for tablet 710edfdfdd7e4720b77f4ec27f5f41c0 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:29.358397  2900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2594ea75a8d44503a64ea4e06916c4b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.375069  2958 raft_consensus.cc:359] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:21:29.382351  2958 raft_consensus.cc:385] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.382469  2904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 710edfdfdd7e4720b77f4ec27f5f41c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.382836  2958 raft_consensus.cc:740] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.383711  2958 consensus_queue.cc:260] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:21:29.347625  2825 tablet_service.cc:1511] Processing CreateTablet for tablet e3545f99312f483eb6dc8965c5bc418b (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:29.386899  2825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3545f99312f483eb6dc8965c5bc418b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.389487  2899 tablet_service.cc:1511] Processing CreateTablet for tablet fe7ee185c0114c079f53469226dee8bf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:29.390645  2868 heartbeater.cc:499] Master 127.1.123.126:36731 was elected leader, sending a full tablet report...
I20260709 12:21:29.391222  2899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe7ee185c0114c079f53469226dee8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.397185  2831 tablet_service.cc:1511] Processing CreateTablet for tablet 34453a67223842ed98d4220533186ade (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:29.393215  2826 tablet_service.cc:1511] Processing CreateTablet for tablet a66a027320bb4d04bb8735cd6fbffabf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:29.404908  2833 tablet_service.cc:1511] Processing CreateTablet for tablet b46ad496aaf14cbc8cf01697dceb4cf5 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:29.413985  2957 tablet_bootstrap.cc:492] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:29.419696  2752 tablet_service.cc:1511] Processing CreateTablet for tablet a66a027320bb4d04bb8735cd6fbffabf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:29.433775  2752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a66a027320bb4d04bb8735cd6fbffabf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.437682  2828 tablet_service.cc:1511] Processing CreateTablet for tablet fe7ee185c0114c079f53469226dee8bf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:29.443635  2753 tablet_service.cc:1511] Processing CreateTablet for tablet 2594ea75a8d44503a64ea4e06916c4b3 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:29.445137  2753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2594ea75a8d44503a64ea4e06916c4b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.375129  2824 tablet_service.cc:1511] Processing CreateTablet for tablet 2594ea75a8d44503a64ea4e06916c4b3 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:29.455837  2750 tablet_service.cc:1511] Processing CreateTablet for tablet 710edfdfdd7e4720b77f4ec27f5f41c0 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:29.428617  2903 tablet_service.cc:1511] Processing CreateTablet for tablet 34453a67223842ed98d4220533186ade (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:29.462858  2903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34453a67223842ed98d4220533186ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.464282  2749 tablet_service.cc:1511] Processing CreateTablet for tablet fe7ee185c0114c079f53469226dee8bf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:29.432364  2831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34453a67223842ed98d4220533186ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.466221  2826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a66a027320bb4d04bb8735cd6fbffabf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.469568  2824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2594ea75a8d44503a64ea4e06916c4b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.474030  2833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46ad496aaf14cbc8cf01697dceb4cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.426923  2827 tablet_service.cc:1511] Processing CreateTablet for tablet 710edfdfdd7e4720b77f4ec27f5f41c0 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:29.478407  2827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 710edfdfdd7e4720b77f4ec27f5f41c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.459612  2959 tablet_bootstrap.cc:492] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:29.452946  2751 tablet_service.cc:1511] Processing CreateTablet for tablet 34453a67223842ed98d4220533186ade (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:29.483968  2906 tablet_service.cc:1511] Processing CreateTablet for tablet 547d5e0cde57459bbad31d06eb64d442 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:29.485384  2906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 547d5e0cde57459bbad31d06eb64d442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.495394  2754 tablet_service.cc:1511] Processing CreateTablet for tablet 547d5e0cde57459bbad31d06eb64d442 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:29.502725  2902 tablet_service.cc:1511] Processing CreateTablet for tablet 2cba50a5280d4b00b9d2617ce7a22ec1 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:29.504333  2902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cba50a5280d4b00b9d2617ce7a22ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.504081  2832 tablet_service.cc:1511] Processing CreateTablet for tablet 547d5e0cde57459bbad31d06eb64d442 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:29.505946  2898 tablet_service.cc:1511] Processing CreateTablet for tablet a9961074150e45dbb36db8e49d840c5d (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:29.507274  2898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9961074150e45dbb36db8e49d840c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.508853  2907 tablet_service.cc:1511] Processing CreateTablet for tablet 1a061e32042842a9a967750b68aa8abf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:29.510136  2907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a061e32042842a9a967750b68aa8abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.516970  2957 tablet_bootstrap.cc:654] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.517729  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe7ee185c0114c079f53469226dee8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.516582  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1bafa30972440499c69fb1221eae27" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
W20260709 12:21:29.519387  2880 leader_election.cc:343] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:29.484812  2905 tablet_service.cc:1511] Processing CreateTablet for tablet 6979a63415d34b6d91bb669ecec01bdf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:29.530731  2905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6979a63415d34b6d91bb669ecec01bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.479090  2959 ts_tablet_manager.cc:1403] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.325s	user 0.012s	sys 0.004s
I20260709 12:21:29.546888  2831 tablet_service.cc:1511] Processing CreateTablet for tablet 1a061e32042842a9a967750b68aa8abf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:29.550333  2829 tablet_service.cc:1511] Processing CreateTablet for tablet 6979a63415d34b6d91bb669ecec01bdf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:29.567844  2959 raft_consensus.cc:359] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.568737  2959 raft_consensus.cc:385] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.569095  2959 raft_consensus.cc:740] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.569851  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1bafa30972440499c69fb1221eae27" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:29.569952  2959 consensus_queue.cc:260] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.578147  2750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 710edfdfdd7e4720b77f4ec27f5f41c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.578781  2828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe7ee185c0114c079f53469226dee8bf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:29.578523  2881 leader_election.cc:343] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:29.579650  2832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 547d5e0cde57459bbad31d06eb64d442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.580375  2754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 547d5e0cde57459bbad31d06eb64d442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.579651  2757 tablet_service.cc:1511] Processing CreateTablet for tablet 6979a63415d34b6d91bb669ecec01bdf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:29.582723  2757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6979a63415d34b6d91bb669ecec01bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.583267  2758 tablet_service.cc:1511] Processing CreateTablet for tablet 1a061e32042842a9a967750b68aa8abf (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:29.584811  2758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a061e32042842a9a967750b68aa8abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.586076  2751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34453a67223842ed98d4220533186ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.588687  2881 leader_election.cc:304] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:29.591948  2831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a061e32042842a9a967750b68aa8abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.592555  2961 raft_consensus.cc:2749] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:29.600466  2904 tablet_service.cc:1511] Processing CreateTablet for tablet 01e8aac6ebd14bf4b952c62c8075f13f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:29.602056  2904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e8aac6ebd14bf4b952c62c8075f13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.609704  2829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6979a63415d34b6d91bb669ecec01bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.614570  2958 ts_tablet_manager.cc:1434] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.290s	user 0.060s	sys 0.076s
I20260709 12:21:29.615855  2958 tablet_bootstrap.cc:492] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:29.615875  2969 raft_consensus.cc:493] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:29.616737  2969 raft_consensus.cc:515] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
W20260709 12:21:29.623467  2651 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 53500.6us
I20260709 12:21:29.630226  2969 leader_election.cc:290] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:29.636145  2757 tablet_service.cc:1511] Processing CreateTablet for tablet 01e8aac6ebd14bf4b952c62c8075f13f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:29.677558  2757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e8aac6ebd14bf4b952c62c8075f13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.648032  2830 tablet_service.cc:1511] Processing CreateTablet for tablet 2cba50a5280d4b00b9d2617ce7a22ec1 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:29.656391  2901 tablet_service.cc:1511] Processing CreateTablet for tablet 7e4cde589c024472b0a4cec236ea69a3 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:29.682864  2830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cba50a5280d4b00b9d2617ce7a22ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.684634  2749 tablet_service.cc:1511] Processing CreateTablet for tablet fe9fc4cceb94485ca488ce52e6d94e25 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:29.625488  2756 tablet_service.cc:1511] Processing CreateTablet for tablet 2cba50a5280d4b00b9d2617ce7a22ec1 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:29.682864  2901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e4cde589c024472b0a4cec236ea69a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.686120  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe9fc4cceb94485ca488ce52e6d94e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.685000  2752 tablet_service.cc:1511] Processing CreateTablet for tablet 7e4cde589c024472b0a4cec236ea69a3 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:29.694631  2752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e4cde589c024472b0a4cec236ea69a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.701921  2756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cba50a5280d4b00b9d2617ce7a22ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.704640  2758 tablet_service.cc:1511] Processing CreateTablet for tablet 0a1a771f5b2742d8bd641ec0afec2cf4 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:29.656195  2958 tablet_bootstrap.cc:654] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.706072  2758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1a771f5b2742d8bd641ec0afec2cf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.739961  2900 tablet_service.cc:1511] Processing CreateTablet for tablet fe9fc4cceb94485ca488ce52e6d94e25 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:29.727520  2753 tablet_service.cc:1511] Processing CreateTablet for tablet a9961074150e45dbb36db8e49d840c5d (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:29.746819  2753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9961074150e45dbb36db8e49d840c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.751271  2906 tablet_service.cc:1511] Processing CreateTablet for tablet 0a1a771f5b2742d8bd641ec0afec2cf4 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:29.741441  2900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe9fc4cceb94485ca488ce52e6d94e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.758945  2903 tablet_service.cc:1511] Processing CreateTablet for tablet 4ff32c54020945d99bd67b7df9574bae (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:29.761421  2903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff32c54020945d99bd67b7df9574bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.770241  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:29.772743  2807 leader_election.cc:343] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:29.775961  2961 raft_consensus.cc:493] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:29.779373  2825 tablet_service.cc:1511] Processing CreateTablet for tablet 7e4cde589c024472b0a4cec236ea69a3 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:29.779389  2961 raft_consensus.cc:515] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.780933  2825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e4cde589c024472b0a4cec236ea69a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.782611  2961 leader_election.cc:290] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:29.783896  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
W20260709 12:21:29.785740  2880 leader_election.cc:343] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:29.785946  2906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1a771f5b2742d8bd641ec0afec2cf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.787637  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:29.761083  2899 tablet_service.cc:1511] Processing CreateTablet for tablet 53752656656243c3abe78c7a9cb488c7 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
W20260709 12:21:29.789366  2881 leader_election.cc:343] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:29.789847  2881 leader_election.cc:304] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:29.791097  2961 raft_consensus.cc:2749] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:29.778028  2959 ts_tablet_manager.cc:1434] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.244s	user 0.014s	sys 0.004s
I20260709 12:21:29.795886  2826 tablet_service.cc:1511] Processing CreateTablet for tablet fe9fc4cceb94485ca488ce52e6d94e25 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:29.797356  2826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe9fc4cceb94485ca488ce52e6d94e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.797061  2904 tablet_service.cc:1511] Processing CreateTablet for tablet 22aad5981ec94ff2a74fdf45111c0692 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:29.801874  2957 tablet_bootstrap.cc:492] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:29.802402  2957 ts_tablet_manager.cc:1403] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.442s	user 0.032s	sys 0.046s
I20260709 12:21:29.802654  2958 tablet_bootstrap.cc:492] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:29.803156  2958 ts_tablet_manager.cc:1403] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.188s	user 0.015s	sys 0.005s
I20260709 12:21:29.805332  2829 tablet_service.cc:1511] Processing CreateTablet for tablet a9961074150e45dbb36db8e49d840c5d (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:29.806748  2902 tablet_service.cc:1511] Processing CreateTablet for tablet 1f0df81094e2493cbe804d2dba9aa9d4 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:29.810942  2754 tablet_service.cc:1511] Processing CreateTablet for tablet 53752656656243c3abe78c7a9cb488c7 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:29.809397  2959 tablet_bootstrap.cc:492] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:29.814378  2751 tablet_service.cc:1511] Processing CreateTablet for tablet 4ff32c54020945d99bd67b7df9574bae (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:29.831278  2754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53752656656243c3abe78c7a9cb488c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.832060  2751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff32c54020945d99bd67b7df9574bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.832157  2905 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa6b7efbb774150809e590d3dde361f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:29.816215  2755 tablet_service.cc:1511] Processing CreateTablet for tablet 1f0df81094e2493cbe804d2dba9aa9d4 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:29.835510  2755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f0df81094e2493cbe804d2dba9aa9d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.836658  2958 raft_consensus.cc:359] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.837479  2958 raft_consensus.cc:385] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.837802  2958 raft_consensus.cc:740] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.806749  2907 tablet_service.cc:1511] Processing CreateTablet for tablet 6487a298e2bb425bbfe83bed12805837 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:29.819316  2832 tablet_service.cc:1511] Processing CreateTablet for tablet 53752656656243c3abe78c7a9cb488c7 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:29.821960  2828 tablet_service.cc:1511] Processing CreateTablet for tablet 0a1a771f5b2742d8bd641ec0afec2cf4 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:29.822645  2899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53752656656243c3abe78c7a9cb488c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.824276  2749 tablet_service.cc:1511] Processing CreateTablet for tablet 22aad5981ec94ff2a74fdf45111c0692 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:29.841568  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22aad5981ec94ff2a74fdf45111c0692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.845733  2757 tablet_service.cc:1511] Processing CreateTablet for tablet 6487a298e2bb425bbfe83bed12805837 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:29.824600  2752 tablet_service.cc:1511] Processing CreateTablet for tablet 7bb611b2490b4682a0c5a45f2bbe7c94 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:29.847031  2757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6487a298e2bb425bbfe83bed12805837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.849961  2902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f0df81094e2493cbe804d2dba9aa9d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.850898  2832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53752656656243c3abe78c7a9cb488c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.853060  2828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1a771f5b2742d8bd641ec0afec2cf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.854063  2831 tablet_service.cc:1511] Processing CreateTablet for tablet 1f0df81094e2493cbe804d2dba9aa9d4 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:29.849458  2959 tablet_bootstrap.cc:654] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.855370  2831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f0df81094e2493cbe804d2dba9aa9d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.824276  2824 tablet_service.cc:1511] Processing CreateTablet for tablet 4ff32c54020945d99bd67b7df9574bae (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:29.858188  2898 tablet_service.cc:1511] Processing CreateTablet for tablet 7bb611b2490b4682a0c5a45f2bbe7c94 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:29.858764  2824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff32c54020945d99bd67b7df9574bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.859560  2898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bb611b2490b4682a0c5a45f2bbe7c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.861876  2905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa6b7efbb774150809e590d3dde361f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.862242  2830 tablet_service.cc:1511] Processing CreateTablet for tablet 22aad5981ec94ff2a74fdf45111c0692 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:29.864101  2829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9961074150e45dbb36db8e49d840c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.865912  2752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bb611b2490b4682a0c5a45f2bbe7c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.865754  2957 raft_consensus.cc:359] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.782137  2827 tablet_service.cc:1511] Processing CreateTablet for tablet 01e8aac6ebd14bf4b952c62c8075f13f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:29.869789  2958 consensus_queue.cc:260] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.871399  2830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22aad5981ec94ff2a74fdf45111c0692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.874775  2907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6487a298e2bb425bbfe83bed12805837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.872383  2957 raft_consensus.cc:385] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.878291  2957 raft_consensus.cc:740] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.889818  2904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22aad5981ec94ff2a74fdf45111c0692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.894816  2750 tablet_service.cc:1511] Processing CreateTablet for tablet 56d2861f96e247a78ce40f9dfe9fe850 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:29.896380  2750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56d2861f96e247a78ce40f9dfe9fe850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.898154  2827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e8aac6ebd14bf4b952c62c8075f13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.897208  2957 consensus_queue.cc:260] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.912868  2957 ts_tablet_manager.cc:1434] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.110s	user 0.008s	sys 0.000s
I20260709 12:21:29.903677  2958 ts_tablet_manager.cc:1434] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.100s	user 0.007s	sys 0.000s
I20260709 12:21:29.916987  2957 tablet_bootstrap.cc:492] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:29.928071  2959 tablet_bootstrap.cc:492] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:29.928659  2959 ts_tablet_manager.cc:1403] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.120s	user 0.019s	sys 0.011s
I20260709 12:21:29.931907  2959 raft_consensus.cc:359] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:29.932720  2959 raft_consensus.cc:385] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.933023  2959 raft_consensus.cc:740] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.934574  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:29.935406  2959 consensus_queue.cc:260] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:29.936254  2807 leader_election.cc:343] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:29.936774  2807 leader_election.cc:304] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:29.907188  2833 tablet_service.cc:1511] Processing CreateTablet for tablet 7bb611b2490b4682a0c5a45f2bbe7c94 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:29.899658  2901 tablet_service.cc:1511] Processing CreateTablet for tablet 56d2861f96e247a78ce40f9dfe9fe850 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:29.939747  2833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bb611b2490b4682a0c5a45f2bbe7c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.940793  2901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56d2861f96e247a78ce40f9dfe9fe850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.906173  2756 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa6b7efbb774150809e590d3dde361f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:29.946868  2756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa6b7efbb774150809e590d3dde361f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.919072  2958 tablet_bootstrap.cc:492] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:29.953280  2959 ts_tablet_manager.cc:1434] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 12:21:29.957396  2905 tablet_service.cc:1511] Processing CreateTablet for tablet 386a0166b36244f789cc8361a1cb0a46 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:29.959301  2905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 386a0166b36244f789cc8361a1cb0a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.970259  2959 tablet_bootstrap.cc:492] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:29.973893  2969 raft_consensus.cc:2749] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:29.977061  2959 tablet_bootstrap.cc:654] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:29.992924  2943 tablet.cc:2406] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:21:29.994730  2753 tablet_service.cc:1511] Processing CreateTablet for tablet 386a0166b36244f789cc8361a1cb0a46 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:29.996281  2753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 386a0166b36244f789cc8361a1cb0a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.008925  2957 tablet_bootstrap.cc:654] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.010017  2906 tablet_service.cc:1511] Processing CreateTablet for tablet 12034fdcbbed4121a134d5d26c1dc5ac (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:30.011523  2906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12034fdcbbed4121a134d5d26c1dc5ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.994827  2826 tablet_service.cc:1511] Processing CreateTablet for tablet 6487a298e2bb425bbfe83bed12805837 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:30.019057  2828 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa6b7efbb774150809e590d3dde361f (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:30.020491  2828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa6b7efbb774150809e590d3dde361f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:30.022760  2714 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:30.031817  2826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6487a298e2bb425bbfe83bed12805837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.053939  2827 tablet_service.cc:1511] Processing CreateTablet for tablet 56d2861f96e247a78ce40f9dfe9fe850 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:30.056514  2827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56d2861f96e247a78ce40f9dfe9fe850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.056233  2755 tablet_service.cc:1511] Processing CreateTablet for tablet e80bb6a87c6e403fa76954737ffda91a (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:30.057585  2755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e80bb6a87c6e403fa76954737ffda91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.058851  2972 raft_consensus.cc:493] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.059422  2972 raft_consensus.cc:515] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.059355  2899 tablet_service.cc:1511] Processing CreateTablet for tablet e80bb6a87c6e403fa76954737ffda91a (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:30.060967  2825 tablet_service.cc:1511] Processing CreateTablet for tablet 386a0166b36244f789cc8361a1cb0a46 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:30.061821  2831 tablet_service.cc:1511] Processing CreateTablet for tablet 12034fdcbbed4121a134d5d26c1dc5ac (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:30.062347  2825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 386a0166b36244f789cc8361a1cb0a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.063095  2831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12034fdcbbed4121a134d5d26c1dc5ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.053021  2751 tablet_service.cc:1511] Processing CreateTablet for tablet 12034fdcbbed4121a134d5d26c1dc5ac (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:30.064220  2751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12034fdcbbed4121a134d5d26c1dc5ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.066689  2959 tablet_bootstrap.cc:492] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:30.067199  2959 ts_tablet_manager.cc:1403] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.097s	user 0.005s	sys 0.011s
I20260709 12:21:30.070407  2959 raft_consensus.cc:359] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:21:30.077163  2899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e80bb6a87c6e403fa76954737ffda91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.071111  2959 raft_consensus.cc:385] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.077669  2959 raft_consensus.cc:740] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.078498  2959 consensus_queue.cc:260] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:21:30.087123  2972 leader_election.cc:290] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.097244  2958 tablet_bootstrap.cc:654] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.105859  2824 tablet_service.cc:1511] Processing CreateTablet for tablet e80bb6a87c6e403fa76954737ffda91a (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:30.119002  2824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e80bb6a87c6e403fa76954737ffda91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.131799  2832 tablet_service.cc:1511] Processing CreateTablet for tablet cf19d36b4d1b4825a45f6749aaa19896 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:30.143224  2902 tablet_service.cc:1511] Processing CreateTablet for tablet cf19d36b4d1b4825a45f6749aaa19896 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:30.147861  2826 tablet_service.cc:1511] Processing CreateTablet for tablet e07072d40c8b441e8c50e662b2daf24c (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:30.164068  2957 tablet_bootstrap.cc:492] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:30.166707  2972 raft_consensus.cc:493] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.169035  2754 tablet_service.cc:1511] Processing CreateTablet for tablet cf19d36b4d1b4825a45f6749aaa19896 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:30.179090  2959 ts_tablet_manager.cc:1434] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.112s	user 0.029s	sys 0.011s
I20260709 12:21:30.179605  2902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf19d36b4d1b4825a45f6749aaa19896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.173116  2972 raft_consensus.cc:515] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.181733  2972 leader_election.cc:290] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.175478  2957 ts_tablet_manager.cc:1403] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.258s	user 0.018s	sys 0.012s
I20260709 12:21:30.182242  2826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e07072d40c8b441e8c50e662b2daf24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.188493  2905 tablet_service.cc:1511] Processing CreateTablet for tablet 63fa205fa4864ef4b19bda073c06a5d4 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:30.190289  2905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63fa205fa4864ef4b19bda073c06a5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.191668  2832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf19d36b4d1b4825a45f6749aaa19896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.196729  2754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf19d36b4d1b4825a45f6749aaa19896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.199846  2907 tablet_service.cc:1511] Processing CreateTablet for tablet e07072d40c8b441e8c50e662b2daf24c (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:30.212842  2833 tablet_service.cc:1511] Processing CreateTablet for tablet fa220c2195724c24b522d44636d98970 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:30.216130  2903 tablet_service.cc:1511] Processing CreateTablet for tablet fa220c2195724c24b522d44636d98970 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:30.213620  2959 tablet_bootstrap.cc:492] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:30.222970  2958 tablet_bootstrap.cc:492] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:30.253990  2958 ts_tablet_manager.cc:1403] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.335s	user 0.026s	sys 0.041s
I20260709 12:21:30.257195  2958 raft_consensus.cc:359] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.257979  2958 raft_consensus.cc:385] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.258255  2958 raft_consensus.cc:740] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.225703  2907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e07072d40c8b441e8c50e662b2daf24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.258739  2957 raft_consensus.cc:359] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.259502  2957 raft_consensus.cc:385] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.259824  2957 raft_consensus.cc:740] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.260666  2827 tablet_service.cc:1511] Processing CreateTablet for tablet 9164313f06e045c3bdef4a46d365873c (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:30.260761  2957 consensus_queue.cc:260] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.262185  2827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9164313f06e045c3bdef4a46d365873c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.231027  2757 tablet_service.cc:1511] Processing CreateTablet for tablet e07072d40c8b441e8c50e662b2daf24c (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:30.219890  2831 tablet_service.cc:1511] Processing CreateTablet for tablet a7c3a03a1304452ebc48e986f5629037 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:30.265888  2830 tablet_service.cc:1511] Processing CreateTablet for tablet 63fa205fa4864ef4b19bda073c06a5d4 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:30.266393  2757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e07072d40c8b441e8c50e662b2daf24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.266613  2831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7c3a03a1304452ebc48e986f5629037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.267194  2830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63fa205fa4864ef4b19bda073c06a5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.269728  2958 consensus_queue.cc:260] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.272498  2958 ts_tablet_manager.cc:1434] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:21:30.273689  2958 tablet_bootstrap.cc:492] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:30.243484  2898 tablet_service.cc:1511] Processing CreateTablet for tablet 9164313f06e045c3bdef4a46d365873c (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:30.247869  2904 tablet_service.cc:1511] Processing CreateTablet for tablet 42af6798b62746449070b7b2bc4e0699 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:30.278432  2904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42af6798b62746449070b7b2bc4e0699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.280333  2969 raft_consensus.cc:493] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.280884  2969 raft_consensus.cc:515] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.283031  2969 leader_election.cc:290] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.284878  2959 tablet_bootstrap.cc:654] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.233834  2901 tablet_service.cc:1511] Processing CreateTablet for tablet a7c3a03a1304452ebc48e986f5629037 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:30.252223  2900 tablet_service.cc:1511] Processing CreateTablet for tablet 0e4b953250214a6f8f3b4794fab90ce0 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:30.285462  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133424caefd34fa8b83015c9fe06d612" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:30.286211  2900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e4b953250214a6f8f3b4794fab90ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.286731  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133424caefd34fa8b83015c9fe06d612" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:30.288066  2901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7c3a03a1304452ebc48e986f5629037. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:30.288492  2807 leader_election.cc:343] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.288816  2833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa220c2195724c24b522d44636d98970. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:30.289225  2807 leader_election.cc:343] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.289687  2807 leader_election.cc:304] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:30.289971  2903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa220c2195724c24b522d44636d98970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.290911  2969 raft_consensus.cc:2749] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:30.304035  2898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9164313f06e045c3bdef4a46d365873c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.305411  2905 tablet_service.cc:1511] Processing CreateTablet for tablet f1605919970c4ce68b488b932d3e81b3 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:30.306989  2905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1605919970c4ce68b488b932d3e81b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.301311  2824 tablet_service.cc:1511] Processing CreateTablet for tablet f1605919970c4ce68b488b932d3e81b3 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:30.311879  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a6389f65afb4adabe1d70b6f09e99e7" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:30.312623  2829 tablet_service.cc:1511] Processing CreateTablet for tablet 0e4b953250214a6f8f3b4794fab90ce0 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:30.313359  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:30.315146  2961 raft_consensus.cc:493] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.315675  2961 raft_consensus.cc:515] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:30.318028  2732 leader_election.cc:343] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:30.318838  2732 leader_election.cc:343] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.320765  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:30.321015  2961 leader_election.cc:290] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:30.312172  2824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1605919970c4ce68b488b932d3e81b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.323305  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
W20260709 12:21:30.324473  2881 leader_election.cc:343] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:30.324999  2880 leader_election.cc:343] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.325507  2880 leader_election.cc:304] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:30.325932  2957 ts_tablet_manager.cc:1434] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.143s	user 0.007s	sys 0.003s
I20260709 12:21:30.326776  2961 raft_consensus.cc:2749] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:30.327418  2957 tablet_bootstrap.cc:492] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:30.327666  2829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e4b953250214a6f8f3b4794fab90ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.328322  2958 tablet_bootstrap.cc:654] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.332429  2906 tablet_service.cc:1511] Processing CreateTablet for tablet 15476716416b4e729148408ed7846297 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:30.332816  2902 tablet_service.cc:1511] Processing CreateTablet for tablet eface8ee310347d4bb08e3c5864ce5d9 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:30.334147  2902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eface8ee310347d4bb08e3c5864ce5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.334591  2899 tablet_service.cc:1511] Processing CreateTablet for tablet 6291f8f287034586b4e9c0c9220bc6e7 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:30.335897  2899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6291f8f287034586b4e9c0c9220bc6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.339774  2828 tablet_service.cc:1511] Processing CreateTablet for tablet 6291f8f287034586b4e9c0c9220bc6e7 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:30.341172  2828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6291f8f287034586b4e9c0c9220bc6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.341917  2826 tablet_service.cc:1511] Processing CreateTablet for tablet 42af6798b62746449070b7b2bc4e0699 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:30.342348  2751 tablet_service.cc:1511] Processing CreateTablet for tablet 9164313f06e045c3bdef4a46d365873c (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:30.342648  2753 tablet_service.cc:1511] Processing CreateTablet for tablet a7c3a03a1304452ebc48e986f5629037 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:30.343287  2826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42af6798b62746449070b7b2bc4e0699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.343654  2751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9164313f06e045c3bdef4a46d365873c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.344242  2753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7c3a03a1304452ebc48e986f5629037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.342350  2756 tablet_service.cc:1511] Processing CreateTablet for tablet 42af6798b62746449070b7b2bc4e0699 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:30.345964  2756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42af6798b62746449070b7b2bc4e0699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.341346  2755 tablet_service.cc:1511] Processing CreateTablet for tablet fa220c2195724c24b522d44636d98970 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:30.341331  2750 tablet_service.cc:1511] Processing CreateTablet for tablet 63fa205fa4864ef4b19bda073c06a5d4 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:30.348670  2755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa220c2195724c24b522d44636d98970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.349943  2906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15476716416b4e729148408ed7846297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.349426  2750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63fa205fa4864ef4b19bda073c06a5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.367394  2752 tablet_service.cc:1511] Processing CreateTablet for tablet 0e4b953250214a6f8f3b4794fab90ce0 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:30.369297  2752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e4b953250214a6f8f3b4794fab90ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.374931  2969 raft_consensus.cc:493] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.375269  2758 tablet_service.cc:1511] Processing CreateTablet for tablet f1605919970c4ce68b488b932d3e81b3 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:30.375480  2969 raft_consensus.cc:515] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.376664  2758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1605919970c4ce68b488b932d3e81b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.378459  2969 leader_election.cc:290] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.378938  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:30.380173  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
W20260709 12:21:30.380559  2807 leader_election.cc:343] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.380823  2916 raft_consensus.cc:2468] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:30.382169  2807 leader_election.cc:304] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:30.383092  2749 tablet_service.cc:1511] Processing CreateTablet for tablet 6291f8f287034586b4e9c0c9220bc6e7 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:30.383770  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a6389f65afb4adabe1d70b6f09e99e7" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:30.384495  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6291f8f287034586b4e9c0c9220bc6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.385710  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:30.386677  2969 raft_consensus.cc:2804] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:30.387049  2969 raft_consensus.cc:493] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:30.387374  2969 raft_consensus.cc:3060] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.395989  2754 tablet_service.cc:1511] Processing CreateTablet for tablet 15476716416b4e729148408ed7846297 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
W20260709 12:21:30.397964  2731 leader_election.cc:343] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.398476  2731 leader_election.cc:304] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:30.399490  2980 raft_consensus.cc:2749] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:30.400933  2731 leader_election.cc:343] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.401377  2731 leader_election.cc:304] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:30.402377  2980 raft_consensus.cc:2749] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:30.411224  2957 tablet_bootstrap.cc:654] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.419700  2969 raft_consensus.cc:515] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.397405  2754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15476716416b4e729148408ed7846297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.449806  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:30.451258  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
W20260709 12:21:30.452222  2807 leader_election.cc:343] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.451884  2758 tablet_service.cc:1511] Processing CreateTablet for tablet eface8ee310347d4bb08e3c5864ce5d9 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:30.451911  2916 raft_consensus.cc:3060] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.459661  2758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eface8ee310347d4bb08e3c5864ce5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.466109  2824 tablet_service.cc:1511] Processing CreateTablet for tablet 15476716416b4e729148408ed7846297 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:30.467643  2824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15476716416b4e729148408ed7846297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.461323  2969 leader_election.cc:290] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.487030  2832 tablet_service.cc:1511] Processing CreateTablet for tablet eface8ee310347d4bb08e3c5864ce5d9 (DEFAULT_TABLE table=test-workload [id=6b661bc00226474da0908bf962632d5b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:30.488588  2832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eface8ee310347d4bb08e3c5864ce5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.492410  2958 tablet_bootstrap.cc:492] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:30.492986  2958 ts_tablet_manager.cc:1403] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.220s	user 0.015s	sys 0.011s
I20260709 12:21:30.496471  2958 raft_consensus.cc:359] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.479365  2916 raft_consensus.cc:2468] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:30.498798  2807 leader_election.cc:304] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:30.500031  2969 raft_consensus.cc:2804] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:30.497251  2958 raft_consensus.cc:385] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.507123  2958 raft_consensus.cc:740] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.508180  2958 consensus_queue.cc:260] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.531773  2980 raft_consensus.cc:493] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.532369  2980 raft_consensus.cc:515] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:21:30.535887  2958 ts_tablet_manager.cc:1434] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.043s	user 0.002s	sys 0.019s
I20260709 12:21:30.536048  2980 leader_election.cc:290] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:30.537065  2958 tablet_bootstrap.cc:492] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:30.536805  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:30.539886  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:30.540518  2959 tablet_bootstrap.cc:492] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:30.540563  2843 raft_consensus.cc:2468] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:30.541263  2959 ts_tablet_manager.cc:1403] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.328s	user 0.015s	sys 0.008s
W20260709 12:21:30.542250  2732 leader_election.cc:343] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.543467  2731 leader_election.cc:304] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:30.544741  2980 raft_consensus.cc:2804] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:30.545168  2980 raft_consensus.cc:493] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:30.545616  2980 raft_consensus.cc:3060] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.533890  2969 raft_consensus.cc:697] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:30.550769  2969 consensus_queue.cc:237] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.551379  2961 raft_consensus.cc:493] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.551981  2961 raft_consensus.cc:515] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.552349  2959 raft_consensus.cc:359] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.553059  2959 raft_consensus.cc:385] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.553524  2959 raft_consensus.cc:740] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.553985  2958 tablet_bootstrap.cc:654] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.555723  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99bcb73e9233433a85aae0da2439d851" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:30.558028  2961 leader_election.cc:290] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:30.559949  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99bcb73e9233433a85aae0da2439d851" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
W20260709 12:21:30.560289  2881 leader_election.cc:343] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.559834  2957 tablet_bootstrap.cc:492] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:30.559785  2959 consensus_queue.cc:260] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.561291  2957 ts_tablet_manager.cc:1403] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.234s	user 0.012s	sys 0.014s
W20260709 12:21:30.561703  2880 leader_election.cc:343] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.562215  2880 leader_election.cc:304] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:30.563330  2961 raft_consensus.cc:2749] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:30.568208  2980 raft_consensus.cc:515] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:21:30.571412  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:30.564646  2957 raft_consensus.cc:359] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.572536  2957 raft_consensus.cc:385] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.572986  2957 raft_consensus.cc:740] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
W20260709 12:21:30.573570  2732 leader_election.cc:343] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.574570  2980 leader_election.cc:290] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:30.574132  2957 consensus_queue.cc:260] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.575843  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:30.576450  2843 raft_consensus.cc:3060] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.577574  2957 ts_tablet_manager.cc:1434] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.001s
I20260709 12:21:30.579814  2959 ts_tablet_manager.cc:1434] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:21:30.582108  2957 tablet_bootstrap.cc:492] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:30.585839  2843 raft_consensus.cc:2468] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:30.587322  2731 leader_election.cc:304] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:30.588593  2980 raft_consensus.cc:2804] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:30.589238  2980 raft_consensus.cc:697] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:30.590358  2980 consensus_queue.cc:237] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:21:30.591446  2959 tablet_bootstrap.cc:492] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:30.600265  2958 tablet_bootstrap.cc:492] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:30.600947  2958 ts_tablet_manager.cc:1403] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.064s	user 0.010s	sys 0.002s
I20260709 12:21:30.604278  2958 raft_consensus.cc:359] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.605125  2958 raft_consensus.cc:385] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.605499  2958 raft_consensus.cc:740] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.606427  2958 consensus_queue.cc:260] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.606866  2664 catalog_manager.cc:5697] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:30.607405  2665 catalog_manager.cc:5697] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:30.608898  2958 ts_tablet_manager.cc:1434] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:30.609305  2959 tablet_bootstrap.cc:654] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.610057  2958 tablet_bootstrap.cc:492] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:30.607151  2957 tablet_bootstrap.cc:654] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.619796  2958 tablet_bootstrap.cc:654] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.632107  2959 tablet_bootstrap.cc:492] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:30.632696  2959 ts_tablet_manager.cc:1403] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.015s
I20260709 12:21:30.640646  2957 tablet_bootstrap.cc:492] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:30.641361  2957 ts_tablet_manager.cc:1403] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.060s	user 0.018s	sys 0.019s
I20260709 12:21:30.641737  2958 tablet_bootstrap.cc:492] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:30.642294  2958 ts_tablet_manager.cc:1403] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.003s
I20260709 12:21:30.636564  2959 raft_consensus.cc:359] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.643888  2959 raft_consensus.cc:385] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.644266  2959 raft_consensus.cc:740] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.645097  2959 consensus_queue.cc:260] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.645290  2958 raft_consensus.cc:359] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.645362  2957 raft_consensus.cc:359] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.646284  2958 raft_consensus.cc:385] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.646371  2957 raft_consensus.cc:385] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.646677  2958 raft_consensus.cc:740] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.646837  2957 raft_consensus.cc:740] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.647758  2959 ts_tablet_manager.cc:1434] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.003s
I20260709 12:21:30.647643  2958 consensus_queue.cc:260] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.647809  2957 consensus_queue.cc:260] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.650401  2958 ts_tablet_manager.cc:1434] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:21:30.652222  2957 ts_tablet_manager.cc:1434] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:21:30.652805  2959 tablet_bootstrap.cc:492] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:30.654243  2957 tablet_bootstrap.cc:492] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:30.661124  2959 tablet_bootstrap.cc:654] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.653191  2958 tablet_bootstrap.cc:492] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:30.661818  2957 tablet_bootstrap.cc:654] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.668881  2957 tablet_bootstrap.cc:492] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:30.668952  2958 tablet_bootstrap.cc:654] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.669595  2957 ts_tablet_manager.cc:1403] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 12:21:30.672948  2957 raft_consensus.cc:359] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:21:30.673751  2957 raft_consensus.cc:385] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.674049  2957 raft_consensus.cc:740] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.675046  2961 raft_consensus.cc:493] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.674893  2957 consensus_queue.cc:260] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:21:30.675673  2961 raft_consensus.cc:515] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.680039  2959 tablet_bootstrap.cc:492] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:30.679971  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:30.680754  2959 ts_tablet_manager.cc:1403] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.005s
W20260709 12:21:30.682154  2881 leader_election.cc:343] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.684507  2961 leader_election.cc:290] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:30.684999  2957 ts_tablet_manager.cc:1434] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 12:21:30.684908  2959 raft_consensus.cc:359] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.685986  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:30.686687  2843 raft_consensus.cc:2468] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:30.687143  2957 tablet_bootstrap.cc:492] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:30.685976  2959 raft_consensus.cc:385] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.688062  2958 tablet_bootstrap.cc:492] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:30.687916  2880 leader_election.cc:304] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:30.688140  2959 raft_consensus.cc:740] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.688616  2958 ts_tablet_manager.cc:1403] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.003s
I20260709 12:21:30.689383  2961 raft_consensus.cc:2804] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:30.689011  2959 consensus_queue.cc:260] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.689936  2961 raft_consensus.cc:493] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:30.690362  2961 raft_consensus.cc:3060] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.693728  2958 raft_consensus.cc:359] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.694505  2958 raft_consensus.cc:385] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.694759  2957 tablet_bootstrap.cc:654] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.694872  2958 raft_consensus.cc:740] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.695775  2958 consensus_queue.cc:260] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.697929  2959 ts_tablet_manager.cc:1434] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.001s
I20260709 12:21:30.698793  2958 ts_tablet_manager.cc:1434] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 12:21:30.699082  2959 tablet_bootstrap.cc:492] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:30.706118  2961 raft_consensus.cc:515] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.708495  2961 leader_election.cc:290] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:30.709910  2958 tablet_bootstrap.cc:492] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:30.712488  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:30.712548  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:30.713478  2843 raft_consensus.cc:3060] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:30.714452  2881 leader_election.cc:343] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.717654  2958 tablet_bootstrap.cc:654] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.723004  2959 tablet_bootstrap.cc:654] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.730899  2843 raft_consensus.cc:2468] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:30.732719  2880 leader_election.cc:304] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:30.733970  2961 raft_consensus.cc:2804] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:30.734681  2961 raft_consensus.cc:697] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:30.735864  2961 consensus_queue.cc:237] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.742525  2957 tablet_bootstrap.cc:492] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:30.742830  2959 tablet_bootstrap.cc:492] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:30.743942  2959 ts_tablet_manager.cc:1403] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.004s
I20260709 12:21:30.743954  2957 ts_tablet_manager.cc:1403] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.057s	user 0.027s	sys 0.024s
I20260709 12:21:30.748317  2957 raft_consensus.cc:359] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.749478  2957 raft_consensus.cc:385] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.749846  2957 raft_consensus.cc:740] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.750080  2958 tablet_bootstrap.cc:492] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:30.749956  2959 raft_consensus.cc:359] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.750626  2958 ts_tablet_manager.cc:1403] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.041s	user 0.021s	sys 0.018s
I20260709 12:21:30.750718  2959 raft_consensus.cc:385] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.751207  2959 raft_consensus.cc:740] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.750916  2957 consensus_queue.cc:260] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.752395  2959 consensus_queue.cc:260] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.753391  2957 ts_tablet_manager.cc:1434] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:21:30.754256  2958 raft_consensus.cc:359] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.754951  2958 raft_consensus.cc:385] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.755038  2959 ts_tablet_manager.cc:1434] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:21:30.755241  2958 raft_consensus.cc:740] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.755499  2957 tablet_bootstrap.cc:492] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:30.755999  2958 consensus_queue.cc:260] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.757148  2985 raft_consensus.cc:493] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.757609  2985 raft_consensus.cc:515] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.759778  2985 leader_election.cc:290] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.761503  2958 ts_tablet_manager.cc:1434] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:21:30.757627  2959 tablet_bootstrap.cc:492] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:30.762507  2958 tablet_bootstrap.cc:492] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:30.763113  2985 raft_consensus.cc:493] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.763437  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a6389f65afb4adabe1d70b6f09e99e7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:30.763569  2985 raft_consensus.cc:515] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.764169  2768 raft_consensus.cc:2468] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:30.765228  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a6389f65afb4adabe1d70b6f09e99e7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:30.765651  2957 tablet_bootstrap.cc:654] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.765836  2985 leader_election.cc:290] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.769927  2959 tablet_bootstrap.cc:654] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.772384  2958 tablet_bootstrap.cc:654] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.779630  2807 leader_election.cc:304] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:30.780900  2985 raft_consensus.cc:2804] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:30.781453  2985 raft_consensus.cc:493] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:30.781931  2985 raft_consensus.cc:3060] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.782884  2959 tablet_bootstrap.cc:492] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:30.783385  2959 ts_tablet_manager.cc:1403] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.008s
I20260709 12:21:30.785846  2958 tablet_bootstrap.cc:492] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:30.785674  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1bafa30972440499c69fb1221eae27" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:30.786348  2958 ts_tablet_manager.cc:1403] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.004s
I20260709 12:21:30.786161  2959 raft_consensus.cc:359] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.786600  2916 raft_consensus.cc:2468] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:30.786844  2959 raft_consensus.cc:385] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.789197  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1bafa30972440499c69fb1221eae27" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:30.791531  2768 raft_consensus.cc:2468] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:30.792297  2959 raft_consensus.cc:740] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.789291  2958 raft_consensus.cc:359] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.793078  2959 consensus_queue.cc:260] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.793776  2958 raft_consensus.cc:385] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.794085  2958 raft_consensus.cc:740] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.794817  2958 consensus_queue.cc:260] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.795508  2959 ts_tablet_manager.cc:1434] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:21:30.796484  2959 tablet_bootstrap.cc:492] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:30.796831  2958 ts_tablet_manager.cc:1434] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.003s
I20260709 12:21:30.797794  2958 tablet_bootstrap.cc:492] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:30.803117  2959 tablet_bootstrap.cc:654] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.803644  2958 tablet_bootstrap.cc:654] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:30.790481  2807 leader_election.cc:343] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.821410  2959 tablet_bootstrap.cc:492] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:30.822363  2959 ts_tablet_manager.cc:1403] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.000s
I20260709 12:21:30.823836  2807 leader_election.cc:304] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:30.824642  2969 raft_consensus.cc:2804] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:30.825022  2969 raft_consensus.cc:493] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:30.825352  2969 raft_consensus.cc:3060] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.826279  2959 raft_consensus.cc:359] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.827268  2959 raft_consensus.cc:385] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.827626  2959 raft_consensus.cc:740] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.828563  2959 consensus_queue.cc:260] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.821425  2958 tablet_bootstrap.cc:492] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:30.830049  2958 ts_tablet_manager.cc:1403] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.002s
I20260709 12:21:30.830998  2663 catalog_manager.cc:5697] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:30.836151  2959 ts_tablet_manager.cc:1434] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:21:30.836934  2987 raft_consensus.cc:493] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.837385  2987 raft_consensus.cc:515] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.839681  2987 leader_election.cc:290] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.839383  2958 raft_consensus.cc:359] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.840473  2958 raft_consensus.cc:385] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.840883  2958 raft_consensus.cc:740] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.841109  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:30.841889  2958 consensus_queue.cc:260] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.843779  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:30.844455  2916 raft_consensus.cc:2468] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:30.844928  2958 ts_tablet_manager.cc:1434] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.006s
W20260709 12:21:30.845595  3005 log.cc:927] Time spent T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:21:30.846627  2959 tablet_bootstrap.cc:492] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:30.857621  2961 raft_consensus.cc:493] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.858654  2807 leader_election.cc:304] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:30.858430  2961 raft_consensus.cc:515] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:21:30.859766  2987 raft_consensus.cc:2804] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:30.860170  2987 raft_consensus.cc:493] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:30.860553  2987 raft_consensus.cc:3060] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.860934  2985 raft_consensus.cc:515] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.862259  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:30.863188  2959 tablet_bootstrap.cc:654] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.865087  2985 leader_election.cc:290] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.865893  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:30.867002  2843 raft_consensus.cc:2393] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cc87046f1414b27928722690a5e388f in current term 1: Already voted for candidate 442c92e5267046cd92f46e6f469570a5 in this term.
I20260709 12:21:30.868590  2881 leader_election.cc:304] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:30.868844  2961 leader_election.cc:290] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:30.847559  2958 tablet_bootstrap.cc:492] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:30.849980  2969 raft_consensus.cc:515] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:30.869897  2807 leader_election.cc:343] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.870922  2969 leader_election.cc:290] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.871047  2957 tablet_bootstrap.cc:492] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:30.871665  2957 ts_tablet_manager.cc:1403] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.117s	user 0.039s	sys 0.066s
I20260709 12:21:30.869930  3001 raft_consensus.cc:3060] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.874907  2957 raft_consensus.cc:359] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.877858  2957 raft_consensus.cc:385] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.877815  2958 tablet_bootstrap.cc:654] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.878866  2957 raft_consensus.cc:740] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.880295  2957 consensus_queue.cc:260] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.878883  3012 raft_consensus.cc:493] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.881368  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a6389f65afb4adabe1d70b6f09e99e7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:30.883077  2916 raft_consensus.cc:3060] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.886237  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1bafa30972440499c69fb1221eae27" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:30.886924  2767 raft_consensus.cc:3060] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.877924  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a6389f65afb4adabe1d70b6f09e99e7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:30.889870  2768 raft_consensus.cc:3060] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.889804  3012 raft_consensus.cc:515] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.893656  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:30.894322  2843 raft_consensus.cc:2468] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:30.895685  2880 leader_election.cc:304] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:30.895787  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:30.896592  2766 raft_consensus.cc:2468] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:30.898134  2916 raft_consensus.cc:2468] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:30.900727  2807 leader_election.cc:304] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:30.898259  2961 raft_consensus.cc:2804] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:30.901728  2961 raft_consensus.cc:493] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:30.901895  2969 raft_consensus.cc:2804] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:30.903201  2969 raft_consensus.cc:697] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:30.869293  2987 raft_consensus.cc:515] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.904182  2969 consensus_queue.cc:237] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.906773  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:30.907039  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1bafa30972440499c69fb1221eae27" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:30.908110  2961 raft_consensus.cc:3060] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:30.908921  2807 leader_election.cc:343] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:30.907819  2916 raft_consensus.cc:3060] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.912907  2767 raft_consensus.cc:2468] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:30.917870  2957 ts_tablet_manager.cc:1434] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.046s	user 0.019s	sys 0.004s
I20260709 12:21:30.918947  2957 tablet_bootstrap.cc:492] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:30.919723  2958 tablet_bootstrap.cc:492] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:30.919849  2987 leader_election.cc:290] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.920238  2958 ts_tablet_manager.cc:1403] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.073s	user 0.005s	sys 0.009s
I20260709 12:21:30.923097  2959 tablet_bootstrap.cc:492] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:30.896590  3012 leader_election.cc:290] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:30.923631  2959 ts_tablet_manager.cc:1403] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.077s	user 0.020s	sys 0.011s
I20260709 12:21:30.924305  2807 leader_election.cc:304] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:30.926117  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:30.934214  2917 raft_consensus.cc:3060] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.916239  3001 raft_consensus.cc:2749] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:30.926652  2958 raft_consensus.cc:359] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.935806  2958 raft_consensus.cc:385] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.936077  2958 raft_consensus.cc:740] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.936493  2959 raft_consensus.cc:359] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.937167  2959 raft_consensus.cc:385] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.936805  2958 consensus_queue.cc:260] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.937414  2959 raft_consensus.cc:740] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.937813  2987 raft_consensus.cc:2804] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:30.938416  2987 raft_consensus.cc:697] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:30.938205  2959 consensus_queue.cc:260] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.938988  2958 ts_tablet_manager.cc:1434] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:21:30.939949  2958 tablet_bootstrap.cc:492] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:30.939584  2987 consensus_queue.cc:237] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.930773  2957 tablet_bootstrap.cc:654] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.945252  2768 raft_consensus.cc:2468] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:30.947134  2959 ts_tablet_manager.cc:1434] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:21:30.948159  2959 tablet_bootstrap.cc:492] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:30.953177  2916 raft_consensus.cc:2468] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:30.933176  2961 raft_consensus.cc:515] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.965477  2961 leader_election.cc:290] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:30.967435  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:30.967439  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:30.972545  2768 raft_consensus.cc:3060] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.972664  2843 raft_consensus.cc:3060] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.967990  2917 raft_consensus.cc:2468] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:30.970871  2666 catalog_manager.cc:5697] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:30.974632  2980 raft_consensus.cc:493] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.975148  2980 raft_consensus.cc:515] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.976878  2959 tablet_bootstrap.cc:654] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.977344  2980 leader_election.cc:290] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:30.979687  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99bcb73e9233433a85aae0da2439d851" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:30.973766  2958 tablet_bootstrap.cc:654] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.980543  2843 raft_consensus.cc:2468] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:30.982240  2807 leader_election.cc:304] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:30.983867  2987 raft_consensus.cc:2804] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:30.980365  2917 raft_consensus.cc:2468] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:30.984584  2987 raft_consensus.cc:697] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:30.983824  2768 raft_consensus.cc:2468] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:30.985641  2987 consensus_queue.cc:237] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.986349  2732 leader_election.cc:304] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:30.982154  2880 leader_election.cc:304] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:30.989380  2957 tablet_bootstrap.cc:492] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:30.989494  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99bcb73e9233433a85aae0da2439d851" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:30.989955  2957 ts_tablet_manager.cc:1403] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.071s	user 0.016s	sys 0.005s
I20260709 12:21:30.990449  2980 raft_consensus.cc:2804] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:30.991068  2980 raft_consensus.cc:493] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:30.991473  2980 raft_consensus.cc:3060] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.993381  2957 raft_consensus.cc:359] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.994071  2957 raft_consensus.cc:385] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.994345  2957 raft_consensus.cc:740] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.995723  2957 consensus_queue.cc:260] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:30.998407  3012 raft_consensus.cc:2804] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:30.998893  3012 raft_consensus.cc:697] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:30.999727  3012 consensus_queue.cc:237] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.000213  2972 raft_consensus.cc:493] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.990429  2843 raft_consensus.cc:2468] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:31.011459  2957 ts_tablet_manager.cc:1434] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:21:31.011152  2969 consensus_queue.cc:1048] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:31.012599  2957 tablet_bootstrap.cc:492] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:31.015671  2980 raft_consensus.cc:515] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.000972  2972 raft_consensus.cc:515] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.020696  2768 raft_consensus.cc:3060] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.024555  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:31.025763  2843 raft_consensus.cc:2393] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
W20260709 12:21:31.027539  2714 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:31.030962  2980 leader_election.cc:290] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.031859  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99bcb73e9233433a85aae0da2439d851" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:31.037616  2916 raft_consensus.cc:3060] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.042204  2987 raft_consensus.cc:493] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.042819  2987 raft_consensus.cc:515] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.046675  2972 leader_election.cc:290] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.045636  2987 leader_election.cc:290] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.048094  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99bcb73e9233433a85aae0da2439d851" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:31.048892  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:31.050777  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:31.052771  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:31.056715  2767 raft_consensus.cc:2468] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:31.057255  2732 leader_election.cc:304] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:31.057528  2843 raft_consensus.cc:3060] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.058943  2958 tablet_bootstrap.cc:492] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:31.059608  2958 ts_tablet_manager.cc:1403] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.120s	user 0.012s	sys 0.005s
I20260709 12:21:31.051836  2666 catalog_manager.cc:5697] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:31.063743  2666 catalog_manager.cc:5697] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:31.061964  2807 leader_election.cc:304] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:31.070047  2987 raft_consensus.cc:2804] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:31.070485  2987 raft_consensus.cc:493] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:31.070838  2987 raft_consensus.cc:3060] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.069949  2915 raft_consensus.cc:2468] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:31.050474  2664 catalog_manager.cc:5697] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:31.053858  2980 raft_consensus.cc:493] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.082530  2980 raft_consensus.cc:515] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.069054  2972 raft_consensus.cc:3060] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.085428  2987 raft_consensus.cc:515] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.101519  2987 leader_election.cc:290] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.162081  2987 raft_consensus.cc:493] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.162845  2987 raft_consensus.cc:515] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.169647  2987 leader_election.cc:290] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.187762  2843 raft_consensus.cc:2468] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:31.346330  2958 raft_consensus.cc:359] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.347143  2958 raft_consensus.cc:385] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.347350  2958 raft_consensus.cc:740] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.347986  2958 consensus_queue.cc:260] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.351373  2958 ts_tablet_manager.cc:1434] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.291s	user 0.013s	sys 0.049s
I20260709 12:21:31.354218  2959 tablet_bootstrap.cc:492] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:31.354763  2959 ts_tablet_manager.cc:1403] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.407s	user 0.021s	sys 0.012s
I20260709 12:21:31.203230  2980 leader_election.cc:290] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.356670  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133424caefd34fa8b83015c9fe06d612" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:31.357322  2843 raft_consensus.cc:2468] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:31.217065  2916 raft_consensus.cc:2468] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:31.237140  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c737810e34ba42deaa41e6478e84fe87" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:31.356017  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:31.204706  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133424caefd34fa8b83015c9fe06d612" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:31.362067  2914 raft_consensus.cc:2468] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:31.238492  2972 raft_consensus.cc:2749] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:31.360023  2987 consensus_queue.cc:1048] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:31.362550  2807 leader_election.cc:343] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:31.362991  3022 raft_consensus.cc:493] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.360886  2768 raft_consensus.cc:3060] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.363476  3022 raft_consensus.cc:515] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.214381  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c737810e34ba42deaa41e6478e84fe87" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:31.204080  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:31.366360  2915 raft_consensus.cc:3060] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.361354  3026 raft_consensus.cc:493] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.368431  3022 leader_election.cc:290] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.370234  3012 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:31.367693  3026 raft_consensus.cc:515] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.372453  2957 tablet_bootstrap.cc:654] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.372553  3026 leader_election.cc:290] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.374897  2958 tablet_bootstrap.cc:492] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:31.219772  2731 leader_election.cc:304] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:31.384011  2958 tablet_bootstrap.cc:654] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.390005  2972 raft_consensus.cc:2804] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:21:31.389904  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 170221us
I20260709 12:21:31.390595  2972 raft_consensus.cc:697] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:31.391530  2972 consensus_queue.cc:237] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.391846  2731 leader_election.cc:304] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:31.395262  3026 raft_consensus.cc:2804] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:31.395675  3026 raft_consensus.cc:493] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:31.396064  3026 raft_consensus.cc:3060] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.395795  2959 raft_consensus.cc:359] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.396835  2959 raft_consensus.cc:385] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.397193  2959 raft_consensus.cc:740] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.402267  3012 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:31.405982  3022 consensus_queue.cc:1048] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:31.407655  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:31.409063  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51910d09518b4757be25a91c35ee2ec2" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
W20260709 12:21:31.407552  2807 leader_election.cc:343] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:31.411128  2807 leader_election.cc:304] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:31.412128  2915 raft_consensus.cc:2468] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:31.412842  2969 raft_consensus.cc:2749] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:31.409114  2959 consensus_queue.cc:260] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.415032  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51910d09518b4757be25a91c35ee2ec2" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:31.415773  2959 ts_tablet_manager.cc:1434] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 12:21:31.417066  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:31.417963  2959 tablet_bootstrap.cc:492] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:31.419562  2768 raft_consensus.cc:2468] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:31.419436  3022 consensus_queue.cc:1048] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:31.420578  2807 leader_election.cc:304] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:31.422107  2987 raft_consensus.cc:2804] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:21:31.422255  2731 leader_election.cc:343] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:31.422966  2987 raft_consensus.cc:697] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:31.424280  2987 consensus_queue.cc:237] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.425957  2732 leader_election.cc:304] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
W20260709 12:21:31.427453  2732 leader_election.cc:343] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:31.427640  2969 consensus_queue.cc:1048] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:31.407176  3026 raft_consensus.cc:515] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.430016  2957 tablet_bootstrap.cc:492] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:31.430677  2957 ts_tablet_manager.cc:1403] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.418s	user 0.019s	sys 0.013s
I20260709 12:21:31.432842  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133424caefd34fa8b83015c9fe06d612" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:31.433634  2842 raft_consensus.cc:3060] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.427964  2732 leader_election.cc:304] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:31.439287  2980 raft_consensus.cc:2749] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:31.442726  3022 raft_consensus.cc:2749] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.446069  3026 leader_election.cc:290] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.446084  2957 raft_consensus.cc:359] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.449488  2957 raft_consensus.cc:385] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.450757  2969 consensus_queue.cc:1048] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:31.450497  2957 raft_consensus.cc:740] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.481048  2957 consensus_queue.cc:260] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.484138  2957 ts_tablet_manager.cc:1434] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.053s	user 0.025s	sys 0.009s
I20260709 12:21:31.485009  2957 tablet_bootstrap.cc:492] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:31.450300  2842 raft_consensus.cc:2468] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:31.490924  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133424caefd34fa8b83015c9fe06d612" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:31.491619  2913 raft_consensus.cc:3060] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.495035  2731 leader_election.cc:304] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:31.497884  2959 tablet_bootstrap.cc:654] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.499552  2664 catalog_manager.cc:5697] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:31.498639  3022 raft_consensus.cc:2804] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:31.510021  3022 raft_consensus.cc:697] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:31.510200  3012 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:31.511050  3022 consensus_queue.cc:237] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.519260  2985 consensus_queue.cc:1048] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:21:31.549325  2958 tablet_bootstrap.cc:492] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:31.550165  2958 ts_tablet_manager.cc:1403] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.176s	user 0.021s	sys 0.005s
W20260709 12:21:31.562148  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 44090.9us
I20260709 12:21:31.566847  2913 raft_consensus.cc:2468] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:31.572541  2958 raft_consensus.cc:359] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.574131  2958 raft_consensus.cc:385] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.574548  2958 raft_consensus.cc:740] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.576583  2958 consensus_queue.cc:260] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.579670  2958 ts_tablet_manager.cc:1434] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.002s
I20260709 12:21:31.580698  2958 tablet_bootstrap.cc:492] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:31.587131  2969 raft_consensus.cc:493] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.591758  2969 raft_consensus.cc:515] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.592893  2957 tablet_bootstrap.cc:654] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.594848  2958 tablet_bootstrap.cc:654] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.593603  2987 consensus_queue.cc:1048] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 12:21:31.606994  2807 consensus_peers.cc:597] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:31.607651  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:31.608145  2985 consensus_queue.cc:1048] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:31.608328  2961 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:31.611313  2807 leader_election.cc:343] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:31.595669  2664 catalog_manager.cc:5697] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:31.621171  2969 leader_election.cc:290] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.625758  2959 tablet_bootstrap.cc:492] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:31.626430  2959 ts_tablet_manager.cc:1403] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.210s	user 0.020s	sys 0.008s
I20260709 12:21:31.629101  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:31.630175  2959 raft_consensus.cc:359] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.631109  2959 raft_consensus.cc:385] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.631461  2959 raft_consensus.cc:740] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.632340  2959 consensus_queue.cc:260] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.635613  2959 ts_tablet_manager.cc:1434] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:21:31.637836  2959 tablet_bootstrap.cc:492] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
W20260709 12:21:31.637836  2807 leader_election.cc:343] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:31.638471  2807 leader_election.cc:304] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:31.640707  2969 raft_consensus.cc:2749] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:31.650422  2665 catalog_manager.cc:5697] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:31.679863  2959 tablet_bootstrap.cc:654] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.685460  2959 tablet_bootstrap.cc:492] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:31.685990  2959 ts_tablet_manager.cc:1403] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.001s
I20260709 12:21:31.686970  2958 tablet_bootstrap.cc:492] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:31.687441  2958 ts_tablet_manager.cc:1403] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.107s	user 0.006s	sys 0.015s
I20260709 12:21:31.688688  2959 raft_consensus.cc:359] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.689265  2959 raft_consensus.cc:385] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.689497  2959 raft_consensus.cc:740] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.689599  2958 raft_consensus.cc:359] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.690343  2958 raft_consensus.cc:385] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.690603  2958 raft_consensus.cc:740] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.690399  2959 consensus_queue.cc:260] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.691236  2958 consensus_queue.cc:260] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.692337  2959 ts_tablet_manager.cc:1434] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:31.692729  2958 ts_tablet_manager.cc:1434] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:21:31.693179  2959 tablet_bootstrap.cc:492] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:31.693395  2958 tablet_bootstrap.cc:492] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:31.695622  2957 tablet_bootstrap.cc:492] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:31.696334  2957 ts_tablet_manager.cc:1403] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.212s	user 0.012s	sys 0.008s
I20260709 12:21:31.699645  2959 tablet_bootstrap.cc:654] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.699363  2957 raft_consensus.cc:359] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.700053  2957 raft_consensus.cc:385] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.700344  2957 raft_consensus.cc:740] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.701478  3022 raft_consensus.cc:493] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.702262  2958 tablet_bootstrap.cc:654] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.702028  3022 raft_consensus.cc:515] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.706430  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d436a6cfd22640bea76c0256f1d01684" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:31.709369  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d436a6cfd22640bea76c0256f1d01684" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
W20260709 12:21:31.709802  2731 leader_election.cc:343] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:31.711194  2732 leader_election.cc:343] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:31.711817  2732 leader_election.cc:304] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:31.712994  2972 raft_consensus.cc:2749] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:31.714229  3022 leader_election.cc:290] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.714919  2958 tablet_bootstrap.cc:492] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:31.715986  2958 ts_tablet_manager.cc:1403] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.010s
I20260709 12:21:31.715554  2957 consensus_queue.cc:260] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.719007  2957 ts_tablet_manager.cc:1434] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:21:31.720395  2957 tablet_bootstrap.cc:492] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:31.719020  2958 raft_consensus.cc:359] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.728430  2958 raft_consensus.cc:385] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.728736  2958 raft_consensus.cc:740] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.729097  2959 tablet_bootstrap.cc:492] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:31.729400  2957 tablet_bootstrap.cc:654] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.729625  2959 ts_tablet_manager.cc:1403] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.037s	user 0.022s	sys 0.012s
I20260709 12:21:31.729453  2958 consensus_queue.cc:260] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.732051  2958 ts_tablet_manager.cc:1434] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 12:21:31.733016  2958 tablet_bootstrap.cc:492] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:31.733512  2959 raft_consensus.cc:359] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.734453  2959 raft_consensus.cc:385] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.734797  2959 raft_consensus.cc:740] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.735862  2959 consensus_queue.cc:260] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.739046  2959 ts_tablet_manager.cc:1434] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:21:31.739758  2958 tablet_bootstrap.cc:654] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.740327  2959 tablet_bootstrap.cc:492] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:31.742455  3022 raft_consensus.cc:493] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.743084  3022 raft_consensus.cc:515] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.746598  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:31.746865  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:31.748449  3022 leader_election.cc:290] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.748423  2731 leader_election.cc:304] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:31.749284  3022 raft_consensus.cc:2749] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:31.751917  2958 tablet_bootstrap.cc:492] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:31.752405  2958 ts_tablet_manager.cc:1403] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:21:31.755167  3001 raft_consensus.cc:493] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.755466  2958 raft_consensus.cc:359] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.755698  3001 raft_consensus.cc:515] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.756253  2958 raft_consensus.cc:385] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.756567  2958 raft_consensus.cc:740] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.756839  2959 tablet_bootstrap.cc:654] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.757264  2958 consensus_queue.cc:260] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.759799  2958 ts_tablet_manager.cc:1434] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:21:31.760519  3001 leader_election.cc:290] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:31.760180  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c737810e34ba42deaa41e6478e84fe87" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:31.761596  2958 tablet_bootstrap.cc:492] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:31.762005  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c737810e34ba42deaa41e6478e84fe87" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:31.762622  2768 raft_consensus.cc:2468] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:31.763950  2881 leader_election.cc:304] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:31.764104  2842 raft_consensus.cc:2468] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:31.766703  3001 raft_consensus.cc:2804] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:31.768512  3001 raft_consensus.cc:493] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:31.773221  3001 raft_consensus.cc:3060] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.775602  2958 tablet_bootstrap.cc:654] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.789361  2957 tablet_bootstrap.cc:492] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:31.790095  2957 ts_tablet_manager.cc:1403] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.070s	user 0.023s	sys 0.027s
I20260709 12:21:31.800037  3001 raft_consensus.cc:515] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.801566  2961 raft_consensus.cc:493] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.802203  2961 raft_consensus.cc:515] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.802873  3001 leader_election.cc:290] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:31.806233  2959 tablet_bootstrap.cc:492] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:31.806057  2957 raft_consensus.cc:359] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.806280  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c737810e34ba42deaa41e6478e84fe87" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:31.806736  2957 raft_consensus.cc:385] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.807132  2957 raft_consensus.cc:740] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.806810  2959 ts_tablet_manager.cc:1403] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.025s
I20260709 12:21:31.808137  2957 consensus_queue.cc:260] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.808598  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c737810e34ba42deaa41e6478e84fe87" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:31.809089  2768 raft_consensus.cc:3060] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.810359  2957 ts_tablet_manager.cc:1434] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:21:31.810989  2959 raft_consensus.cc:359] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.811535  2959 raft_consensus.cc:385] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.811700  2959 raft_consensus.cc:740] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.812386  2959 consensus_queue.cc:260] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.814144  2969 raft_consensus.cc:493] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.814271  2959 ts_tablet_manager.cc:1434] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:31.814607  2969 raft_consensus.cc:515] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.815829  2959 tablet_bootstrap.cc:492] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:31.816598  2969 leader_election.cc:290] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.818576  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d436a6cfd22640bea76c0256f1d01684" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:31.819082  2958 tablet_bootstrap.cc:492] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:31.819320  2766 raft_consensus.cc:2468] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:31.820815  2807 leader_election.cc:304] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:31.822000  3042 raft_consensus.cc:2804] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:31.822816  3042 raft_consensus.cc:493] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:31.822896  2842 raft_consensus.cc:3060] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.822952  2959 tablet_bootstrap.cc:654] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.823231  3042 raft_consensus.cc:3060] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.822453  2957 tablet_bootstrap.cc:492] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:31.825922  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:31.827924  2881 leader_election.cc:343] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:31.830890  2961 leader_election.cc:290] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:31.832100  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:31.833384  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d436a6cfd22640bea76c0256f1d01684" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:31.834036  2913 raft_consensus.cc:2468] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:31.834609  2969 consensus_queue.cc:1048] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:31.835536  2880 leader_election.cc:343] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:31.836199  2957 tablet_bootstrap.cc:654] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.835990  2880 leader_election.cc:304] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:31.839529  3001 raft_consensus.cc:2749] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:31.842877  2768 raft_consensus.cc:2468] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:31.844831  2881 leader_election.cc:304] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:31.846093  2961 raft_consensus.cc:2804] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:31.846761  2961 raft_consensus.cc:697] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:31.847877  2961 consensus_queue.cc:237] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.854974  2842 raft_consensus.cc:2468] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:31.819566  2958 ts_tablet_manager.cc:1403] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.058s	user 0.025s	sys 0.013s
I20260709 12:21:31.857062  3042 raft_consensus.cc:515] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.862110  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d436a6cfd22640bea76c0256f1d01684" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:31.862843  2766 raft_consensus.cc:3060] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.863690  2958 raft_consensus.cc:359] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.864641  2958 raft_consensus.cc:385] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.864985  2958 raft_consensus.cc:740] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.870805  2958 consensus_queue.cc:260] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.874711  2766 raft_consensus.cc:2468] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:31.878685  2958 ts_tablet_manager.cc:1434] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:21:31.894837  2958 tablet_bootstrap.cc:492] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:31.902025  2985 consensus_queue.cc:1048] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:31.908167  2958 tablet_bootstrap.cc:654] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.912928  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d436a6cfd22640bea76c0256f1d01684" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:31.912873  3042 leader_election.cc:290] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.913738  2913 raft_consensus.cc:3060] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.918910  2807 leader_election.cc:304] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:31.920627  3042 raft_consensus.cc:2804] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:31.922454  2959 tablet_bootstrap.cc:492] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:31.922344  2913 raft_consensus.cc:2468] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:31.921414  3042 raft_consensus.cc:697] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:31.923264  2959 ts_tablet_manager.cc:1403] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.107s	user 0.062s	sys 0.034s
I20260709 12:21:31.933527  2972 consensus_queue.cc:1048] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:31.940446  2985 raft_consensus.cc:493] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:31.941005  2985 raft_consensus.cc:515] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.943624  2985 leader_election.cc:290] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:31.944720  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51910d09518b4757be25a91c35ee2ec2" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:31.945415  2766 raft_consensus.cc:2468] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:31.945695  2959 raft_consensus.cc:359] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.946424  2959 raft_consensus.cc:385] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.946549  2807 leader_election.cc:304] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:31.946723  2959 raft_consensus.cc:740] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.946671  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51910d09518b4757be25a91c35ee2ec2" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:31.949738  2972 consensus_queue.cc:1048] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:31.950734  2985 raft_consensus.cc:2804] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:31.951131  2985 raft_consensus.cc:493] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:31.951507  2985 raft_consensus.cc:3060] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:31.951313  2807 leader_election.cc:343] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:31.949837  2959 consensus_queue.cc:260] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.956849  2958 tablet_bootstrap.cc:492] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:31.957458  2958 ts_tablet_manager.cc:1403] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.007s
I20260709 12:21:31.958595  2959 ts_tablet_manager.cc:1434] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.035s	user 0.010s	sys 0.000s
I20260709 12:21:31.959968  2959 tablet_bootstrap.cc:492] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:31.960433  2958 raft_consensus.cc:359] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.961127  2958 raft_consensus.cc:385] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.925017  3042 consensus_queue.cc:237] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.961417  2958 raft_consensus.cc:740] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.969586  2958 consensus_queue.cc:260] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.971792  2958 ts_tablet_manager.cc:1434] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:21:31.978924  2958 tablet_bootstrap.cc:492] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:31.986522  2959 tablet_bootstrap.cc:654] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.987452  2957 tablet_bootstrap.cc:492] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:31.987979  2957 ts_tablet_manager.cc:1403] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.166s	user 0.042s	sys 0.061s
I20260709 12:21:31.987986  2985 raft_consensus.cc:515] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:31.992278  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51910d09518b4757be25a91c35ee2ec2" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:31.992914  2766 raft_consensus.cc:3060] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.008651  2766 raft_consensus.cc:2468] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:32.011139  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51910d09518b4757be25a91c35ee2ec2" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:32.012544  2985 leader_election.cc:290] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.017939  2957 raft_consensus.cc:359] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.018761  2957 raft_consensus.cc:385] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.019001  2957 raft_consensus.cc:740] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.021679  2807 leader_election.cc:304] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:32.022634  2987 raft_consensus.cc:2804] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:32.023126  2987 raft_consensus.cc:697] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:32.024022  2987 consensus_queue.cc:237] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.024979  2957 consensus_queue.cc:260] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.029603  2913 raft_consensus.cc:3060] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.031985  2957 ts_tablet_manager.cc:1434] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.002s
I20260709 12:21:32.033502  2958 tablet_bootstrap.cc:654] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.036083  2957 tablet_bootstrap.cc:492] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:32.039319  3042 raft_consensus.cc:493] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.041857  2666 catalog_manager.cc:5697] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:32.042363  2665 catalog_manager.cc:5697] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:32.049741  2959 tablet_bootstrap.cc:492] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
W20260709 12:21:32.072018  2714 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:32.105389  3026 raft_consensus.cc:493] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.111261  3026 raft_consensus.cc:515] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.114502  3026 leader_election.cc:290] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.104097  2913 raft_consensus.cc:2468] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:32.118002  3022 raft_consensus.cc:493] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.121239  3022 raft_consensus.cc:515] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.126693  2959 ts_tablet_manager.cc:1403] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.167s	user 0.018s	sys 0.033s
I20260709 12:21:32.049656  3042 raft_consensus.cc:515] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.098016  2969 raft_consensus.cc:493] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.132037  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:32.132829  2841 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:32.133117  3042 leader_election.cc:290] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.134184  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.138602  3022 leader_election.cc:290] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.131340  2969 raft_consensus.cc:515] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.142270  2969 leader_election.cc:290] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:32.145290  2732 leader_election.cc:343] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.146634  2731 leader_election.cc:304] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:32.147433  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.152510  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.159430  2958 tablet_bootstrap.cc:492] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:32.160177  2958 ts_tablet_manager.cc:1403] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.186s	user 0.009s	sys 0.031s
I20260709 12:21:32.152940  2972 raft_consensus.cc:2804] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:32.153921  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.151268  3026 consensus_queue.cc:1048] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 12:21:32.154999  2807 leader_election.cc:343] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.155547  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:32.168435  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.162839  2972 raft_consensus.cc:493] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:32.169631  2768 raft_consensus.cc:2468] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:32.171631  2959 raft_consensus.cc:359] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.172421  2959 raft_consensus.cc:385] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.172762  2959 raft_consensus.cc:740] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.174562  2959 consensus_queue.cc:260] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.176492  2959 ts_tablet_manager.cc:1434] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.002s
I20260709 12:21:32.171590  2841 raft_consensus.cc:2468] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:32.179590  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.172405  2972 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.183228  2957 tablet_bootstrap.cc:654] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.184350  2959 tablet_bootstrap.cc:492] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
W20260709 12:21:32.191411  2732 leader_election.cc:343] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.198460  2969 raft_consensus.cc:493] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.198673  2958 raft_consensus.cc:359] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.199035  2969 raft_consensus.cc:515] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.201115  2969 leader_election.cc:290] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.202098  2731 leader_election.cc:304] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:32.203763  2969 raft_consensus.cc:493] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.204877  2807 leader_election.cc:304] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:32.205682  3071 raft_consensus.cc:2804] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:32.206059  3071 raft_consensus.cc:493] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:32.206314  3026 raft_consensus.cc:2804] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:32.206382  3071 raft_consensus.cc:3060] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.206749  3026 raft_consensus.cc:493] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:32.199677  2958 raft_consensus.cc:385] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.207692  2958 raft_consensus.cc:740] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.209609  3026 raft_consensus.cc:3060] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.204308  2969 raft_consensus.cc:515] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.210387  2958 consensus_queue.cc:260] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.213421  3022 raft_consensus.cc:493] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.213331  3073 consensus_queue.cc:1048] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:32.213976  3022 raft_consensus.cc:515] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.216135  3022 leader_election.cc:290] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.218231  2969 leader_election.cc:290] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.218770  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41f5e1f4763f4d978619c2696e7aaa71" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
W20260709 12:21:32.220883  2732 leader_election.cc:343] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.224061  2958 ts_tablet_manager.cc:1434] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.004s
I20260709 12:21:32.224985  2958 tablet_bootstrap.cc:492] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:32.226420  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41f5e1f4763f4d978619c2696e7aaa71" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.227108  2768 raft_consensus.cc:2468] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:32.226816  3026 raft_consensus.cc:515] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.231943  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41f5e1f4763f4d978619c2696e7aaa71" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.232482  3026 leader_election.cc:290] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.232170  3071 raft_consensus.cc:515] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.231741  2666 catalog_manager.cc:5697] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:32.234390  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:32.234730  2972 raft_consensus.cc:515] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.237124  2972 leader_election.cc:290] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.238248  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:32.238508  2768 raft_consensus.cc:3060] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.241168  3071 leader_election.cc:290] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:32.242195  2732 leader_election.cc:343] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:32.245428  2732 leader_election.cc:343] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:32.246158  2807 leader_election.cc:343] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.246654  2807 leader_election.cc:304] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
W20260709 12:21:32.247848  2807 leader_election.cc:343] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.248932  2969 raft_consensus.cc:2749] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:32.254781  2957 tablet_bootstrap.cc:492] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:32.255641  2957 ts_tablet_manager.cc:1403] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.220s	user 0.012s	sys 0.005s
I20260709 12:21:32.255759  2959 tablet_bootstrap.cc:654] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:32.254676  2807 leader_election.cc:343] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.259038  2957 raft_consensus.cc:359] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.260107  2958 tablet_bootstrap.cc:654] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.260106  2957 raft_consensus.cc:385] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.260704  2957 raft_consensus.cc:740] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.260754  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.261526  2957 consensus_queue.cc:260] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.263723  2957 ts_tablet_manager.cc:1434] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:32.263906  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.264675  2957 tablet_bootstrap.cc:492] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:32.264605  2914 raft_consensus.cc:2468] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:32.269297  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:32.269834  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:32.270678  2765 raft_consensus.cc:2393] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9a10672cd53140d59ff945f75345cc06 in current term 1: Already voted for candidate 442c92e5267046cd92f46e6f469570a5 in this term.
I20260709 12:21:32.271065  2807 leader_election.cc:304] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:32.271924  2957 tablet_bootstrap.cc:654] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.272616  3042 raft_consensus.cc:2804] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:32.273078  3042 raft_consensus.cc:493] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:32.273454  3042 raft_consensus.cc:3060] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.273639  3001 consensus_queue.cc:1048] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:32.276686  2957 tablet_bootstrap.cc:492] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:32.277146  2957 ts_tablet_manager.cc:1403] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 12:21:32.280123  2957 raft_consensus.cc:359] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.280791  2957 raft_consensus.cc:385] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.281100  2957 raft_consensus.cc:740] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.281788  2957 consensus_queue.cc:260] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.283859  2957 ts_tablet_manager.cc:1434] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:21:32.285152  2957 tablet_bootstrap.cc:492] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:32.291154  2957 tablet_bootstrap.cc:654] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.290762  3012 consensus_queue.cc:1048] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:32.290705  3042 raft_consensus.cc:515] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.293702  3042 leader_election.cc:290] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:32.294854  2807 leader_election.cc:343] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.301304  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41f5e1f4763f4d978619c2696e7aaa71" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:32.302492  2841 raft_consensus.cc:2393] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 9a10672cd53140d59ff945f75345cc06 in this term.
I20260709 12:21:32.304823  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:32.305955  2840 raft_consensus.cc:2393] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 9a10672cd53140d59ff945f75345cc06 in this term.
I20260709 12:21:32.306353  2959 tablet_bootstrap.cc:492] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:32.306869  2959 ts_tablet_manager.cc:1403] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.123s	user 0.012s	sys 0.003s
I20260709 12:21:32.309442  2958 tablet_bootstrap.cc:492] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:32.310067  2958 ts_tablet_manager.cc:1403] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.085s	user 0.010s	sys 0.004s
I20260709 12:21:32.311161  2959 raft_consensus.cc:359] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.311658  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:32.312058  2959 raft_consensus.cc:385] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.312469  2959 raft_consensus.cc:740] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.312300  2840 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.314316  2957 tablet_bootstrap.cc:492] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:32.314874  2957 ts_tablet_manager.cc:1403] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.000s
I20260709 12:21:32.315558  2959 consensus_queue.cc:260] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.318455  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41f5e1f4763f4d978619c2696e7aaa71" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:32.319343  2766 raft_consensus.cc:3060] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.324492  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41f5e1f4763f4d978619c2696e7aaa71" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:32.325887  2731 leader_election.cc:304] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:32.327018  2958 raft_consensus.cc:359] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.327457  2807 leader_election.cc:304] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:32.327750  2958 raft_consensus.cc:385] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.328061  2958 raft_consensus.cc:740] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.328250  3042 raft_consensus.cc:2804] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:21:32.328595  2807 leader_election.cc:343] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.328593  3042 raft_consensus.cc:493] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:32.329085  2807 leader_election.cc:304] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:32.329252  3042 raft_consensus.cc:3060] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.328780  2958 consensus_queue.cc:260] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.331883  2958 ts_tablet_manager.cc:1434] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:21:32.332903  2958 tablet_bootstrap.cc:492] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:32.332958  2840 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:32.336680  2957 raft_consensus.cc:359] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.337400  2957 raft_consensus.cc:385] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.337713  2957 raft_consensus.cc:740] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.338433  2957 consensus_queue.cc:260] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.338989  2731 leader_election.cc:304] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:32.339504  2969 raft_consensus.cc:493] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.336829  3042 raft_consensus.cc:515] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.339975  2969 raft_consensus.cc:515] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.340349  3073 raft_consensus.cc:2749] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:32.345753  3071 raft_consensus.cc:2749] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:21:32.346308  2807 leader_election.cc:343] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.346972  2958 tablet_bootstrap.cc:654] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.347232  2731 leader_election.cc:304] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:32.348100  2959 ts_tablet_manager.cc:1434] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.001s
I20260709 12:21:32.348891  2959 tablet_bootstrap.cc:492] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:32.347553  2969 leader_election.cc:290] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.350107  2972 raft_consensus.cc:493] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.350607  2972 raft_consensus.cc:515] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.351155  3026 raft_consensus.cc:2804] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:32.352660  2972 leader_election.cc:290] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.355144  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659c2c3a61024d07a2296bc83d87a52d" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.356487  3042 leader_election.cc:290] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:32.358992  2732 leader_election.cc:343] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.344193  2766 raft_consensus.cc:2468] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:32.361497  3022 raft_consensus.cc:2749] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:32.362028  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.340929  2957 ts_tablet_manager.cc:1434] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:21:32.363235  2959 tablet_bootstrap.cc:654] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.363970  2957 tablet_bootstrap.cc:492] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:32.364768  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.366786  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:32.367368  2914 raft_consensus.cc:3060] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.368844  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659c2c3a61024d07a2296bc83d87a52d" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
W20260709 12:21:32.373258  2731 leader_election.cc:343] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.373792  2731 leader_election.cc:304] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:32.375144  2972 raft_consensus.cc:2749] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:32.376706  2807 leader_election.cc:343] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.366372  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:32.378473  2766 raft_consensus.cc:3060] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.379508  2807 leader_election.cc:304] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:32.382157  2969 raft_consensus.cc:2804] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:32.382639  2969 raft_consensus.cc:697] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:32.383522  2969 consensus_queue.cc:237] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.391366  2914 raft_consensus.cc:2468] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:32.394888  2957 tablet_bootstrap.cc:654] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.396938  2958 tablet_bootstrap.cc:492] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:32.397864  2958 ts_tablet_manager.cc:1403] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.065s	user 0.020s	sys 0.009s
I20260709 12:21:32.402067  2958 raft_consensus.cc:359] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.393093  3026 raft_consensus.cc:697] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:32.402968  2958 raft_consensus.cc:385] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.403412  2958 raft_consensus.cc:740] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
W20260709 12:21:32.402738  2807 leader_election.cc:343] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.404448  3026 consensus_queue.cc:237] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.404935  2807 leader_election.cc:304] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:32.404361  2958 consensus_queue.cc:260] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.406188  3042 raft_consensus.cc:2749] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:32.409168  2958 ts_tablet_manager.cc:1434] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:21:32.412024  2766 raft_consensus.cc:2468] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:32.412885  2957 tablet_bootstrap.cc:492] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:32.413465  2957 ts_tablet_manager.cc:1403] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.050s	user 0.009s	sys 0.013s
I20260709 12:21:32.413335  2807 leader_election.cc:304] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:32.412243  2958 tablet_bootstrap.cc:492] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:32.414909  3042 raft_consensus.cc:2804] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:32.415395  3042 raft_consensus.cc:697] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:32.416249  3042 consensus_queue.cc:237] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.416600  2957 raft_consensus.cc:359] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.417241  2957 raft_consensus.cc:385] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.417526  2957 raft_consensus.cc:740] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.418817  2959 tablet_bootstrap.cc:492] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:32.419121  3001 raft_consensus.cc:493] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.420432  2959 ts_tablet_manager.cc:1403] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.072s	user 0.018s	sys 0.012s
I20260709 12:21:32.420277  3001 raft_consensus.cc:515] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.424592  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:32.426843  2881 leader_election.cc:343] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.427762  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:32.428531  2840 raft_consensus.cc:2468] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:32.429469  3012 raft_consensus.cc:493] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.430166  3012 raft_consensus.cc:515] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.429658  2665 catalog_manager.cc:5697] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:32.418229  2957 consensus_queue.cc:260] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.434347  3012 leader_election.cc:290] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:32.435657  3001 leader_election.cc:290] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:32.445040  2957 ts_tablet_manager.cc:1434] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 12:21:32.447747  2957 tablet_bootstrap.cc:492] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:32.446911  2880 leader_election.cc:304] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:32.450192  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.450134  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:32.448819  3001 raft_consensus.cc:2804] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:32.453522  3001 raft_consensus.cc:493] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:32.453476  2959 raft_consensus.cc:359] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.454039  3001 raft_consensus.cc:3060] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.455446  2959 raft_consensus.cc:385] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.455688  2959 raft_consensus.cc:740] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.458544  2840 raft_consensus.cc:2393] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cc87046f1414b27928722690a5e388f in current term 1: Already voted for candidate 442c92e5267046cd92f46e6f469570a5 in this term.
I20260709 12:21:32.459952  2880 leader_election.cc:304] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:32.461216  2961 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.466064  2959 consensus_queue.cc:260] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.475219  2961 raft_consensus.cc:2749] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:32.465240  3001 raft_consensus.cc:515] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.486387  2664 catalog_manager.cc:5697] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:32.489890  3042 consensus_queue.cc:1048] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:32.492121  2959 ts_tablet_manager.cc:1434] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.071s	user 0.004s	sys 0.003s
I20260709 12:21:32.493170  2959 tablet_bootstrap.cc:492] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:32.496933  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:32.498292  2958 tablet_bootstrap.cc:654] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.498816  2957 tablet_bootstrap.cc:654] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:32.502055  2881 leader_election.cc:343] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.502471  3001 leader_election.cc:290] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:32.503937  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:32.504669  2840 raft_consensus.cc:3060] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.504020  2665 catalog_manager.cc:5697] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:32.509603  2969 consensus_queue.cc:1048] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:32.512463  2957 tablet_bootstrap.cc:492] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:32.513177  2957 ts_tablet_manager.cc:1403] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.066s	user 0.015s	sys 0.002s
I20260709 12:21:32.517355  2957 raft_consensus.cc:359] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.518424  2957 raft_consensus.cc:385] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.518792  2957 raft_consensus.cc:740] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.521914  2957 consensus_queue.cc:260] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.523972  2840 raft_consensus.cc:2468] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:32.528121  2880 leader_election.cc:304] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:32.529958  3001 raft_consensus.cc:2804] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:32.530476  3001 raft_consensus.cc:697] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:32.531406  3001 consensus_queue.cc:237] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.532814  2959 tablet_bootstrap.cc:654] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.536228  2957 ts_tablet_manager.cc:1434] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.002s
I20260709 12:21:32.538158  2957 tablet_bootstrap.cc:492] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:32.552309  2957 tablet_bootstrap.cc:654] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.561944  2665 catalog_manager.cc:5697] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:32.575798  3071 consensus_queue.cc:1048] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:32.587044  3001 raft_consensus.cc:493] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.587833  3001 raft_consensus.cc:515] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.591588  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.594111  3001 leader_election.cc:290] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:32.595638  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
W20260709 12:21:32.595897  2881 leader_election.cc:343] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.596261  2843 raft_consensus.cc:2468] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:32.597257  2880 leader_election.cc:304] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:32.598102  3001 raft_consensus.cc:2804] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:32.598503  3001 raft_consensus.cc:493] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:32.599014  3001 raft_consensus.cc:3060] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.605495  3042 consensus_queue.cc:1048] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:32.607297  2958 tablet_bootstrap.cc:492] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:32.607884  2958 ts_tablet_manager.cc:1403] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.196s	user 0.014s	sys 0.006s
I20260709 12:21:32.609596  2961 raft_consensus.cc:493] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.610147  2961 raft_consensus.cc:515] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.613694  2958 raft_consensus.cc:359] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.614471  2958 raft_consensus.cc:385] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.614755  2958 raft_consensus.cc:740] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.614933  2957 tablet_bootstrap.cc:492] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:32.615270  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:32.615438  2957 ts_tablet_manager.cc:1403] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.078s	user 0.011s	sys 0.031s
I20260709 12:21:32.615530  2958 consensus_queue.cc:260] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.616732  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.617501  2766 raft_consensus.cc:2393] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cc87046f1414b27928722690a5e388f in current term 1: Already voted for candidate 442c92e5267046cd92f46e6f469570a5 in this term.
I20260709 12:21:32.617988  2958 ts_tablet_manager.cc:1434] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:21:32.616101  2840 raft_consensus.cc:2393] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cc87046f1414b27928722690a5e388f in current term 1: Already voted for candidate 9a10672cd53140d59ff945f75345cc06 in this term.
I20260709 12:21:32.618497  2957 raft_consensus.cc:359] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.620177  2959 tablet_bootstrap.cc:492] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:32.619953  2880 leader_election.cc:304] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:32.619206  2957 raft_consensus.cc:385] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.620851  2957 raft_consensus.cc:740] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.620838  2959 ts_tablet_manager.cc:1403] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.128s	user 0.009s	sys 0.010s
I20260709 12:21:32.621528  3012 raft_consensus.cc:3060] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.621802  2957 consensus_queue.cc:260] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.618993  2958 tablet_bootstrap.cc:492] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:32.624338  2957 ts_tablet_manager.cc:1434] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:21:32.625275  2957 tablet_bootstrap.cc:492] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:32.626276  2961 leader_election.cc:290] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:32.631843  2961 raft_consensus.cc:493] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.632027  2958 tablet_bootstrap.cc:654] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.632373  2961 raft_consensus.cc:515] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.634970  3042 raft_consensus.cc:493] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.635251  2961 leader_election.cc:290] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:32.635529  3042 raft_consensus.cc:515] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.637864  3042 leader_election.cc:290] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.635797  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:32.639758  2881 leader_election.cc:343] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.640013  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34453a67223842ed98d4220533186ade" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.640631  3012 raft_consensus.cc:2749] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:32.641772  2807 leader_election.cc:343] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.641700  3001 raft_consensus.cc:515] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.642957  2959 raft_consensus.cc:359] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.643308  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34453a67223842ed98d4220533186ade" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.645372  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:32.646564  2959 raft_consensus.cc:385] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:32.646700  2807 leader_election.cc:343] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.646944  2959 raft_consensus.cc:740] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.647171  2807 leader_election.cc:304] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:32.648274  3042 raft_consensus.cc:2749] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:32.649696  2959 consensus_queue.cc:260] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.650717  2765 raft_consensus.cc:3060] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.652894  2959 ts_tablet_manager.cc:1434] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.031s	user 0.000s	sys 0.006s
I20260709 12:21:32.653928  2959 tablet_bootstrap.cc:492] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:32.660583  2959 tablet_bootstrap.cc:654] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.662245  3001 leader_election.cc:290] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:21:32.663944  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 44081.2us
I20260709 12:21:32.665632  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:32.666910  2765 raft_consensus.cc:2468] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:32.667837  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:32.668481  2840 raft_consensus.cc:3060] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.675843  2840 raft_consensus.cc:2468] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
W20260709 12:21:32.677217  2880 leader_election.cc:343] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.678604  2959 tablet_bootstrap.cc:492] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:32.678684  2880 leader_election.cc:304] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:32.679152  2959 ts_tablet_manager.cc:1403] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.025s	user 0.003s	sys 0.007s
I20260709 12:21:32.679659  3100 raft_consensus.cc:2749] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:32.680166  2957 tablet_bootstrap.cc:654] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.680042  2880 leader_election.cc:304] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:32.680868  3100 raft_consensus.cc:2804] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:32.681347  3100 raft_consensus.cc:697] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:32.682111  2959 raft_consensus.cc:359] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.682195  3100 consensus_queue.cc:237] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.682901  2959 raft_consensus.cc:385] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.683208  2959 raft_consensus.cc:740] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.683879  2959 consensus_queue.cc:260] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.686028  2959 ts_tablet_manager.cc:1434] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:32.686972  2959 tablet_bootstrap.cc:492] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:32.690449  3026 raft_consensus.cc:493] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.691128  3026 raft_consensus.cc:515] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.693081  2958 tablet_bootstrap.cc:492] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:32.693579  2958 ts_tablet_manager.cc:1403] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.075s	user 0.021s	sys 0.020s
I20260709 12:21:32.694168  2957 tablet_bootstrap.cc:492] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:32.703629  2957 ts_tablet_manager.cc:1403] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.079s	user 0.017s	sys 0.005s
I20260709 12:21:32.703899  2958 raft_consensus.cc:359] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.704653  2958 raft_consensus.cc:385] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.704917  2958 raft_consensus.cc:740] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.705636  2958 consensus_queue.cc:260] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.706295  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:32.706437  2957 raft_consensus.cc:359] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.707121  2957 raft_consensus.cc:385] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.707393  2957 raft_consensus.cc:740] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.707832  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.708122  2958 ts_tablet_manager.cc:1434] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.006s
I20260709 12:21:32.701212  2959 tablet_bootstrap.cc:654] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.708194  2957 consensus_queue.cc:260] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.708559  2913 raft_consensus.cc:2468] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:32.710577  2957 ts_tablet_manager.cc:1434] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:32.711089  2732 leader_election.cc:304] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:32.711547  2957 tablet_bootstrap.cc:492] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:32.712025  2958 tablet_bootstrap.cc:492] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:32.712273  3084 raft_consensus.cc:2804] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:32.712670  3084 raft_consensus.cc:493] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:32.712989  3084 raft_consensus.cc:3060] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.719089  2957 tablet_bootstrap.cc:654] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:32.719990  2731 leader_election.cc:343] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.721149  2958 tablet_bootstrap.cc:654] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.723393  3026 leader_election.cc:290] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.725376  2969 raft_consensus.cc:493] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.726046  2969 raft_consensus.cc:515] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.729216  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659c2c3a61024d07a2296bc83d87a52d" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.729956  2765 raft_consensus.cc:2468] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:32.730847  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659c2c3a61024d07a2296bc83d87a52d" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.731251  2807 leader_election.cc:304] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
W20260709 12:21:32.732252  2807 leader_election.cc:343] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.732363  2969 leader_election.cc:290] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.732939  3042 raft_consensus.cc:2804] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:32.733333  3042 raft_consensus.cc:493] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:32.733692  3042 raft_consensus.cc:3060] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.737902  3084 raft_consensus.cc:515] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.744478  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:32.746948  3042 raft_consensus.cc:515] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:32.747943  2731 leader_election.cc:343] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.745193  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:32.748974  2913 raft_consensus.cc:3060] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.750614  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659c2c3a61024d07a2296bc83d87a52d" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:32.751238  2765 raft_consensus.cc:3060] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.754451  2958 tablet_bootstrap.cc:492] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:32.754959  2958 ts_tablet_manager.cc:1403] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.000s
I20260709 12:21:32.756152  2957 tablet_bootstrap.cc:492] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:32.756595  2957 ts_tablet_manager.cc:1403] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.045s	user 0.007s	sys 0.007s
I20260709 12:21:32.757941  2958 raft_consensus.cc:359] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.758302  2969 raft_consensus.cc:493] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.758643  2958 raft_consensus.cc:385] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.758915  2765 raft_consensus.cc:2468] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:32.759130  2957 raft_consensus.cc:359] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.759541  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659c2c3a61024d07a2296bc83d87a52d" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:32.759824  2957 raft_consensus.cc:385] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.758947  2958 raft_consensus.cc:740] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.760576  3042 leader_election.cc:290] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.760423  2807 leader_election.cc:304] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:32.758841  2969 raft_consensus.cc:515] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.760870  2958 consensus_queue.cc:260] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.761412  2987 raft_consensus.cc:2804] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:32.760097  2957 raft_consensus.cc:740] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.761924  2987 raft_consensus.cc:697] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:32.762799  2987 consensus_queue.cc:237] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.763823  2969 leader_election.cc:290] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.764488  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.764566  2663 catalog_manager.cc:5697] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:32.766884  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.768230  2958 ts_tablet_manager.cc:1434] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
W20260709 12:21:32.768702  2807 leader_election.cc:343] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.762769  2957 consensus_queue.cc:260] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.769295  2958 tablet_bootstrap.cc:492] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
W20260709 12:21:32.769455  2807 leader_election.cc:343] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.774317  2914 raft_consensus.cc:3060] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.774554  2807 leader_election.cc:304] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:32.775365  2985 raft_consensus.cc:2749] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:32.769232  3084 leader_election.cc:290] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.770084  2913 raft_consensus.cc:2468] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:32.777951  2732 leader_election.cc:304] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:32.779899  3084 raft_consensus.cc:2804] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:32.780460  3084 raft_consensus.cc:697] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:32.781389  3084 consensus_queue.cc:237] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.783491  2959 tablet_bootstrap.cc:492] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:32.784000  2985 consensus_queue.cc:1048] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:32.786475  2958 tablet_bootstrap.cc:654] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.784014  2959 ts_tablet_manager.cc:1403] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.097s	user 0.019s	sys 0.016s
I20260709 12:21:32.792032  2959 raft_consensus.cc:359] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.792577  2959 raft_consensus.cc:385] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.792824  2959 raft_consensus.cc:740] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.793465  2959 consensus_queue.cc:260] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.795833  2959 ts_tablet_manager.cc:1434] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 12:21:32.795944  2666 catalog_manager.cc:5697] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:32.798609  2959 tablet_bootstrap.cc:492] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:32.802569  2958 tablet_bootstrap.cc:492] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:32.803185  2958 ts_tablet_manager.cc:1403] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.002s
I20260709 12:21:32.804370  2664 catalog_manager.cc:5697] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:32.805992  2959 tablet_bootstrap.cc:654] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.806533  2958 raft_consensus.cc:359] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.807358  2958 raft_consensus.cc:385] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.807677  2958 raft_consensus.cc:740] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.808691  3100 raft_consensus.cc:493] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.809216  3100 raft_consensus.cc:515] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.808503  2958 consensus_queue.cc:260] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.811631  3100 leader_election.cc:290] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:32.812849  2958 ts_tablet_manager.cc:1434] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:21:32.814438  2958 tablet_bootstrap.cc:492] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:32.814772  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.815557  2768 raft_consensus.cc:2468] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:32.817065  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:32.817718  2959 tablet_bootstrap.cc:492] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:32.818270  2959 ts_tablet_manager.cc:1403] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.012s
W20260709 12:21:32.819299  2880 leader_election.cc:343] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.820169  2881 leader_election.cc:304] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:32.820960  3026 raft_consensus.cc:493] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.821609  2958 tablet_bootstrap.cc:654] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.821199  2959 raft_consensus.cc:359] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.821506  3026 raft_consensus.cc:515] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.822100  2959 raft_consensus.cc:385] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.822232  3100 raft_consensus.cc:2804] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:32.822423  2959 raft_consensus.cc:740] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.822719  3100 raft_consensus.cc:493] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:32.823130  3100 raft_consensus.cc:3060] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.823256  2959 consensus_queue.cc:260] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.824024  3026 leader_election.cc:290] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.827131  2959 ts_tablet_manager.cc:1434] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:21:32.828481  2959 tablet_bootstrap.cc:492] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:32.828927  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
W20260709 12:21:32.830546  2731 leader_election.cc:343] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.835004  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.838205  3084 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:32.840039  2959 tablet_bootstrap.cc:654] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.852697  2914 raft_consensus.cc:2468] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:32.859592  2957 ts_tablet_manager.cc:1434] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.103s	user 0.009s	sys 0.001s
I20260709 12:21:32.859810  3100 raft_consensus.cc:515] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.860772  2957 tablet_bootstrap.cc:492] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:32.861662  2915 raft_consensus.cc:2393] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:32.862744  3100 leader_election.cc:290] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:32.865346  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:32.866097  2768 raft_consensus.cc:3060] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.869407  2957 tablet_bootstrap.cc:654] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.873452  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:32.874432  2732 leader_election.cc:304] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:32.874436  2958 tablet_bootstrap.cc:492] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:32.875754  2958 ts_tablet_manager.cc:1403] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.062s	user 0.020s	sys 0.019s
I20260709 12:21:32.876514  2987 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:32.877749  2880 leader_election.cc:343] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.891749  2958 raft_consensus.cc:359] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.894212  2958 raft_consensus.cc:385] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.895293  2958 raft_consensus.cc:740] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.895153  2987 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:32.899405  2958 consensus_queue.cc:260] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.900934  2987 raft_consensus.cc:493] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.904865  2987 raft_consensus.cc:515] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.907011  2958 ts_tablet_manager.cc:1434] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.002s
I20260709 12:21:32.903148  2768 raft_consensus.cc:2468] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:32.908164  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.910900  3084 raft_consensus.cc:2749] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:32.908958  2987 leader_election.cc:290] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:32.904315  2957 tablet_bootstrap.cc:492] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:32.912462  2957 ts_tablet_manager.cc:1403] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.003s
I20260709 12:21:32.913448  2881 leader_election.cc:304] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:32.914970  3100 raft_consensus.cc:2804] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:32.910526  2958 tablet_bootstrap.cc:492] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:32.915518  3100 raft_consensus.cc:697] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:32.912973  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.916719  2914 raft_consensus.cc:2393] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a10672cd53140d59ff945f75345cc06 in current term 1: Already voted for candidate 442c92e5267046cd92f46e6f469570a5 in this term.
I20260709 12:21:32.916460  3100 consensus_queue.cc:237] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.918610  2957 raft_consensus.cc:359] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.918527  2807 leader_election.cc:304] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:32.929067  2985 raft_consensus.cc:3060] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:32.930208  2881 consensus_peers.cc:597] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:32.937350  2985 raft_consensus.cc:2749] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:32.938304  2957 raft_consensus.cc:385] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.939023  2957 raft_consensus.cc:740] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.940601  2957 consensus_queue.cc:260] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.944316  2957 ts_tablet_manager.cc:1434] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 12:21:32.947321  2957 tablet_bootstrap.cc:492] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:32.954919  2985 raft_consensus.cc:493] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.955479  2985 raft_consensus.cc:515] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.956490  2958 tablet_bootstrap.cc:654] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.958416  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710edfdfdd7e4720b77f4ec27f5f41c0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:32.959028  2985 leader_election.cc:290] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:32.959873  2807 leader_election.cc:343] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:32.959978  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710edfdfdd7e4720b77f4ec27f5f41c0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:32.960997  2914 raft_consensus.cc:2468] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:32.969918  3026 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:32.972973  2957 tablet_bootstrap.cc:654] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.979519  2958 tablet_bootstrap.cc:492] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:32.980130  2958 ts_tablet_manager.cc:1403] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.005s
I20260709 12:21:32.984131  2958 raft_consensus.cc:359] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.985255  2958 raft_consensus.cc:385] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.985615  2958 raft_consensus.cc:740] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.987475  2958 consensus_queue.cc:260] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:32.950647  2915 raft_consensus.cc:3060] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:32.990520  2958 ts_tablet_manager.cc:1434] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:21:32.991518  2958 tablet_bootstrap.cc:492] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:32.994041  3100 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.014379  2807 leader_election.cc:304] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:33.015784  2969 raft_consensus.cc:2804] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.016151  2969 raft_consensus.cc:493] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.016475  2969 raft_consensus.cc:3060] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.018393  2959 tablet_bootstrap.cc:492] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:33.018921  2959 ts_tablet_manager.cc:1403] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.191s	user 0.008s	sys 0.034s
I20260709 12:21:33.026093  2969 raft_consensus.cc:515] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.027925  2957 tablet_bootstrap.cc:492] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:33.028652  2957 ts_tablet_manager.cc:1403] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.082s	user 0.009s	sys 0.005s
I20260709 12:21:33.029309  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710edfdfdd7e4720b77f4ec27f5f41c0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
W20260709 12:21:33.030923  2807 leader_election.cc:343] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.031682  2959 raft_consensus.cc:359] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.032428  2959 raft_consensus.cc:385] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.032301  2957 raft_consensus.cc:359] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.032765  2959 raft_consensus.cc:740] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.033015  2957 raft_consensus.cc:385] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.033342  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710edfdfdd7e4720b77f4ec27f5f41c0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:33.034142  2914 raft_consensus.cc:3060] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.040491  2958 tablet_bootstrap.cc:654] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.040902  2969 leader_election.cc:290] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.033347  2957 raft_consensus.cc:740] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.042495  2957 consensus_queue.cc:260] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.034142  2959 consensus_queue.cc:260] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.045589  3071 consensus_queue.cc:1048] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:33.048260  2914 raft_consensus.cc:2468] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:33.050830  2957 ts_tablet_manager.cc:1434] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.001s
I20260709 12:21:33.059507  2957 tablet_bootstrap.cc:492] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:33.051576  2959 ts_tablet_manager.cc:1434] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.002s
I20260709 12:21:33.066270  2957 tablet_bootstrap.cc:654] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.067194  2959 tablet_bootstrap.cc:492] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:33.067682  2958 tablet_bootstrap.cc:492] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:33.068197  2958 ts_tablet_manager.cc:1403] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.077s	user 0.016s	sys 0.024s
I20260709 12:21:33.068879  2807 leader_election.cc:304] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:33.070256  2985 raft_consensus.cc:2804] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.070785  2985 raft_consensus.cc:697] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:33.071969  2985 consensus_queue.cc:237] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.078480  2959 tablet_bootstrap.cc:654] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.085520  2957 tablet_bootstrap.cc:492] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:33.086040  2957 ts_tablet_manager.cc:1403] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.006s
I20260709 12:21:33.086061  2666 catalog_manager.cc:5697] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.087576  2985 raft_consensus.cc:493] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.088063  2985 raft_consensus.cc:515] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.089896  2957 raft_consensus.cc:359] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.090656  2957 raft_consensus.cc:385] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.090967  2957 raft_consensus.cc:740] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.091180  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:33.091833  2957 consensus_queue.cc:260] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:33.093616  2807 leader_election.cc:343] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.094548  2957 ts_tablet_manager.cc:1434] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:33.095801  2957 tablet_bootstrap.cc:492] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:33.100091  2985 leader_election.cc:290] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.101948  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:33.102619  2957 tablet_bootstrap.cc:654] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:33.103883  2807 leader_election.cc:343] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.104332  2807 leader_election.cc:304] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:33.105376  2985 raft_consensus.cc:2749] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:33.101051  2958 raft_consensus.cc:359] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.106793  2958 raft_consensus.cc:385] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.107067  2958 raft_consensus.cc:740] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.107849  2958 consensus_queue.cc:260] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.109730  2958 ts_tablet_manager.cc:1434] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.002s
I20260709 12:21:33.110725  2958 tablet_bootstrap.cc:492] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:33.110653  2985 raft_consensus.cc:493] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.111367  2985 raft_consensus.cc:515] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.114327  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6979a63415d34b6d91bb669ecec01bdf" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:33.115000  2765 raft_consensus.cc:2468] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:33.115572  2985 leader_election.cc:290] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.117813  2807 leader_election.cc:304] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:33.114979  2957 tablet_bootstrap.cc:492] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:33.121378  3071 raft_consensus.cc:2804] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.121871  3071 raft_consensus.cc:493] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.122226  3071 raft_consensus.cc:3060] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.121440  2957 ts_tablet_manager.cc:1403] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.004s
I20260709 12:21:33.125500  2957 raft_consensus.cc:359] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.129771  2957 raft_consensus.cc:385] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.130100  2957 raft_consensus.cc:740] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.130847  2957 consensus_queue.cc:260] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.126600  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6979a63415d34b6d91bb669ecec01bdf" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:33.127470  2959 tablet_bootstrap.cc:492] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
W20260709 12:21:33.128427  2714 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:33.132455  2959 ts_tablet_manager.cc:1403] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.066s	user 0.016s	sys 0.002s
W20260709 12:21:33.132836  2807 leader_election.cc:343] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.131945  3071 raft_consensus.cc:515] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.133843  2957 ts_tablet_manager.cc:1434] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 12:21:33.135154  2957 tablet_bootstrap.cc:492] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:33.135216  2959 raft_consensus.cc:359] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.135888  2959 raft_consensus.cc:385] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.136138  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6979a63415d34b6d91bb669ecec01bdf" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:33.136698  2765 raft_consensus.cc:3060] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.131057  2663 catalog_manager.cc:5697] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.141968  3100 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:33.142984  2957 tablet_bootstrap.cc:654] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.145020  2765 raft_consensus.cc:2468] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:33.145694  2958 tablet_bootstrap.cc:654] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.136173  2959 raft_consensus.cc:740] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.148854  2959 consensus_queue.cc:260] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.149737  2807 leader_election.cc:304] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:33.151265  2959 ts_tablet_manager.cc:1434] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:21:33.152312  2959 tablet_bootstrap.cc:492] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:33.154856  3022 raft_consensus.cc:493] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.155427  3022 raft_consensus.cc:515] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.158079  3071 leader_election.cc:290] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.158720  3071 raft_consensus.cc:2804] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.159211  3022 leader_election.cc:290] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.159590  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2594ea75a8d44503a64ea4e06916c4b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:33.161146  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2594ea75a8d44503a64ea4e06916c4b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
W20260709 12:21:33.161604  2732 leader_election.cc:343] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.161845  2843 raft_consensus.cc:2468] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:33.160679  3071 raft_consensus.cc:697] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:33.162451  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6979a63415d34b6d91bb669ecec01bdf" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:33.163403  3071 consensus_queue.cc:237] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:33.165659  2807 leader_election.cc:343] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.166785  2731 leader_election.cc:304] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:33.167343  3100 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:33.168023  2957 tablet_bootstrap.cc:492] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:33.168490  2957 ts_tablet_manager.cc:1403] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.005s
I20260709 12:21:33.169628  2958 tablet_bootstrap.cc:492] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:33.170750  2958 ts_tablet_manager.cc:1403] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.000s
I20260709 12:21:33.172277  2957 raft_consensus.cc:359] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.173609  2957 raft_consensus.cc:385] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.173426  2958 raft_consensus.cc:359] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.174033  2957 raft_consensus.cc:740] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.174212  2958 raft_consensus.cc:385] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.174546  2958 raft_consensus.cc:740] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.174942  2957 consensus_queue.cc:260] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.177850  2957 ts_tablet_manager.cc:1434] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 12:21:33.178148  3026 raft_consensus.cc:2804] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.178546  3026 raft_consensus.cc:493] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.178776  2957 tablet_bootstrap.cc:492] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:33.178917  3026 raft_consensus.cc:3060] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.181676  3084 consensus_queue.cc:1048] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:33.195705  2961 raft_consensus.cc:493] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.196513  2961 raft_consensus.cc:515] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.206746  3022 raft_consensus.cc:493] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.207597  3022 raft_consensus.cc:515] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.214695  2961 leader_election.cc:290] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:33.175226  2958 consensus_queue.cc:260] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.218689  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34453a67223842ed98d4220533186ade" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:33.219476  2843 raft_consensus.cc:2468] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:33.221793  2880 leader_election.cc:304] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:33.222496  2957 tablet_bootstrap.cc:654] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.222738  2961 raft_consensus.cc:2804] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.222982  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34453a67223842ed98d4220533186ade" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:33.223371  2958 ts_tablet_manager.cc:1434] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 12:21:33.223165  2961 raft_consensus.cc:493] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.223930  2961 raft_consensus.cc:3060] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.224471  2958 tablet_bootstrap.cc:492] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
W20260709 12:21:33.224567  2881 leader_election.cc:343] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.219444  2959 tablet_bootstrap.cc:654] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.232936  2961 raft_consensus.cc:515] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.234711  2959 tablet_bootstrap.cc:492] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:33.235239  2959 ts_tablet_manager.cc:1403] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.083s	user 0.010s	sys 0.001s
I20260709 12:21:33.236888  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34453a67223842ed98d4220533186ade" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:33.238984  3022 leader_election.cc:290] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:33.239864  2881 leader_election.cc:343] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.240464  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:33.241221  2913 raft_consensus.cc:2468] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:33.242730  2732 leader_election.cc:304] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:33.243301  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34453a67223842ed98d4220533186ade" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:33.244061  2841 raft_consensus.cc:3060] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.247334  3022 raft_consensus.cc:2804] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.247766  3022 raft_consensus.cc:493] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.248117  3022 raft_consensus.cc:3060] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.253791  2841 raft_consensus.cc:2468] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:33.254786  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:33.256342  2842 raft_consensus.cc:2468] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:33.255491  2961 leader_election.cc:290] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:33.259413  2666 catalog_manager.cc:5697] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.259951  2880 leader_election.cc:304] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:33.261444  2961 raft_consensus.cc:2804] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.262279  2961 raft_consensus.cc:697] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:33.263294  2961 consensus_queue.cc:237] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.247768  3026 raft_consensus.cc:515] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.277717  3026 leader_election.cc:290] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.278384  3026 consensus_queue.cc:1048] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:33.279697  2957 tablet_bootstrap.cc:492] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:33.280205  2957 ts_tablet_manager.cc:1403] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.102s	user 0.027s	sys 0.038s
I20260709 12:21:33.285012  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2594ea75a8d44503a64ea4e06916c4b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:33.285694  2913 raft_consensus.cc:3060] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.283025  2957 raft_consensus.cc:359] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.287943  2957 raft_consensus.cc:385] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.288255  2957 raft_consensus.cc:740] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.288945  2957 consensus_queue.cc:260] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.294728  2959 raft_consensus.cc:359] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.295758  2959 raft_consensus.cc:385] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.296093  2959 raft_consensus.cc:740] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.301370  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2594ea75a8d44503a64ea4e06916c4b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:33.302248  2843 raft_consensus.cc:3060] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.304507  2957 ts_tablet_manager.cc:1434] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 12:21:33.303838  2958 tablet_bootstrap.cc:654] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.303076  2959 consensus_queue.cc:260] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.304447  3022 raft_consensus.cc:515] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.313812  2959 ts_tablet_manager.cc:1434] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.078s	user 0.006s	sys 0.006s
I20260709 12:21:33.315285  2959 tablet_bootstrap.cc:492] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:33.315904  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:33.316574  2914 raft_consensus.cc:3060] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.317094  2913 raft_consensus.cc:2468] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:33.306629  2957 tablet_bootstrap.cc:492] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:33.323999  2959 tablet_bootstrap.cc:654] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.323872  3022 leader_election.cc:290] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.322137  2732 leader_election.cc:304] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:33.342350  3022 raft_consensus.cc:2804] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.343146  3022 raft_consensus.cc:697] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:33.343528  2958 tablet_bootstrap.cc:492] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:33.338917  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:33.344084  2958 ts_tablet_manager.cc:1403] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.120s	user 0.010s	sys 0.007s
I20260709 12:21:33.344403  2840 raft_consensus.cc:3060] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.335767  2914 raft_consensus.cc:2468] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:33.344103  3022 consensus_queue.cc:237] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.347632  2732 leader_election.cc:304] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:33.348846  3026 raft_consensus.cc:2804] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.349356  3026 raft_consensus.cc:697] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:33.323154  2843 raft_consensus.cc:2468] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:33.353372  2957 tablet_bootstrap.cc:654] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.353950  3026 consensus_queue.cc:237] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.356506  2958 raft_consensus.cc:359] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.358067  2958 raft_consensus.cc:385] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.357968  2985 consensus_queue.cc:1048] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:33.358418  3140 raft_consensus.cc:493] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.358492  2958 raft_consensus.cc:740] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.359005  3140 raft_consensus.cc:515] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.359827  2958 consensus_queue.cc:260] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.361980  3140 leader_election.cc:290] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.362133  2958 ts_tablet_manager.cc:1434] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:21:33.363739  2958 tablet_bootstrap.cc:492] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:33.366339  2961 raft_consensus.cc:493] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.367535  3071 raft_consensus.cc:493] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.367726  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe7ee185c0114c079f53469226dee8bf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:33.368048  3071 raft_consensus.cc:515] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.370174  3071 leader_election.cc:290] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.367909  2961 raft_consensus.cc:515] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.379611  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe7ee185c0114c079f53469226dee8bf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:33.381239  2961 leader_election.cc:290] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:33.381672  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:33.381879  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:33.382424  2766 raft_consensus.cc:2468] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:33.382592  2841 raft_consensus.cc:2468] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:33.383399  2840 raft_consensus.cc:2468] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:33.383591  3071 consensus_queue.cc:1048] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:33.385246  2880 leader_election.cc:304] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:33.387486  2961 raft_consensus.cc:2804] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.388557  2843 raft_consensus.cc:2468] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:33.391301  2731 leader_election.cc:304] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:33.396812  3012 raft_consensus.cc:493] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.398293  3012 raft_consensus.cc:515] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:33.399170  2732 leader_election.cc:343] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.388899  2961 raft_consensus.cc:493] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.401650  2961 raft_consensus.cc:3060] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.403798  3026 raft_consensus.cc:2804] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.404394  3026 raft_consensus.cc:493] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.396282  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe7ee185c0114c079f53469226dee8bf" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:33.420904  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:33.422194  2807 leader_election.cc:343] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.422621  2958 tablet_bootstrap.cc:654] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:33.422899  2881 leader_election.cc:343] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.425410  2666 catalog_manager.cc:5697] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.423470  3100 consensus_queue.cc:1048] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:21:33.419317  3026 raft_consensus.cc:3060] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.441756  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:33.442713  2843 raft_consensus.cc:2468] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:33.453495  2880 leader_election.cc:304] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:33.454545  3100 raft_consensus.cc:2804] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.454937  3100 raft_consensus.cc:493] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.455312  3100 raft_consensus.cc:3060] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:33.457638  2880 consensus_peers.cc:597] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:33.435715  3012 leader_election.cc:290] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:21:33.461200  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 68262.6us
I20260709 12:21:33.462918  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe7ee185c0114c079f53469226dee8bf" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:33.463716  2957 tablet_bootstrap.cc:492] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:33.462425  2664 catalog_manager.cc:5697] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.464272  2957 ts_tablet_manager.cc:1403] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.158s	user 0.005s	sys 0.011s
I20260709 12:21:33.465106  2664 catalog_manager.cc:5697] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.481005  2957 raft_consensus.cc:359] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.483144  2957 raft_consensus.cc:385] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.483611  2957 raft_consensus.cc:740] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.484483  2957 consensus_queue.cc:260] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.472681  2961 raft_consensus.cc:515] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.512760  2961 leader_election.cc:290] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:33.489336  3026 raft_consensus.cc:515] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.516324  2959 tablet_bootstrap.cc:492] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:33.516912  2959 ts_tablet_manager.cc:1403] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.202s	user 0.008s	sys 0.015s
I20260709 12:21:33.519556  3100 raft_consensus.cc:515] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.522327  2768 raft_consensus.cc:2393] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a10672cd53140d59ff945f75345cc06 in current term 1: Already voted for candidate 442c92e5267046cd92f46e6f469570a5 in this term.
I20260709 12:21:33.523947  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:33.524940  2843 raft_consensus.cc:3060] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.526224  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:33.524530  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:33.527400  2841 raft_consensus.cc:3060] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.527544  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:33.528203  2767 raft_consensus.cc:3060] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.530249  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe7ee185c0114c079f53469226dee8bf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:33.531093  2913 raft_consensus.cc:3060] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.535655  2767 raft_consensus.cc:2468] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:33.516940  3026 leader_election.cc:290] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.537496  2881 leader_election.cc:304] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:33.545792  3152 raft_consensus.cc:2804] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.546397  3152 raft_consensus.cc:697] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:33.546197  2959 raft_consensus.cc:359] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.548566  2959 raft_consensus.cc:385] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.548841  2959 raft_consensus.cc:740] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.550917  2957 ts_tablet_manager.cc:1434] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.086s	user 0.015s	sys 0.022s
I20260709 12:21:33.553004  2957 tablet_bootstrap.cc:492] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:33.554644  2959 consensus_queue.cc:260] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.539628  3100 leader_election.cc:290] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:33.564790  2987 raft_consensus.cc:493] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.547677  3152 consensus_queue.cc:237] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.565387  2987 raft_consensus.cc:515] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.565958  2959 ts_tablet_manager.cc:1434] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.049s	user 0.004s	sys 0.006s
I20260709 12:21:33.567068  2959 tablet_bootstrap.cc:492] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:33.567524  2987 leader_election.cc:290] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.567910  2958 tablet_bootstrap.cc:492] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:33.568353  2958 ts_tablet_manager.cc:1403] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.205s	user 0.032s	sys 0.014s
I20260709 12:21:33.569478  2807 leader_election.cc:304] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:33.576867  2958 raft_consensus.cc:359] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.577680  2958 raft_consensus.cc:385] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.577953  2958 raft_consensus.cc:740] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.579303  2765 raft_consensus.cc:3060] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.579921  2987 raft_consensus.cc:3060] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.601089  2958 consensus_queue.cc:260] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.603497  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe7ee185c0114c079f53469226dee8bf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:33.610443  2957 tablet_bootstrap.cc:654] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.610790  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:33.612295  2841 raft_consensus.cc:2468] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:33.613427  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:33.613790  2880 leader_election.cc:304] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:33.615410  3100 raft_consensus.cc:2804] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.615911  3100 raft_consensus.cc:697] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:33.617048  3100 consensus_queue.cc:237] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.625679  2958 ts_tablet_manager.cc:1434] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.057s	user 0.011s	sys 0.005s
W20260709 12:21:33.625885  2807 leader_election.cc:343] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.626665  2958 tablet_bootstrap.cc:492] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
W20260709 12:21:33.626899  2807 consensus_peers.cc:597] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:33.628392  2807 leader_election.cc:343] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.629029  2807 leader_election.cc:304] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:33.622962  2913 raft_consensus.cc:2468] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:33.630275  2843 raft_consensus.cc:2468] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:33.629483  2985 consensus_queue.cc:1048] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:21:33.630873  3071 raft_consensus.cc:2749] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:33.648053  2732 leader_election.cc:304] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:33.649570  3084 raft_consensus.cc:2804] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.650902  3084 raft_consensus.cc:697] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:33.657560  2765 raft_consensus.cc:2468] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
W20260709 12:21:33.658313  2807 consensus_peers.cc:597] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:33.659210  3140 raft_consensus.cc:493] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.659718  3140 raft_consensus.cc:515] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.662811  3140 leader_election.cc:290] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.664238  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:33.664776  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:33.664974  2843 raft_consensus.cc:2468] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:33.665409  3084 consensus_queue.cc:237] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.683736  2731 leader_election.cc:304] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:33.685600  3026 raft_consensus.cc:2804] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:33.687587  3026 raft_consensus.cc:493] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.695053  3026 raft_consensus.cc:3060] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.702251  2959 tablet_bootstrap.cc:654] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.707973  3100 consensus_queue.cc:1048] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.719494  2987 raft_consensus.cc:2749] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:33.723734  3026 raft_consensus.cc:515] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.773164  3157 consensus_queue.cc:1048] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.773195  2840 raft_consensus.cc:2468] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:33.778992  2957 tablet_bootstrap.cc:492] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:33.779572  2957 ts_tablet_manager.cc:1403] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.227s	user 0.028s	sys 0.018s
I20260709 12:21:33.781926  2959 tablet_bootstrap.cc:492] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:33.781972  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:33.782539  2959 ts_tablet_manager.cc:1403] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.216s	user 0.015s	sys 0.026s
I20260709 12:21:33.782624  2843 raft_consensus.cc:3060] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.783784  3084 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.783823  2957 raft_consensus.cc:359] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.784550  2957 raft_consensus.cc:385] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.784823  2957 raft_consensus.cc:740] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.786094  2957 consensus_queue.cc:260] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.787639  2959 raft_consensus.cc:359] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.781308  3026 leader_election.cc:290] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:33.788338  2959 raft_consensus.cc:385] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.788637  2959 raft_consensus.cc:740] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.789356  2959 consensus_queue.cc:260] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.796128  2959 ts_tablet_manager.cc:1434] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:21:33.799100  2959 tablet_bootstrap.cc:492] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:33.801249  2957 ts_tablet_manager.cc:1434] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.001s
I20260709 12:21:33.801824  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:33.802404  2957 tablet_bootstrap.cc:492] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:33.792774  2843 raft_consensus.cc:2468] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:21:33.804804  2731 leader_election.cc:304] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:33.805881  3084 raft_consensus.cc:2804] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:33.806457  3084 raft_consensus.cc:697] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:33.808326  3084 consensus_queue.cc:237] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.796968  2664 catalog_manager.cc:5697] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.814285  2959 tablet_bootstrap.cc:654] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.809832  2957 tablet_bootstrap.cc:654] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.811434  2958 tablet_bootstrap.cc:654] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.825950  3100 raft_consensus.cc:493] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.826530  3100 raft_consensus.cc:515] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.828846  3100 leader_election.cc:290] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:33.831676  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:33.832451  2840 raft_consensus.cc:2468] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:33.833820  2880 leader_election.cc:304] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:33.834782  3100 raft_consensus.cc:2804] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.835187  3100 raft_consensus.cc:493] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.835598  3100 raft_consensus.cc:3060] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.836078  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:33.837666  2881 leader_election.cc:343] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.849076  2663 catalog_manager.cc:5697] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.853034  2663 catalog_manager.cc:5697] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.860601  3140 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.864704  2959 tablet_bootstrap.cc:492] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:33.865243  2959 ts_tablet_manager.cc:1403] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.003s
I20260709 12:21:33.869446  2959 raft_consensus.cc:359] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.870148  2959 raft_consensus.cc:385] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.870424  2959 raft_consensus.cc:740] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.878479  2959 consensus_queue.cc:260] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.879496  2957 tablet_bootstrap.cc:492] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:33.885186  2957 ts_tablet_manager.cc:1403] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.083s	user 0.012s	sys 0.005s
I20260709 12:21:33.885300  2958 tablet_bootstrap.cc:492] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:33.885818  2958 ts_tablet_manager.cc:1403] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.259s	user 0.027s	sys 0.012s
I20260709 12:21:33.882575  3140 consensus_queue.cc:1048] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:33.882426  3100 raft_consensus.cc:515] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.903481  3100 leader_election.cc:290] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:33.904194  2958 raft_consensus.cc:359] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.905071  2958 raft_consensus.cc:385] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.905406  2958 raft_consensus.cc:740] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.906335  2958 consensus_queue.cc:260] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.880554  2765 raft_consensus.cc:3060] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.888231  2957 raft_consensus.cc:359] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.906975  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:33.907435  2957 raft_consensus.cc:385] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.890525  2766 raft_consensus.cc:3060] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.890038  2664 catalog_manager.cc:5697] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.910329  2958 ts_tablet_manager.cc:1434] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
W20260709 12:21:33.911134  2881 leader_election.cc:343] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.911332  2958 tablet_bootstrap.cc:492] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:33.912582  3135 consensus_queue.cc:1048] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.891422  2959 ts_tablet_manager.cc:1434] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:21:33.907770  2957 raft_consensus.cc:740] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.922279  2959 tablet_bootstrap.cc:492] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:33.921934  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:33.923882  2957 consensus_queue.cc:260] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.924289  2841 raft_consensus.cc:3060] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.928870  2957 ts_tablet_manager.cc:1434] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.003s
I20260709 12:21:33.930197  2957 tablet_bootstrap.cc:492] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:33.953094  3152 consensus_queue.cc:1048] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.948894  2841 raft_consensus.cc:2468] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:33.955962  2959 tablet_bootstrap.cc:654] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.955881  2880 leader_election.cc:304] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:33.956768  3152 raft_consensus.cc:2804] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.957311  3152 raft_consensus.cc:697] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:33.958127  3152 consensus_queue.cc:237] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.964749  3157 raft_consensus.cc:493] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.965502  3157 raft_consensus.cc:515] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.983022  2987 raft_consensus.cc:493] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.983626  2987 raft_consensus.cc:515] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:33.987615  3100 consensus_queue.cc:1048] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.992677  2958 tablet_bootstrap.cc:654] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.997588  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:34.000604  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:34.003118  2957 tablet_bootstrap.cc:654] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.996441  2987 leader_election.cc:290] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.004619  3157 leader_election.cc:290] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.006484  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:34.006839  2959 tablet_bootstrap.cc:492] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:34.007138  2768 raft_consensus.cc:2468] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:34.007304  2959 ts_tablet_manager.cc:1403] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.092s	user 0.014s	sys 0.008s
I20260709 12:21:34.007182  2807 leader_election.cc:304] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:34.010032  3154 raft_consensus.cc:2749] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:34.014083  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:34.014843  2915 raft_consensus.cc:2468] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:34.015095  2807 leader_election.cc:304] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:34.019436  3157 raft_consensus.cc:2804] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:34.020984  3157 raft_consensus.cc:493] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:34.021297  2958 tablet_bootstrap.cc:492] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.012701  2959 raft_consensus.cc:359] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.022627  2959 raft_consensus.cc:385] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.023653  2959 raft_consensus.cc:740] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.021472  3157 raft_consensus.cc:3060] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.027827  2959 consensus_queue.cc:260] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.029595  2663 catalog_manager.cc:5697] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:34.021822  2958 ts_tablet_manager.cc:1403] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.111s	user 0.011s	sys 0.004s
I20260709 12:21:34.038937  2958 raft_consensus.cc:359] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.039709  2958 raft_consensus.cc:385] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.040032  2958 raft_consensus.cc:740] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.040719  2958 consensus_queue.cc:260] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.042862  2958 ts_tablet_manager.cc:1434] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:34.043830  2958 tablet_bootstrap.cc:492] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.048341  3157 raft_consensus.cc:515] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.050009  2961 consensus_queue.cc:1048] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:34.051316  3157 leader_election.cc:290] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.052117  2841 raft_consensus.cc:3060] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.056550  2958 tablet_bootstrap.cc:654] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.058528  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:34.059190  2914 raft_consensus.cc:3060] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.065228  2957 tablet_bootstrap.cc:492] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:34.065802  2957 ts_tablet_manager.cc:1403] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.136s	user 0.014s	sys 0.007s
I20260709 12:21:34.069278  2957 raft_consensus.cc:359] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.070106  2957 raft_consensus.cc:385] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.070454  2957 raft_consensus.cc:740] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.071178  2957 consensus_queue.cc:260] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.073455  2957 ts_tablet_manager.cc:1434] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:21:34.076640  2957 tablet_bootstrap.cc:492] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:34.080142  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021d84b2e9364749b9ae4e7fde0b14bc" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:34.080906  2768 raft_consensus.cc:3060] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.084339  2959 ts_tablet_manager.cc:1434] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.002s
I20260709 12:21:34.086292  2959 tablet_bootstrap.cc:492] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:34.088536  2768 raft_consensus.cc:2468] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 2.
I20260709 12:21:34.090739  2957 tablet_bootstrap.cc:654] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.104678  2807 leader_election.cc:304] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:34.105964  3154 raft_consensus.cc:2804] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:34.106480  3154 raft_consensus.cc:697] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 2 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:34.107415  3154 consensus_queue.cc:237] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.093019  2914 raft_consensus.cc:2468] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 2.
I20260709 12:21:34.126057  2959 tablet_bootstrap.cc:654] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.127460  2840 raft_consensus.cc:1240] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Rejecting Update request from peer 7cc87046f1414b27928722690a5e388f for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:34.130657  2665 catalog_manager.cc:5697] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 2, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:34.131323  3153 consensus_queue.cc:1059] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:34.132503  3153 raft_consensus.cc:3055] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:34.132854  3153 raft_consensus.cc:740] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:34.133735  3153 consensus_queue.cc:260] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.134997  3153 raft_consensus.cc:3060] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.135031  3152 raft_consensus.cc:493] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.135617  3152 raft_consensus.cc:515] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.139096  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e8aac6ebd14bf4b952c62c8075f13f" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:34.140753  2881 leader_election.cc:343] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.141737  3152 leader_election.cc:290] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:34.148028  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e8aac6ebd14bf4b952c62c8075f13f" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
W20260709 12:21:34.149705  2880 leader_election.cc:343] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.150192  2880 leader_election.cc:304] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:34.151463  2961 raft_consensus.cc:2749] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:34.159237  2958 tablet_bootstrap.cc:492] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.159963  2958 ts_tablet_manager.cc:1403] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.116s	user 0.041s	sys 0.015s
I20260709 12:21:34.163043  2957 tablet_bootstrap.cc:492] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:34.163111  2958 raft_consensus.cc:359] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.163694  2957 ts_tablet_manager.cc:1403] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.087s	user 0.023s	sys 0.015s
I20260709 12:21:34.163978  2958 raft_consensus.cc:385] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.164279  2958 raft_consensus.cc:740] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.165413  2958 consensus_queue.cc:260] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.166697  2957 raft_consensus.cc:359] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.167441  2957 raft_consensus.cc:385] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.167575  2958 ts_tablet_manager.cc:1434] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:34.167735  2957 raft_consensus.cc:740] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.168619  2958 tablet_bootstrap.cc:492] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.168570  2957 consensus_queue.cc:260] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.175028  2958 tablet_bootstrap.cc:654] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.177258  2957 ts_tablet_manager.cc:1434] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:21:34.180934  2969 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.181701  2969 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.184003  2969 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.186729  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:34.191123  2958 tablet_bootstrap.cc:492] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.191695  2958 ts_tablet_manager.cc:1403] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.000s
W20260709 12:21:34.191776  2714 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:21:34.192993  2807 leader_election.cc:343] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.194176  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:34.194818  2958 raft_consensus.cc:359] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.195562  2958 raft_consensus.cc:385] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.195843  2958 raft_consensus.cc:740] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.196719  2958 consensus_queue.cc:260] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.199069  2958 ts_tablet_manager.cc:1434] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:34.200055  2958 tablet_bootstrap.cc:492] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.204814  2959 tablet_bootstrap.cc:492] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:34.206475  2957 tablet_bootstrap.cc:492] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:34.206874  2958 tablet_bootstrap.cc:654] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:34.209705  2807 leader_election.cc:343] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.213654  2807 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:34.214746  2969 raft_consensus.cc:2749] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:34.217197  2957 tablet_bootstrap.cc:654] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.218353  2958 tablet_bootstrap.cc:492] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.218856  2958 ts_tablet_manager.cc:1403] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.002s
I20260709 12:21:34.205391  2959 ts_tablet_manager.cc:1403] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.119s	user 0.020s	sys 0.013s
I20260709 12:21:34.221705  2958 raft_consensus.cc:359] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.222419  2958 raft_consensus.cc:385] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.222710  2958 raft_consensus.cc:740] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.223311  2958 consensus_queue.cc:260] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.224871  2959 raft_consensus.cc:359] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.225441  2958 ts_tablet_manager.cc:1434] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:34.225602  2959 raft_consensus.cc:385] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.225955  2959 raft_consensus.cc:740] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.226574  2959 consensus_queue.cc:260] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.227358  2958 tablet_bootstrap.cc:492] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.228683  2959 ts_tablet_manager.cc:1434] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.000s
I20260709 12:21:34.229508  2959 tablet_bootstrap.cc:492] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:34.233911  3022 raft_consensus.cc:493] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.234197  2958 tablet_bootstrap.cc:654] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.234577  3022 raft_consensus.cc:515] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.238739  3022 leader_election.cc:290] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.246960  2959 tablet_bootstrap.cc:654] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.250344  2914 raft_consensus.cc:3060] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.272130  2958 tablet_bootstrap.cc:492] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.273468  2958 ts_tablet_manager.cc:1403] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.046s	user 0.023s	sys 0.012s
I20260709 12:21:34.275918  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4cde589c024472b0a4cec236ea69a3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:34.277033  2958 raft_consensus.cc:359] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:34.277596  2732 leader_election.cc:343] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.277843  2958 raft_consensus.cc:385] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.277947  2959 tablet_bootstrap.cc:492] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:34.278393  2958 raft_consensus.cc:740] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.278612  2959 ts_tablet_manager.cc:1403] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.049s	user 0.000s	sys 0.021s
I20260709 12:21:34.279229  2958 consensus_queue.cc:260] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.281507  2958 ts_tablet_manager.cc:1434] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:34.281433  2959 raft_consensus.cc:359] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.282236  2959 raft_consensus.cc:385] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.282505  2959 raft_consensus.cc:740] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.283077  2959 consensus_queue.cc:260] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.285171  2959 ts_tablet_manager.cc:1434] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:21:34.286652  2958 tablet_bootstrap.cc:492] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.288441  3022 consensus_queue.cc:1048] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:34.287498  2959 tablet_bootstrap.cc:492] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:34.292881  3139 consensus_queue.cc:1048] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:34.300362  2959 tablet_bootstrap.cc:654] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.302317  2958 tablet_bootstrap.cc:654] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.308351  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4cde589c024472b0a4cec236ea69a3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:34.309296  2840 raft_consensus.cc:2468] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:34.313166  2840 raft_consensus.cc:1275] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:21:34.328511  2913 raft_consensus.cc:1275] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:21:34.329069  2913 pending_rounds.cc:85] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f: Aborting all ops after (but not including) 0
I20260709 12:21:34.329401  2913 pending_rounds.cc:107] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:21:34.329792  2913 raft_consensus.cc:2889] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:34.332093  3139 consensus_queue.cc:1048] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:34.355625  3022 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.356413  3022 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.360909  2731 leader_election.cc:304] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:34.362661  3139 raft_consensus.cc:2804] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:34.363176  3139 raft_consensus.cc:493] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:34.363703  3139 raft_consensus.cc:3060] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.374369  3022 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.394186  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:34.394654  2957 tablet_bootstrap.cc:492] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:34.394995  2843 raft_consensus.cc:2468] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:34.395182  2957 ts_tablet_manager.cc:1403] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.189s	user 0.047s	sys 0.094s
I20260709 12:21:34.383654  3154 raft_consensus.cc:493] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.397583  3154 raft_consensus.cc:515] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.400754  3140 consensus_queue.cc:1048] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:34.404034  2731 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:34.405341  3140 raft_consensus.cc:2804] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:34.406277  3140 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:34.406741  3140 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.411757  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:34.413249  3154 leader_election.cc:290] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.418624  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0df81094e2493cbe804d2dba9aa9d4" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:34.424691  2958 tablet_bootstrap.cc:492] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.425299  2958 ts_tablet_manager.cc:1403] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.139s	user 0.023s	sys 0.024s
W20260709 12:21:34.427438  2807 leader_election.cc:343] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.435546  2958 raft_consensus.cc:359] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.436328  2958 raft_consensus.cc:385] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.436623  2958 raft_consensus.cc:740] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.439273  2969 raft_consensus.cc:493] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.439860  2969 raft_consensus.cc:515] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.432720  2957 raft_consensus.cc:359] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.442456  2957 raft_consensus.cc:385] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.443414  2958 consensus_queue.cc:260] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.446323  2958 ts_tablet_manager.cc:1434] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:21:34.447715  3071 raft_consensus.cc:493] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.448444  2969 leader_election.cc:290] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.448233  3071 raft_consensus.cc:515] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.450469  3071 leader_election.cc:290] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.453855  3153 raft_consensus.cc:493] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:21:34.453804  3139 raft_consensus.cc:515] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.454582  3153 raft_consensus.cc:515] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.456274  3139 leader_election.cc:290] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.456836  2958 tablet_bootstrap.cc:492] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.458297  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6979a63415d34b6d91bb669ecec01bdf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:34.443432  2957 raft_consensus.cc:740] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.458879  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1a771f5b2742d8bd641ec0afec2cf4" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:34.460909  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:34.461961  2807 leader_election.cc:343] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.462000  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4cde589c024472b0a4cec236ea69a3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:34.462733  2840 raft_consensus.cc:3060] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.463735  2959 tablet_bootstrap.cc:492] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:34.470301  2959 ts_tablet_manager.cc:1403] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.183s	user 0.018s	sys 0.041s
I20260709 12:21:34.472822  2840 raft_consensus.cc:2468] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:34.474309  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6979a63415d34b6d91bb669ecec01bdf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:34.474413  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4cde589c024472b0a4cec236ea69a3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:34.476084  2880 leader_election.cc:304] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:34.467444  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0df81094e2493cbe804d2dba9aa9d4" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:34.476490  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1a771f5b2742d8bd641ec0afec2cf4" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:34.477587  3001 raft_consensus.cc:2749] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:34.467533  2732 leader_election.cc:343] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.472232  3153 leader_election.cc:290] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:34.481359  2731 leader_election.cc:304] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:34.483116  3026 raft_consensus.cc:2804] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:34.483654  3026 raft_consensus.cc:697] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:34.484473  3026 consensus_queue.cc:237] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.477912  2957 consensus_queue.cc:260] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.486526  2917 raft_consensus.cc:3060] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.466112  3140 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.491250  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
W20260709 12:21:34.493983  2807 leader_election.cc:343] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.495417  2959 raft_consensus.cc:359] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:34.495915  2807 leader_election.cc:343] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.501919  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:34.496022  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:34.506508  2840 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.507056  2958 tablet_bootstrap.cc:654] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:34.507241  2732 leader_election.cc:343] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.511492  2917 raft_consensus.cc:2468] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:34.504251  2959 raft_consensus.cc:385] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.513342  2959 raft_consensus.cc:740] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.504844  2961 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:34.506387  2807 leader_election.cc:304] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:34.514286  2959 consensus_queue.cc:260] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:34.514907  2807 leader_election.cc:343] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.515389  2807 leader_election.cc:304] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:34.516670  2959 ts_tablet_manager.cc:1434] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.000s
I20260709 12:21:34.516848  3157 raft_consensus.cc:2749] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:34.517102  2969 raft_consensus.cc:2749] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:34.517746  2959 tablet_bootstrap.cc:492] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:34.518450  2961 raft_consensus.cc:493] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.518908  2961 raft_consensus.cc:515] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:34.516700  2807 leader_election.cc:343] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.521035  2807 leader_election.cc:304] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:34.521870  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:34.523334  3071 raft_consensus.cc:2749] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:34.521956  2961 leader_election.cc:290] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:34.524891  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:34.501451  3140 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:34.526683  2880 leader_election.cc:343] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.526998  2840 raft_consensus.cc:2468] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
W20260709 12:21:34.530390  2881 leader_election.cc:343] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.530887  2881 leader_election.cc:304] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:34.533731  3001 raft_consensus.cc:2749] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:34.535826  2958 tablet_bootstrap.cc:492] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.536450  2958 ts_tablet_manager.cc:1403] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.089s	user 0.013s	sys 0.000s
I20260709 12:21:34.536788  2768 raft_consensus.cc:3060] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.537600  2959 tablet_bootstrap.cc:654] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.541106  2957 ts_tablet_manager.cc:1434] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.146s	user 0.010s	sys 0.002s
I20260709 12:21:34.542266  2957 tablet_bootstrap.cc:492] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:34.542857  2731 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:34.542956  2958 raft_consensus.cc:359] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.543913  2958 raft_consensus.cc:385] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.544198  3026 raft_consensus.cc:2804] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:34.544267  2958 raft_consensus.cc:740] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.544870  3026 raft_consensus.cc:697] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:34.545434  2958 consensus_queue.cc:260] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.545806  3026 consensus_queue.cc:237] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.548162  2958 ts_tablet_manager.cc:1434] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:21:34.549207  2958 tablet_bootstrap.cc:492] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.558845  2958 tablet_bootstrap.cc:654] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.564165  2957 tablet_bootstrap.cc:654] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.573740  2665 catalog_manager.cc:5697] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:34.578369  2957 tablet_bootstrap.cc:492] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:34.578856  3152 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:34.580478  2959 tablet_bootstrap.cc:492] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:34.580701  2957 ts_tablet_manager.cc:1403] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.004s
I20260709 12:21:34.580791  2958 tablet_bootstrap.cc:492] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.580953  2959 ts_tablet_manager.cc:1403] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.063s	user 0.007s	sys 0.004s
I20260709 12:21:34.581279  2958 ts_tablet_manager.cc:1403] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.004s
I20260709 12:21:34.583707  2957 raft_consensus.cc:359] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.584466  2957 raft_consensus.cc:385] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.584770  2957 raft_consensus.cc:740] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.584916  2958 raft_consensus.cc:359] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.585896  2958 raft_consensus.cc:385] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.586244  2958 raft_consensus.cc:740] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.586934  2957 consensus_queue.cc:260] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.586969  2958 consensus_queue.cc:260] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.589516  2958 ts_tablet_manager.cc:1434] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:21:34.590667  2958 tablet_bootstrap.cc:492] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.589577  2957 ts_tablet_manager.cc:1434] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:21:34.594955  2957 tablet_bootstrap.cc:492] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:34.604959  2958 tablet_bootstrap.cc:654] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.605513  2957 tablet_bootstrap.cc:654] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.584040  2959 raft_consensus.cc:359] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.614612  2959 raft_consensus.cc:385] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.615048  2959 raft_consensus.cc:740] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.616016  3157 raft_consensus.cc:493] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.616652  3157 raft_consensus.cc:515] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.619367  3157 leader_election.cc:290] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.620357  2959 consensus_queue.cc:260] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.624670  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e8aac6ebd14bf4b952c62c8075f13f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:34.625671  2917 raft_consensus.cc:2468] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:34.635767  2807 leader_election.cc:304] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:34.637123  3157 raft_consensus.cc:2804] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:34.637568  3157 raft_consensus.cc:493] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:34.637990  3157 raft_consensus.cc:3060] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.638427  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e8aac6ebd14bf4b952c62c8075f13f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:34.640461  2765 raft_consensus.cc:2468] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:34.642371  2987 raft_consensus.cc:493] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.642938  2987 raft_consensus.cc:515] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.646988  2665 catalog_manager.cc:5697] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:34.649703  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:34.651391  2807 leader_election.cc:343] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.653764  2987 leader_election.cc:290] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.655854  3026 raft_consensus.cc:493] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.656358  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:34.656488  3026 raft_consensus.cc:515] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.657279  2957 tablet_bootstrap.cc:492] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:34.657902  2957 ts_tablet_manager.cc:1403] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.017s
I20260709 12:21:34.658911  3026 leader_election.cc:290] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.661473  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710edfdfdd7e4720b77f4ec27f5f41c0" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:34.663149  2959 ts_tablet_manager.cc:1434] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.082s	user 0.004s	sys 0.004s
I20260709 12:21:34.662279  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710edfdfdd7e4720b77f4ec27f5f41c0" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:34.664265  2959 tablet_bootstrap.cc:492] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
W20260709 12:21:34.662587  2807 leader_election.cc:343] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.664973  2807 leader_election.cc:304] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:34.666536  2731 leader_election.cc:304] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:34.667743  3139 raft_consensus.cc:2749] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:34.669061  3157 raft_consensus.cc:515] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.669961  2958 tablet_bootstrap.cc:492] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.670469  2958 ts_tablet_manager.cc:1403] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.080s	user 0.011s	sys 0.022s
I20260709 12:21:34.672047  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e8aac6ebd14bf4b952c62c8075f13f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:34.672501  2987 raft_consensus.cc:2749] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:34.672688  2765 raft_consensus.cc:3060] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.671011  3157 leader_election.cc:290] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.674365  2957 raft_consensus.cc:359] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.675056  2957 raft_consensus.cc:385] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.675339  2957 raft_consensus.cc:740] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.676028  2957 consensus_queue.cc:260] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.678403  2958 raft_consensus.cc:359] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.679069  2958 raft_consensus.cc:385] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.679337  2958 raft_consensus.cc:740] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.680215  2958 consensus_queue.cc:260] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.682331  2765 raft_consensus.cc:2468] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:34.682967  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e8aac6ebd14bf4b952c62c8075f13f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:34.683552  2914 raft_consensus.cc:3060] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.688228  2958 ts_tablet_manager.cc:1434] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:21:34.690155  2958 tablet_bootstrap.cc:492] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.693025  2807 leader_election.cc:304] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:34.682603  2957 ts_tablet_manager.cc:1434] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 12:21:34.693646  2914 raft_consensus.cc:2468] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:34.694324  2987 raft_consensus.cc:2804] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:34.694821  2957 tablet_bootstrap.cc:492] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:34.694861  2987 raft_consensus.cc:697] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:34.698012  2987 consensus_queue.cc:237] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.698699  2768 raft_consensus.cc:1275] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:21:34.701731  3071 raft_consensus.cc:493] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.702261  3071 raft_consensus.cc:515] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.703680  3154 consensus_queue.cc:1048] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:34.705777  2958 tablet_bootstrap.cc:654] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.707783  2917 raft_consensus.cc:1275] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:21:34.709473  3152 raft_consensus.cc:493] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.710093  3152 raft_consensus.cc:515] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.713040  2959 tablet_bootstrap.cc:654] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.713760  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:34.705698  3071 leader_election.cc:290] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:34.716199  2880 leader_election.cc:343] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.717121  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:34.718446  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:34.719352  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
W20260709 12:21:34.719890  2881 leader_election.cc:343] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.720427  2881 leader_election.cc:304] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:34.723214  3001 raft_consensus.cc:2749] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:34.725870  3152 leader_election.cc:290] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:34.707834  2957 tablet_bootstrap.cc:654] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.737979  3154 consensus_queue.cc:1048] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:34.740253  2959 tablet_bootstrap.cc:492] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:34.740811  2959 ts_tablet_manager.cc:1403] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.077s	user 0.007s	sys 0.006s
W20260709 12:21:34.741129  2807 leader_election.cc:343] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.743778  2959 raft_consensus.cc:359] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.744568  2959 raft_consensus.cc:385] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.744846  2959 raft_consensus.cc:740] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.745488  2959 consensus_queue.cc:260] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.748698  2959 ts_tablet_manager.cc:1434] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
W20260709 12:21:34.751382  2807 leader_election.cc:343] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.755676  2959 tablet_bootstrap.cc:492] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:34.751926  2807 leader_election.cc:304] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:34.763019  3154 raft_consensus.cc:2749] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:34.765661  2957 tablet_bootstrap.cc:492] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:34.775434  2957 ts_tablet_manager.cc:1403] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.081s	user 0.015s	sys 0.000s
I20260709 12:21:34.774928  2666 catalog_manager.cc:5697] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:34.776786  2958 tablet_bootstrap.cc:492] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.777264  2958 ts_tablet_manager.cc:1403] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.087s	user 0.020s	sys 0.030s
I20260709 12:21:34.768266  2959 tablet_bootstrap.cc:654] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.780825  2957 raft_consensus.cc:359] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.781670  2957 raft_consensus.cc:385] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.781455  2958 raft_consensus.cc:359] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.782058  2957 raft_consensus.cc:740] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.782269  2958 raft_consensus.cc:385] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.782598  2958 raft_consensus.cc:740] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.782848  2957 consensus_queue.cc:260] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.783278  2958 consensus_queue.cc:260] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.785147  2957 ts_tablet_manager.cc:1434] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 12:21:34.785472  2958 ts_tablet_manager.cc:1434] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:21:34.786206  2957 tablet_bootstrap.cc:492] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:34.786343  2958 tablet_bootstrap.cc:492] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.792557  2958 tablet_bootstrap.cc:654] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.794080  2957 tablet_bootstrap.cc:654] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.795606  2765 raft_consensus.cc:3060] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.804709  2957 tablet_bootstrap.cc:492] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:34.806131  2957 ts_tablet_manager.cc:1403] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.003s
I20260709 12:21:34.804857  2958 tablet_bootstrap.cc:492] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.809165  2958 ts_tablet_manager.cc:1403] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.004s
I20260709 12:21:34.809820  2957 raft_consensus.cc:359] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.810576  2957 raft_consensus.cc:385] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.810894  2957 raft_consensus.cc:740] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.811632  2957 consensus_queue.cc:260] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.812274  2958 raft_consensus.cc:359] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.812933  2958 raft_consensus.cc:385] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.813215  2958 raft_consensus.cc:740] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.813230  2959 tablet_bootstrap.cc:492] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:34.814005  2959 ts_tablet_manager.cc:1403] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.065s	user 0.007s	sys 0.007s
I20260709 12:21:34.814011  2957 ts_tablet_manager.cc:1434] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:34.815274  2957 tablet_bootstrap.cc:492] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:34.816568  2959 raft_consensus.cc:359] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.817224  2959 raft_consensus.cc:385] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.817503  2959 raft_consensus.cc:740] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.818233  2959 consensus_queue.cc:260] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.821239  2959 ts_tablet_manager.cc:1434] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:21:34.822358  2957 tablet_bootstrap.cc:654] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.822043  2958 consensus_queue.cc:260] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.830523  2959 tablet_bootstrap.cc:492] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:34.837954  2958 ts_tablet_manager.cc:1434] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:21:34.838997  2958 tablet_bootstrap.cc:492] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.839613  2957 tablet_bootstrap.cc:492] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:34.840085  2957 ts_tablet_manager.cc:1403] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.007s
I20260709 12:21:34.843128  2957 raft_consensus.cc:359] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.843816  2957 raft_consensus.cc:385] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.844094  2957 raft_consensus.cc:740] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.844751  2958 tablet_bootstrap.cc:654] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.844753  2957 consensus_queue.cc:260] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.847028  2957 ts_tablet_manager.cc:1434] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:34.848124  2957 tablet_bootstrap.cc:492] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:34.854501  2957 tablet_bootstrap.cc:654] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.856310  2959 tablet_bootstrap.cc:654] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.869277  2957 tablet_bootstrap.cc:492] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:34.869511  2959 tablet_bootstrap.cc:492] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:34.869588  2958 tablet_bootstrap.cc:492] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.870594  2957 ts_tablet_manager.cc:1403] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.001s
I20260709 12:21:34.870812  2959 ts_tablet_manager.cc:1403] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.007s
I20260709 12:21:34.870936  2958 ts_tablet_manager.cc:1403] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.006s
I20260709 12:21:34.872910  2958 raft_consensus.cc:359] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.873397  2958 raft_consensus.cc:385] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.873734  2958 raft_consensus.cc:740] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.873427  2957 raft_consensus.cc:359] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.873596  2959 raft_consensus.cc:359] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.874190  2957 raft_consensus.cc:385] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.874244  2959 raft_consensus.cc:385] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.874529  2969 raft_consensus.cc:493] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.874567  2957 raft_consensus.cc:740] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.874740  2959 raft_consensus.cc:740] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.875272  2969 raft_consensus.cc:515] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.875653  2957 consensus_queue.cc:260] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.875653  2959 consensus_queue.cc:260] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.874410  2958 consensus_queue.cc:260] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.877812  2957 ts_tablet_manager.cc:1434] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:34.877842  2969 leader_election.cc:290] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.879366  2957 tablet_bootstrap.cc:492] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:34.879771  2958 ts_tablet_manager.cc:1434] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:21:34.877987  2959 ts_tablet_manager.cc:1434] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:34.881012  2958 tablet_bootstrap.cc:492] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.881235  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53752656656243c3abe78c7a9cb488c7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:34.881888  2914 raft_consensus.cc:2468] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:34.884093  2807 leader_election.cc:304] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:34.884855  3071 raft_consensus.cc:2804] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:34.885193  3071 raft_consensus.cc:493] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:34.885558  3071 raft_consensus.cc:3060] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.887480  2958 tablet_bootstrap.cc:654] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.887486  2957 tablet_bootstrap.cc:654] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.888708  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53752656656243c3abe78c7a9cb488c7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:34.890189  2807 leader_election.cc:343] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.891464  2959 tablet_bootstrap.cc:492] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:34.892771  3071 raft_consensus.cc:515] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.896756  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53752656656243c3abe78c7a9cb488c7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
W20260709 12:21:34.900755  2807 leader_election.cc:343] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.915071  3071 leader_election.cc:290] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:34.917773  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53752656656243c3abe78c7a9cb488c7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:34.918166  2957 tablet_bootstrap.cc:492] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:34.918411  2914 raft_consensus.cc:3060] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.923036  2957 ts_tablet_manager.cc:1403] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.005s
I20260709 12:21:34.925561  2957 raft_consensus.cc:359] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.926210  2957 raft_consensus.cc:385] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.926472  2957 raft_consensus.cc:740] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.926762  2958 tablet_bootstrap.cc:492] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:34.927242  2958 ts_tablet_manager.cc:1403] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.016s
I20260709 12:21:34.927129  2957 consensus_queue.cc:260] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.931813  2914 raft_consensus.cc:2468] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:34.931996  2959 tablet_bootstrap.cc:654] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.933123  2957 ts_tablet_manager.cc:1434] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:21:34.934072  2957 tablet_bootstrap.cc:492] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:34.937091  2958 raft_consensus.cc:359] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.937862  2958 raft_consensus.cc:385] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.938232  2958 raft_consensus.cc:740] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.938920  2958 consensus_queue.cc:260] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.939723  2957 tablet_bootstrap.cc:654] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.941274  2958 ts_tablet_manager.cc:1434] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:21:34.942526  2958 tablet_bootstrap.cc:492] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:34.939680  2807 leader_election.cc:304] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:34.945461  2969 raft_consensus.cc:2804] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:34.946098  2969 raft_consensus.cc:697] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:34.946789  2969 consensus_queue.cc:237] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.949445  2958 tablet_bootstrap.cc:654] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.959328  2957 tablet_bootstrap.cc:492] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:34.960246  2957 ts_tablet_manager.cc:1403] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.003s
I20260709 12:21:34.963003  2957 raft_consensus.cc:359] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.963626  2957 raft_consensus.cc:385] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.963887  2957 raft_consensus.cc:740] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.964473  2957 consensus_queue.cc:260] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.967012  2957 ts_tablet_manager.cc:1434] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:21:34.967896  2957 tablet_bootstrap.cc:492] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:34.978272  3026 raft_consensus.cc:493] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.979028  3026 raft_consensus.cc:515] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.983888  2914 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.985268  3071 raft_consensus.cc:493] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.985922  3071 raft_consensus.cc:515] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:34.990213  2969 raft_consensus.cc:493] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.990835  2969 raft_consensus.cc:515] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.008456  3071 leader_election.cc:290] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.008677  2958 tablet_bootstrap.cc:492] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:35.010396  2958 ts_tablet_manager.cc:1403] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.068s	user 0.025s	sys 0.024s
I20260709 12:21:35.011667  3022 raft_consensus.cc:493] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.012329  3022 raft_consensus.cc:515] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.014197  2969 leader_election.cc:290] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.013909  2958 raft_consensus.cc:359] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.014679  2958 raft_consensus.cc:385] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.015009  2958 raft_consensus.cc:740] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.015679  2958 consensus_queue.cc:260] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.016655  2957 tablet_bootstrap.cc:654] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.016711  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.017270  2958 ts_tablet_manager.cc:1434] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:35.020587  2917 raft_consensus.cc:2468] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:35.021142  2958 tablet_bootstrap.cc:492] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:35.022706  2732 leader_election.cc:304] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:35.023780  3140 raft_consensus.cc:2804] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.024235  3140 raft_consensus.cc:493] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.024686  3140 raft_consensus.cc:3060] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.027598  2958 tablet_bootstrap.cc:654] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.029901  2957 tablet_bootstrap.cc:492] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:35.030377  2957 ts_tablet_manager.cc:1403] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.063s	user 0.008s	sys 0.002s
I20260709 12:21:35.032989  2957 raft_consensus.cc:359] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.033689  2957 raft_consensus.cc:385] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.033964  2957 raft_consensus.cc:740] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.034533  3022 leader_election.cc:290] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.034662  2957 consensus_queue.cc:260] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.035609  2958 tablet_bootstrap.cc:492] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:35.036130  2958 ts_tablet_manager.cc:1403] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.008s
I20260709 12:21:35.036190  2957 ts_tablet_manager.cc:1434] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:35.037312  2957 tablet_bootstrap.cc:492] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:35.039870  2958 raft_consensus.cc:359] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.040836  2958 raft_consensus.cc:385] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.041224  2958 raft_consensus.cc:740] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.042100  2958 consensus_queue.cc:260] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.043179  2957 tablet_bootstrap.cc:654] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.045246  2958 ts_tablet_manager.cc:1434] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 12:21:35.046618  2958 tablet_bootstrap.cc:492] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:35.049265  2969 raft_consensus.cc:493] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.049824  2969 raft_consensus.cc:515] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.051118  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.052124  2969 leader_election.cc:290] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.052771  2969 raft_consensus.cc:493] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.053340  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.053295  2969 raft_consensus.cc:515] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.055467  2969 leader_election.cc:290] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.055421  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.056550  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.062366  2958 tablet_bootstrap.cc:654] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.062561  2917 raft_consensus.cc:2468] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:35.073987  2958 tablet_bootstrap.cc:492] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:35.074640  2958 ts_tablet_manager.cc:1403] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.004s
I20260709 12:21:35.078280  2958 raft_consensus.cc:359] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.079236  2958 raft_consensus.cc:385] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.079559  2958 raft_consensus.cc:740] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.080353  2958 consensus_queue.cc:260] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.083122  2958 ts_tablet_manager.cc:1434] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:35.085073  2958 tablet_bootstrap.cc:492] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:35.086509  3230 consensus_queue.cc:1048] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.086964  3026 leader_election.cc:290] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.092835  2958 tablet_bootstrap.cc:654] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:35.098878  2807 leader_election.cc:343] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.100349  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:35.101135  2842 raft_consensus.cc:2468] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:35.104928  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:35.105624  2842 raft_consensus.cc:2468] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:35.107381  2987 raft_consensus.cc:493] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.107676  2958 tablet_bootstrap.cc:492] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:35.108093  2958 ts_tablet_manager.cc:1403] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.000s
I20260709 12:21:35.107877  2987 raft_consensus.cc:515] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.109947  2987 leader_election.cc:290] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.110757  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.111402  2917 raft_consensus.cc:2468] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:35.112228  2958 raft_consensus.cc:359] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.112883  2958 raft_consensus.cc:385] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.113148  2958 raft_consensus.cc:740] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.113660  2969 raft_consensus.cc:493] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.113796  2958 consensus_queue.cc:260] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.114118  2969 raft_consensus.cc:515] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.115612  2969 leader_election.cc:290] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.116199  2958 ts_tablet_manager.cc:1434] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:35.117172  2958 tablet_bootstrap.cc:492] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:35.045387  2664 catalog_manager.cc:5697] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:35.119283  2732 leader_election.cc:304] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:35.121945  3230 raft_consensus.cc:2804] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.122326  3230 raft_consensus.cc:493] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.122654  3230 raft_consensus.cc:3060] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.123863  3152 raft_consensus.cc:493] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.124042  3139 consensus_queue.cc:1048] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.124472  3152 raft_consensus.cc:515] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.129172  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.129842  2768 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:35.131172  3001 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.131904  3001 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.135505  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:35.136194  2842 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:35.136848  2881 leader_election.cc:304] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:35.140774  3230 raft_consensus.cc:515] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.142866  3230 leader_election.cc:290] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.139422  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:35.145434  2807 leader_election.cc:343] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.145855  2807 leader_election.cc:304] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:35.145864  3152 leader_election.cc:290] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.146584  2969 raft_consensus.cc:2749] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.146593  3152 raft_consensus.cc:2804] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.146951  2807 leader_election.cc:304] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:35.147241  3152 raft_consensus.cc:493] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.147763  3152 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.148640  2969 raft_consensus.cc:2804] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.149049  2969 raft_consensus.cc:493] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.149372  2969 raft_consensus.cc:3060] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.119092  3140 raft_consensus.cc:515] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.146869  3001 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.150890  2765 raft_consensus.cc:2393] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a10672cd53140d59ff945f75345cc06 in current term 1: Already voted for candidate 442c92e5267046cd92f46e6f469570a5 in this term.
I20260709 12:21:35.151542  3001 raft_consensus.cc:493] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.151804  3140 leader_election.cc:290] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.151988  3001 raft_consensus.cc:515] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.159173  2969 raft_consensus.cc:515] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.161278  3152 raft_consensus.cc:515] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:35.165972  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 45855us
I20260709 12:21:35.167135  2959 tablet_bootstrap.cc:492] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:35.167596  2959 ts_tablet_manager.cc:1403] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.276s	user 0.049s	sys 0.077s
I20260709 12:21:35.168031  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.168222  3152 leader_election.cc:290] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.164836  2958 tablet_bootstrap.cc:654] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.170441  2969 leader_election.cc:290] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.170117  2959 raft_consensus.cc:359] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.171484  2959 raft_consensus.cc:385] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.171742  2959 raft_consensus.cc:740] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.172721  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.174517  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80bb6a87c6e403fa76954737ffda91a" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.175611  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:35.176195  2766 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.176457  3001 leader_election.cc:290] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.176400  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.178543  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:35.179103  2917 raft_consensus.cc:3060] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.181907  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1a771f5b2742d8bd641ec0afec2cf4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.183254  3140 consensus_queue.cc:1048] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.184687  2766 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:35.186096  2957 tablet_bootstrap.cc:492] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:35.186599  2957 ts_tablet_manager.cc:1403] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.149s	user 0.030s	sys 0.028s
I20260709 12:21:35.186496  2917 raft_consensus.cc:2468] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:35.187439  2959 consensus_queue.cc:260] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.189306  2957 raft_consensus.cc:359] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.190069  2957 raft_consensus.cc:385] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.190353  2957 raft_consensus.cc:740] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.189898  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:35.191108  2843 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.191035  2957 consensus_queue.cc:260] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.194094  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb611b2490b4682a0c5a45f2bbe7c94" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.195792  3071 raft_consensus.cc:493] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.196264  2959 ts_tablet_manager.cc:1434] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 12:21:35.196240  3071 raft_consensus.cc:515] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.197244  2959 tablet_bootstrap.cc:492] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:35.198385  3071 leader_election.cc:290] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.198571  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80bb6a87c6e403fa76954737ffda91a" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
W20260709 12:21:35.199576  2807 leader_election.cc:343] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:35.200191  2807 leader_election.cc:343] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:35.200852  2807 leader_election.cc:343] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:35.201401  2807 leader_election.cc:343] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:35.201968  2881 leader_election.cc:343] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.203518  3234 consensus_queue.cc:1048] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.201921  3235 raft_consensus.cc:493] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.205119  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:35.205670  3235 raft_consensus.cc:515] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.206158  2765 raft_consensus.cc:2393] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9a10672cd53140d59ff945f75345cc06 in current term 1: Already voted for candidate 442c92e5267046cd92f46e6f469570a5 in this term.
I20260709 12:21:35.207547  2881 leader_election.cc:304] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:35.208341  3001 raft_consensus.cc:2804] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.208835  2843 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:35.209832  2957 ts_tablet_manager.cc:1434] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:21:35.210790  2957 tablet_bootstrap.cc:492] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:35.216364  2959 tablet_bootstrap.cc:654] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.219215  2957 tablet_bootstrap.cc:654] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.227589  2958 tablet_bootstrap.cc:492] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:35.228135  2958 ts_tablet_manager.cc:1403] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.111s	user 0.016s	sys 0.000s
I20260709 12:21:35.229784  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb611b2490b4682a0c5a45f2bbe7c94" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.231211  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:35.231880  2842 raft_consensus.cc:2468] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:35.232863  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1a771f5b2742d8bd641ec0afec2cf4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:35.233182  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.233487  2841 raft_consensus.cc:2468] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:35.233803  2917 raft_consensus.cc:2468] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:35.238777  2880 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:35.239759  3237 raft_consensus.cc:2804] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:35.244196  2959 tablet_bootstrap.cc:492] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:35.244853  2959 ts_tablet_manager.cc:1403] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.048s	user 0.007s	sys 0.003s
W20260709 12:21:35.247663  2807 leader_election.cc:343] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.248298  2807 leader_election.cc:304] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:35.249692  2969 raft_consensus.cc:2749] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.252336  2959 raft_consensus.cc:359] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.253176  2959 raft_consensus.cc:385] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.253443  2959 raft_consensus.cc:740] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.253368  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12034fdcbbed4121a134d5d26c1dc5ac" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.254192  2959 consensus_queue.cc:260] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.256306  2959 ts_tablet_manager.cc:1434] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:21:35.258159  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:35.258935  2841 raft_consensus.cc:3060] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.262055  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:35.263526  2842 raft_consensus.cc:2393] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 9a10672cd53140d59ff945f75345cc06 in this term.
I20260709 12:21:35.270540  2959 tablet_bootstrap.cc:492] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:35.272347  3235 leader_election.cc:290] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.240101  2880 leader_election.cc:304] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:35.240195  3237 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:21:35.290313  2958 raft_consensus.cc:359] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.291302  2958 raft_consensus.cc:385] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.291666  2958 raft_consensus.cc:740] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
W20260709 12:21:35.279569  2807 leader_election.cc:343] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.270342  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:35.295574  2958 consensus_queue.cc:260] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.297490  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:35.298134  2916 raft_consensus.cc:3060] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.302508  2969 consensus_queue.cc:1048] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.302145  2958 ts_tablet_manager.cc:1434] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.000s
I20260709 12:21:35.291961  3237 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:35.304600  2958 tablet_bootstrap.cc:492] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:35.301601  3235 raft_consensus.cc:2804] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.301036  3001 raft_consensus.cc:697] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
W20260709 12:21:35.305313  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 65294.4us
I20260709 12:21:35.310741  3235 raft_consensus.cc:493] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.341740  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12034fdcbbed4121a134d5d26c1dc5ac" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.342639  3235 raft_consensus.cc:3060] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.354126  3152 raft_consensus.cc:493] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.350263  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0df81094e2493cbe804d2dba9aa9d4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.359375  2768 raft_consensus.cc:2468] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:35.366688  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0df81094e2493cbe804d2dba9aa9d4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:35.367337  2842 raft_consensus.cc:2468] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:35.353057  3240 raft_consensus.cc:493] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.355816  2841 raft_consensus.cc:2468] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 1.
I20260709 12:21:35.364778  3152 raft_consensus.cc:515] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.374503  3152 leader_election.cc:290] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.375756  2959 tablet_bootstrap.cc:654] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:35.384487  2807 leader_election.cc:343] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.386068  2807 leader_election.cc:304] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:35.386948  3157 raft_consensus.cc:2749] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.389295  2732 leader_election.cc:304] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
W20260709 12:21:35.392949  2807 leader_election.cc:343] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.393488  2807 leader_election.cc:304] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:35.353925  3001 consensus_queue.cc:237] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.400427  2958 tablet_bootstrap.cc:654] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.405923  3234 raft_consensus.cc:2804] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.406427  3234 raft_consensus.cc:697] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:35.407239  3234 consensus_queue.cc:237] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.410378  2957 tablet_bootstrap.cc:492] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:35.410959  2957 ts_tablet_manager.cc:1403] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.200s	user 0.022s	sys 0.000s
I20260709 12:21:35.379096  3240 raft_consensus.cc:515] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.413736  3240 leader_election.cc:290] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.416095  2807 leader_election.cc:304] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:35.419230  2969 raft_consensus.cc:2804] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.419693  2969 raft_consensus.cc:493] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.420056  2969 raft_consensus.cc:3060] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:35.431520  2807 leader_election.cc:343] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.432214  2807 leader_election.cc:304] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:35.415266  3154 raft_consensus.cc:493] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.417500  3071 raft_consensus.cc:2749] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.446671  2969 raft_consensus.cc:515] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.391743  2880 leader_election.cc:304] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:35.431695  2916 raft_consensus.cc:2468] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:35.457671  2969 leader_election.cc:290] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:35.458117  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 50735.3us
W20260709 12:21:35.437744  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 58449.6us
I20260709 12:21:35.461912  2732 leader_election.cc:304] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:35.463052  3026 raft_consensus.cc:2749] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.453670  3071 raft_consensus.cc:2749] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.475143  2969 consensus_queue.cc:1048] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.460732  3001 raft_consensus.cc:2804] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.450567  3154 raft_consensus.cc:515] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:35.458007  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 75288.3us
I20260709 12:21:35.492401  2959 tablet_bootstrap.cc:492] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:35.493080  2959 ts_tablet_manager.cc:1403] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.223s	user 0.009s	sys 0.016s
W20260709 12:21:35.500258  2714 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:35.501456  3154 leader_election.cc:290] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.502264  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:35.501591  3253 raft_consensus.cc:493] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.479821  3001 raft_consensus.cc:493] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.503072  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:35.503500  3001 raft_consensus.cc:3060] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:35.505456  2807 leader_election.cc:343] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.480391  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:35.506067  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:35.515558  2807 leader_election.cc:343] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.503314  3253 raft_consensus.cc:515] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.525286  2957 raft_consensus.cc:359] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.526180  2957 raft_consensus.cc:385] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.530246  2957 raft_consensus.cc:740] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.511132  2840 raft_consensus.cc:2468] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:35.533730  2957 consensus_queue.cc:260] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.527154  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:35.544991  2880 leader_election.cc:343] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.546283  2880 leader_election.cc:304] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:35.548120  2958 tablet_bootstrap.cc:492] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:35.548619  2958 ts_tablet_manager.cc:1403] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.244s	user 0.035s	sys 0.012s
I20260709 12:21:35.544948  2959 raft_consensus.cc:359] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.550387  2959 raft_consensus.cc:385] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.550660  2959 raft_consensus.cc:740] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.554159  2957 ts_tablet_manager.cc:1434] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.143s	user 0.012s	sys 0.009s
I20260709 12:21:35.554519  2807 leader_election.cc:304] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:35.555734  3242 raft_consensus.cc:2804] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.556134  3242 raft_consensus.cc:493] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.556448  3242 raft_consensus.cc:3060] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.557221  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.561836  2914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:35.562441  2914 raft_consensus.cc:3060] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.565919  2959 consensus_queue.cc:260] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.569752  3236 raft_consensus.cc:493] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:35.570896  2807 leader_election.cc:343] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.571509  2807 leader_election.cc:304] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:35.572497  2969 consensus_queue.cc:1048] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:35.572945  2914 raft_consensus.cc:2468] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:35.566902  2663 catalog_manager.cc:5697] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:35.579907  2957 tablet_bootstrap.cc:492] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:35.578636  2807 leader_election.cc:304] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:35.583134  2969 raft_consensus.cc:2804] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.583359  3254 raft_consensus.cc:2749] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.583951  3254 raft_consensus.cc:493] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.584290  3254 raft_consensus.cc:515] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.587842  2959 ts_tablet_manager.cc:1434] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.094s	user 0.010s	sys 0.001s
I20260709 12:21:35.574869  3071 raft_consensus.cc:2804] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.570263  3236 raft_consensus.cc:515] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.592114  2959 tablet_bootstrap.cc:492] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:35.586105  3254 leader_election.cc:290] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.596361  2958 raft_consensus.cc:359] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.597337  2958 raft_consensus.cc:385] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.597730  2958 raft_consensus.cc:740] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.586366  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.599466  2958 consensus_queue.cc:260] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.606350  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fa205fa4864ef4b19bda073c06a5d4" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.612363  2969 raft_consensus.cc:697] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:35.595690  3071 raft_consensus.cc:697] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
W20260709 12:21:35.615226  2807 consensus_peers.cc:597] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:35.616138  2807 leader_election.cc:343] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.619804  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.621733  3071 consensus_queue.cc:237] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.623745  2958 ts_tablet_manager.cc:1434] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.075s	user 0.008s	sys 0.000s
I20260709 12:21:35.623515  3001 raft_consensus.cc:515] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.624687  2958 tablet_bootstrap.cc:492] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
W20260709 12:21:35.628412  2807 leader_election.cc:343] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.613381  2969 consensus_queue.cc:237] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.629628  3001 leader_election.cc:290] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.629951  3236 leader_election.cc:290] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.632086  3253 leader_election.cc:290] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:35.638842  2881 leader_election.cc:343] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.649164  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fa205fa4864ef4b19bda073c06a5d4" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.652148  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0df81094e2493cbe804d2dba9aa9d4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:35.652827  2764 raft_consensus.cc:3060] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:35.655941  2881 leader_election.cc:343] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.656464  2881 leader_election.cc:304] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:35.663123  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.640360  3237 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.664259  3248 raft_consensus.cc:2749] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.645037  3242 raft_consensus.cc:515] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.694486  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:35.695201  2840 raft_consensus.cc:2468] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
W20260709 12:21:35.695828  2807 leader_election.cc:343] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.685460  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.698102  2807 leader_election.cc:304] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:35.702190  3260 raft_consensus.cc:493] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.704623  3258 raft_consensus.cc:2749] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.712303  2959 tablet_bootstrap.cc:654] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.711373  2838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0df81094e2493cbe804d2dba9aa9d4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:35.713127  2838 raft_consensus.cc:3060] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:35.714931  2807 leader_election.cc:343] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.715395  2807 leader_election.cc:304] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:35.702713  3260 raft_consensus.cc:515] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.722295  3242 leader_election.cc:290] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.718022  3258 raft_consensus.cc:2749] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.722215  3237 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.723426  3235 raft_consensus.cc:515] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.737033  3235 leader_election.cc:290] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.727905  2880 leader_election.cc:304] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:35.739529  3237 raft_consensus.cc:2804] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.739918  3237 raft_consensus.cc:493] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.740260  3237 raft_consensus.cc:3060] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.742739  3260 leader_election.cc:290] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.743239  2764 raft_consensus.cc:2468] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:35.746419  2838 raft_consensus.cc:2468] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:35.746505  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1a771f5b2742d8bd641ec0afec2cf4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:35.747126  2837 raft_consensus.cc:3060] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.747047  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:35.747773  2880 leader_election.cc:304] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:35.747828  2840 raft_consensus.cc:3060] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.750562  2839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb611b2490b4682a0c5a45f2bbe7c94" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:35.751190  2839 raft_consensus.cc:2468] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:35.751287  3230 raft_consensus.cc:493] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.751479  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:35.752002  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
W20260709 12:21:35.749889  2881 leader_election.cc:343] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.751852  3230 raft_consensus.cc:515] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.755270  3235 raft_consensus.cc:2804] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.755991  3235 raft_consensus.cc:697] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:35.756764  3230 leader_election.cc:290] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.756989  2915 raft_consensus.cc:1240] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Rejecting Update request from peer 442c92e5267046cd92f46e6f469570a5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:35.757233  3235 consensus_queue.cc:237] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.754719  2880 leader_election.cc:304] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:35.760637  3230 consensus_queue.cc:1059] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:35.760761  2663 catalog_manager.cc:5697] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:35.763298  2663 catalog_manager.cc:5697] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:35.765429  3255 raft_consensus.cc:3055] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:35.765630  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1a771f5b2742d8bd641ec0afec2cf4" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.765877  3255 raft_consensus.cc:740] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:21:35.767350  2915 raft_consensus.cc:2393] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:35.760582  2840 raft_consensus.cc:2468] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:35.773821  2958 tablet_bootstrap.cc:654] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.769690  3236 raft_consensus.cc:2804] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.780750  2763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1a771f5b2742d8bd641ec0afec2cf4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:35.781481  2763 raft_consensus.cc:3060] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.775223  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1a771f5b2742d8bd641ec0afec2cf4" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:35.777519  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:35.780126  3236 raft_consensus.cc:493] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.802179  2881 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:35.789868  3255 consensus_queue.cc:260] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.805676  2731 leader_election.cc:304] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:35.805954  3001 raft_consensus.cc:2749] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:35.804041  2880 leader_election.cc:304] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:35.808182  3248 raft_consensus.cc:2804] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.808724  3248 raft_consensus.cc:697] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:35.805857  3255 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:35.809782  3248 consensus_queue.cc:237] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.804538  3236 raft_consensus.cc:3060] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.825938  2664 catalog_manager.cc:5697] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:35.837560  2881 leader_election.cc:343] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.830693  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb611b2490b4682a0c5a45f2bbe7c94" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.855150  2959 tablet_bootstrap.cc:492] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:35.859465  2959 ts_tablet_manager.cc:1403] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.268s	user 0.028s	sys 0.038s
W20260709 12:21:35.857394  3265 log.cc:927] Time spent T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 12:21:35.875926  3264 log.cc:927] Time spent T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:21:35.899852  2969 raft_consensus.cc:493] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.902292  2969 raft_consensus.cc:515] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.905159  2969 leader_election.cc:290] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:35.902326  2763 raft_consensus.cc:2468] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:35.907503  3234 raft_consensus.cc:2749] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.911711  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:35.912221  2959 raft_consensus.cc:359] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.857470  2837 raft_consensus.cc:2468] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:35.912985  2959 raft_consensus.cc:385] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.913491  2959 raft_consensus.cc:740] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
W20260709 12:21:35.913694  2807 leader_election.cc:343] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.917888  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:35.918964  2917 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:35.919696  2958 tablet_bootstrap.cc:492] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:35.920234  2958 ts_tablet_manager.cc:1403] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.296s	user 0.019s	sys 0.012s
I20260709 12:21:35.912936  3237 raft_consensus.cc:515] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.927983  3237 leader_election.cc:290] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:35.920926  3236 raft_consensus.cc:515] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.924827  2959 consensus_queue.cc:260] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.930361  2957 tablet_bootstrap.cc:654] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.932058  3234 consensus_queue.cc:1048] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.931416  2807 leader_election.cc:304] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:35.933916  3254 raft_consensus.cc:2804] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.934435  3254 raft_consensus.cc:493] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.934900  3254 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.866448  2762 raft_consensus.cc:2468] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:35.944833  2881 leader_election.cc:304] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:35.945895  3152 raft_consensus.cc:2804] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.946444  3152 raft_consensus.cc:697] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:35.947330  3152 consensus_queue.cc:237] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.948073  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:35.948724  2842 raft_consensus.cc:3060] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.949553  2959 ts_tablet_manager.cc:1434] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.090s	user 0.009s	sys 0.001s
I20260709 12:21:35.948576  3240 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.951057  2959 tablet_bootstrap.cc:492] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:35.961794  2958 raft_consensus.cc:359] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.966899  2958 raft_consensus.cc:385] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.967217  2958 raft_consensus.cc:740] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.967891  2958 consensus_queue.cc:260] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.983413  2958 ts_tablet_manager.cc:1434] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.000s
I20260709 12:21:35.985286  2958 tablet_bootstrap.cc:492] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:35.987566  2664 catalog_manager.cc:5697] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:35.989703  2664 catalog_manager.cc:5697] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:35.975947  2842 raft_consensus.cc:2468] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:35.999467  3026 consensus_queue.cc:1048] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.999929  3254 raft_consensus.cc:515] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:35.999333  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
W20260709 12:21:36.009028  2881 leader_election.cc:343] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.009789  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:36.011734  3254 leader_election.cc:290] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:36.010154  2880 leader_election.cc:304] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:36.012501  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:36.013113  2917 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:36.013756  2807 leader_election.cc:343] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.014740  3237 raft_consensus.cc:2804] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.015260  3237 raft_consensus.cc:697] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:36.016175  3237 consensus_queue.cc:237] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.016692  2959 tablet_bootstrap.cc:654] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.009176  2957 tablet_bootstrap.cc:492] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:36.018147  2957 ts_tablet_manager.cc:1403] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.438s	user 0.017s	sys 0.015s
I20260709 12:21:36.019985  2958 tablet_bootstrap.cc:654] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.024901  2917 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:36.027074  2969 consensus_queue.cc:1048] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.040436  2807 leader_election.cc:304] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:36.042521  2969 raft_consensus.cc:2804] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.043248  2969 raft_consensus.cc:697] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:36.044323  2969 consensus_queue.cc:237] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.063823  2957 raft_consensus.cc:359] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.064661  2957 raft_consensus.cc:385] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.064924  2957 raft_consensus.cc:740] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.067425  2957 consensus_queue.cc:260] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.070060  2957 ts_tablet_manager.cc:1434] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 12:21:36.071116  2957 tablet_bootstrap.cc:492] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:36.075397  2958 tablet_bootstrap.cc:492] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:36.075963  2958 ts_tablet_manager.cc:1403] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.091s	user 0.016s	sys 0.019s
I20260709 12:21:36.079574  3236 leader_election.cc:290] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:36.080963  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb611b2490b4682a0c5a45f2bbe7c94" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:36.081720  2766 raft_consensus.cc:3060] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.088574  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb611b2490b4682a0c5a45f2bbe7c94" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:36.089339  2837 raft_consensus.cc:3060] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.093926  3001 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.102905  2969 consensus_queue.cc:1048] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:21:36.136345  2958 raft_consensus.cc:359] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.137475  2958 raft_consensus.cc:385] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.137878  2958 raft_consensus.cc:740] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.138831  2958 consensus_queue.cc:260] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.148967  2959 tablet_bootstrap.cc:492] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:36.153255  2959 ts_tablet_manager.cc:1403] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.202s	user 0.044s	sys 0.024s
I20260709 12:21:36.165622  2837 raft_consensus.cc:2468] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:36.172924  2880 leader_election.cc:304] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:36.179307  2666 catalog_manager.cc:5697] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.201683  2957 tablet_bootstrap.cc:654] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.203483  3001 raft_consensus.cc:493] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.213191  3001 raft_consensus.cc:515] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.227052  2958 ts_tablet_manager.cc:1434] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.151s	user 0.016s	sys 0.004s
I20260709 12:21:36.228037  3237 raft_consensus.cc:2804] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.228405  2958 tablet_bootstrap.cc:492] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:36.243865  2663 catalog_manager.cc:5697] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.247488  2663 catalog_manager.cc:5697] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.254925  2959 raft_consensus.cc:359] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.255609  2959 raft_consensus.cc:385] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.255858  2959 raft_consensus.cc:740] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.243266  3237 raft_consensus.cc:697] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:36.257495  3237 consensus_queue.cc:237] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.259514  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:36.261287  3154 consensus_queue.cc:1048] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:36.262712  2881 leader_election.cc:343] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.267576  3001 leader_election.cc:290] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:36.260771  2959 consensus_queue.cc:260] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.280212  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:36.281107  2842 raft_consensus.cc:2468] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:36.240139  2766 raft_consensus.cc:2468] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:36.293531  2958 tablet_bootstrap.cc:654] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.284570  2880 leader_election.cc:304] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:36.295502  3152 raft_consensus.cc:2804] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.296105  3152 raft_consensus.cc:493] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.296554  3152 raft_consensus.cc:3060] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.325450  2959 ts_tablet_manager.cc:1434] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.172s	user 0.009s	sys 0.007s
I20260709 12:21:36.329452  2957 tablet_bootstrap.cc:492] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:36.330087  2957 ts_tablet_manager.cc:1403] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.259s	user 0.022s	sys 0.003s
I20260709 12:21:36.328920  2666 catalog_manager.cc:5697] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.341411  2957 raft_consensus.cc:359] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.345726  2957 raft_consensus.cc:385] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.345036  2959 tablet_bootstrap.cc:492] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:36.346037  3152 raft_consensus.cc:515] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:36.346601  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 61093.8us
W20260709 12:21:36.347414  2807 consensus_peers.cc:597] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:36.349116  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:36.350957  3152 leader_election.cc:290] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:36.346077  2957 raft_consensus.cc:740] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.352231  2957 consensus_queue.cc:260] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.354225  2957 ts_tablet_manager.cc:1434] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
W20260709 12:21:36.354346  2881 leader_election.cc:343] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.354839  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:36.355265  2957 tablet_bootstrap.cc:492] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:36.355484  2837 raft_consensus.cc:3060] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.363626  2837 raft_consensus.cc:2468] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:36.366415  3001 consensus_queue.cc:1048] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.366834  2880 leader_election.cc:304] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:36.373344  3001 raft_consensus.cc:2804] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.373955  3001 raft_consensus.cc:697] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:36.374747  3001 consensus_queue.cc:237] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.383669  2957 tablet_bootstrap.cc:654] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.379231  2969 raft_consensus.cc:493] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.384512  2969 raft_consensus.cc:515] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.386620  2969 leader_election.cc:290] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:36.388731  2958 tablet_bootstrap.cc:492] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:36.389410  2958 ts_tablet_manager.cc:1403] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.161s	user 0.032s	sys 0.020s
I20260709 12:21:36.390841  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4b953250214a6f8f3b4794fab90ce0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:36.391705  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4b953250214a6f8f3b4794fab90ce0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:36.394104  2807 leader_election.cc:343] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:36.397039  2807 leader_election.cc:343] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.397312  3231 raft_consensus.cc:493] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.378441  2959 tablet_bootstrap.cc:654] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.397897  3231 raft_consensus.cc:515] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.403009  3231 leader_election.cc:290] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:36.398926  3240 consensus_queue.cc:1048] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:21:36.404217  3236 consensus_queue.cc:1048] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.406427  3152 consensus_queue.cc:1048] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:36.397689  2807 leader_election.cc:304] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:36.412693  3154 raft_consensus.cc:2749] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:36.417564  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:36.420015  2842 raft_consensus.cc:2393] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:36.436120  2764 raft_consensus.cc:3060] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.440553  3240 consensus_queue.cc:1048] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.443622  2664 catalog_manager.cc:5697] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.447144  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:36.461621  2958 raft_consensus.cc:359] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.462677  2958 raft_consensus.cc:385] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.463022  2958 raft_consensus.cc:740] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.463615  3236 consensus_queue.cc:1048] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:36.463922  2958 consensus_queue.cc:260] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.481348  2732 leader_election.cc:304] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:36.501924  3026 raft_consensus.cc:2749] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:36.522747  2958 ts_tablet_manager.cc:1434] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.133s	user 0.008s	sys 0.000s
I20260709 12:21:36.524304  2958 tablet_bootstrap.cc:492] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:36.543320  3154 raft_consensus.cc:493] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.555470  3154 raft_consensus.cc:515] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.552783  2959 tablet_bootstrap.cc:492] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:36.542891  3026 raft_consensus.cc:493] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.560956  3240 consensus_queue.cc:1048] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.561270  3026 raft_consensus.cc:515] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.558198  2959 ts_tablet_manager.cc:1403] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.213s	user 0.034s	sys 0.027s
W20260709 12:21:36.567034  2807 consensus_peers.cc:597] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:36.570905  3288 log.cc:927] Time spent T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:21:36.573925  2957 tablet_bootstrap.cc:492] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:36.574429  2957 ts_tablet_manager.cc:1403] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.219s	user 0.052s	sys 0.045s
I20260709 12:21:36.580516  2957 raft_consensus.cc:359] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.581364  2957 raft_consensus.cc:385] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.581889  2957 raft_consensus.cc:740] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.582540  2957 consensus_queue.cc:260] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.586256  2957 ts_tablet_manager.cc:1434] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:21:36.587260  2957 tablet_bootstrap.cc:492] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:36.590634  2969 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.592737  3026 leader_election.cc:290] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:36.591979  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53752656656243c3abe78c7a9cb488c7" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:36.599432  3154 leader_election.cc:290] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:36.611063  2969 raft_consensus.cc:493] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:36.612892  2714 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:36.615242  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eface8ee310347d4bb08e3c5864ce5d9" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:36.611687  2969 raft_consensus.cc:515] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.607255  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53752656656243c3abe78c7a9cb488c7" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:36.621208  2732 leader_election.cc:304] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
W20260709 12:21:36.623613  2881 consensus_peers.cc:597] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:36.623476  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:36.626968  2969 leader_election.cc:290] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:36.628561  2807 leader_election.cc:343] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:36.629365  2807 leader_election.cc:343] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.630015  2959 raft_consensus.cc:359] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.630760  2959 raft_consensus.cc:385] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.631018  2959 raft_consensus.cc:740] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.631647  2959 consensus_queue.cc:260] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.631683  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:36.616192  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eface8ee310347d4bb08e3c5864ce5d9" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:36.637619  2807 leader_election.cc:343] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.638471  2807 leader_election.cc:304] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:36.640265  3154 raft_consensus.cc:2749] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:36.627914  3026 raft_consensus.cc:2749] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:36.640985  2807 leader_election.cc:343] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.641692  2807 leader_election.cc:304] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:36.642503  3154 raft_consensus.cc:2749] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:36.641809  2766 raft_consensus.cc:3060] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.642278  2959 ts_tablet_manager.cc:1434] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 12:21:36.644217  2959 tablet_bootstrap.cc:492] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:36.665236  3237 raft_consensus.cc:493] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.665946  3237 raft_consensus.cc:515] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.668586  3237 leader_election.cc:290] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:36.669956  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12034fdcbbed4121a134d5d26c1dc5ac" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:36.672205  2881 leader_election.cc:343] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.702239  2958 tablet_bootstrap.cc:654] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.696007  3240 raft_consensus.cc:493] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.703318  3240 raft_consensus.cc:515] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.704012  2959 tablet_bootstrap.cc:654] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.726347  2768 raft_consensus.cc:3060] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.707772  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12034fdcbbed4121a134d5d26c1dc5ac" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:36.741851  2841 raft_consensus.cc:2468] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:36.743371  3253 raft_consensus.cc:493] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.734663  3237 consensus_queue.cc:1048] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:21:36.758147  2958 tablet_bootstrap.cc:492] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:36.759039  2958 ts_tablet_manager.cc:1403] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.235s	user 0.012s	sys 0.005s
I20260709 12:21:36.779814  2880 leader_election.cc:304] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:36.780999  3152 raft_consensus.cc:2804] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.781404  3152 raft_consensus.cc:493] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.781780  3152 raft_consensus.cc:3060] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.794541  3240 leader_election.cc:290] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:36.751123  3253 raft_consensus.cc:515] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.798187  3253 leader_election.cc:290] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:36.802793  2763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:36.804095  2957 tablet_bootstrap.cc:654] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:36.805383  2807 leader_election.cc:343] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.809293  2763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fa205fa4864ef4b19bda073c06a5d4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:36.811722  2958 raft_consensus.cc:359] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.812506  2958 raft_consensus.cc:385] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.812785  2958 raft_consensus.cc:740] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
W20260709 12:21:36.825881  2881 leader_election.cc:343] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.854534  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fa205fa4864ef4b19bda073c06a5d4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:36.855409  2837 raft_consensus.cc:2468] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:36.879256  2958 consensus_queue.cc:260] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.885334  2959 tablet_bootstrap.cc:492] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:36.885895  2959 ts_tablet_manager.cc:1403] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.242s	user 0.034s	sys 0.037s
I20260709 12:21:36.894379  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:36.825209  3235 consensus_queue.cc:1048] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.052s
W20260709 12:21:36.916996  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 92898.3us
W20260709 12:21:36.925416  2807 leader_election.cc:343] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.925935  2807 leader_election.cc:304] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:36.884305  3152 raft_consensus.cc:515] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.928184  3258 raft_consensus.cc:2749] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:36.934196  3152 leader_election.cc:290] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:36.938983  2959 raft_consensus.cc:359] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.939821  2959 raft_consensus.cc:385] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.939733  2763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12034fdcbbed4121a134d5d26c1dc5ac" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:36.940291  2959 raft_consensus.cc:740] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.941032  2959 consensus_queue.cc:260] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:36.941903  2763 raft_consensus.cc:3060] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:36.951699  2881 consensus_peers.cc:597] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:36.954144  2958 ts_tablet_manager.cc:1434] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.195s	user 0.013s	sys 0.025s
I20260709 12:21:36.954491  2880 leader_election.cc:304] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:36.956219  2958 tablet_bootstrap.cc:492] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:36.961525  3240 raft_consensus.cc:2804] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.962002  3240 raft_consensus.cc:493] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.962332  3240 raft_consensus.cc:3060] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.970062  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12034fdcbbed4121a134d5d26c1dc5ac" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:36.971076  2840 raft_consensus.cc:3060] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.987502  2957 tablet_bootstrap.cc:492] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:36.988261  2957 ts_tablet_manager.cc:1403] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.401s	user 0.023s	sys 0.008s
I20260709 12:21:36.988457  2958 tablet_bootstrap.cc:654] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.049256  2957 raft_consensus.cc:359] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.054505  2957 raft_consensus.cc:385] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.055083  2957 raft_consensus.cc:740] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.056268  2957 consensus_queue.cc:260] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.071259  3307 consensus_queue.cc:1048] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.990859  3240 raft_consensus.cc:515] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.087693  2957 ts_tablet_manager.cc:1434] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.099s	user 0.012s	sys 0.002s
I20260709 12:21:37.091508  2957 tablet_bootstrap.cc:492] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:37.095158  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fa205fa4864ef4b19bda073c06a5d4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
W20260709 12:21:37.099045  2881 leader_election.cc:343] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.131409  2763 raft_consensus.cc:2468] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:37.142237  2959 ts_tablet_manager.cc:1434] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.256s	user 0.015s	sys 0.002s
I20260709 12:21:37.143512  2957 tablet_bootstrap.cc:654] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.151247  2959 tablet_bootstrap.cc:492] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:37.154681  2958 tablet_bootstrap.cc:492] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:37.155575  2958 ts_tablet_manager.cc:1403] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.200s	user 0.025s	sys 0.014s
I20260709 12:21:37.120955  3240 leader_election.cc:290] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:37.170578  2840 raft_consensus.cc:2468] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:37.171414  2881 leader_election.cc:304] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:37.173946  3001 raft_consensus.cc:2804] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.174577  3001 raft_consensus.cc:697] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:37.182749  3001 consensus_queue.cc:237] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.201220  2957 tablet_bootstrap.cc:492] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:37.202066  2957 ts_tablet_manager.cc:1403] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.113s	user 0.020s	sys 0.013s
I20260709 12:21:37.195454  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fa205fa4864ef4b19bda073c06a5d4" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:37.211174  2840 raft_consensus.cc:3060] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.217660  3254 raft_consensus.cc:493] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.218247  2958 raft_consensus.cc:359] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.218508  3254 raft_consensus.cc:515] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.219133  2958 raft_consensus.cc:385] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.219460  2958 raft_consensus.cc:740] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.220263  2958 consensus_queue.cc:260] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.228775  2958 ts_tablet_manager.cc:1434] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.073s	user 0.010s	sys 0.002s
I20260709 12:21:37.230149  2763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80bb6a87c6e403fa76954737ffda91a" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:37.242533  2807 leader_election.cc:343] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.248984  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80bb6a87c6e403fa76954737ffda91a" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:37.221300  3254 leader_election.cc:290] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:37.261649  2840 raft_consensus.cc:2468] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:37.259274  2915 raft_consensus.cc:2468] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:37.263159  2880 leader_election.cc:304] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:37.264832  3001 raft_consensus.cc:2804] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.266026  3001 raft_consensus.cc:697] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:37.266793  3001 consensus_queue.cc:237] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.271479  2959 tablet_bootstrap.cc:654] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.274904  2957 raft_consensus.cc:359] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.275990  2957 raft_consensus.cc:385] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.276340  2957 raft_consensus.cc:740] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.277277  2957 consensus_queue.cc:260] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.286231  2807 leader_election.cc:304] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:37.289497  3154 raft_consensus.cc:2804] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.289980  3154 raft_consensus.cc:493] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.290413  3154 raft_consensus.cc:3060] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.298657  2958 tablet_bootstrap.cc:492] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:37.310611  2958 tablet_bootstrap.cc:654] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.311106  3305 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:21:37.313285  3305 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.315882  3305 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:37.317749  2763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:37.318375  2957 ts_tablet_manager.cc:1434] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.116s	user 0.010s	sys 0.000s
I20260709 12:21:37.318493  2763 raft_consensus.cc:2468] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 2.
I20260709 12:21:37.319715  2957 tablet_bootstrap.cc:492] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:37.321373  3154 raft_consensus.cc:515] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.334409  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:37.337148  2909 raft_consensus.cc:2393] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a10672cd53140d59ff945f75345cc06 in current term 2: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:37.349838  2763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80bb6a87c6e403fa76954737ffda91a" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:37.355309  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80bb6a87c6e403fa76954737ffda91a" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:37.356104  2915 raft_consensus.cc:3060] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.354533  3154 leader_election.cc:290] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:37.369772  3253 raft_consensus.cc:493] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.370653  3253 raft_consensus.cc:515] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.372807  2915 raft_consensus.cc:2468] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:37.373692  3253 leader_election.cc:290] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:37.381465  2807 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
W20260709 12:21:37.382421  2807 leader_election.cc:343] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.383131  3253 raft_consensus.cc:2804] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:37.383602  3253 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:21:37.385506  2957 tablet_bootstrap.cc:654] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.387908  2959 tablet_bootstrap.cc:492] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:37.388376  2959 ts_tablet_manager.cc:1403] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.245s	user 0.008s	sys 0.032s
I20260709 12:21:37.386750  3253 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.392912  2807 leader_election.cc:304] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:37.396368  3154 raft_consensus.cc:2804] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.396926  3154 raft_consensus.cc:697] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:37.397625  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6291f8f287034586b4e9c0c9220bc6e7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:37.397894  3154 consensus_queue.cc:237] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.399996  3001 raft_consensus.cc:493] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.400418  3001 raft_consensus.cc:515] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:37.399813  2807 leader_election.cc:343] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.399068  2959 raft_consensus.cc:359] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.403689  2959 raft_consensus.cc:385] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.403944  2959 raft_consensus.cc:740] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.404146  2763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:37.405565  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6291f8f287034586b4e9c0c9220bc6e7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
W20260709 12:21:37.405761  2881 leader_election.cc:343] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.406350  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:37.407238  2842 raft_consensus.cc:2468] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:37.408654  2880 leader_election.cc:304] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:37.409556  3236 raft_consensus.cc:2804] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.409994  3236 raft_consensus.cc:493] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.409621  2959 consensus_queue.cc:260] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:37.410161  2807 leader_election.cc:343] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.410386  3236 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.410650  3001 leader_election.cc:290] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:37.410641  2807 leader_election.cc:304] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:37.419672  2959 ts_tablet_manager.cc:1434] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 12:21:37.420733  2969 raft_consensus.cc:2749] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.421494  2959 tablet_bootstrap.cc:492] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:37.423419  2664 catalog_manager.cc:5697] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.430578  2663 catalog_manager.cc:5697] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.428181  3236 raft_consensus.cc:515] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.441322  2958 tablet_bootstrap.cc:492] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:37.438974  2664 catalog_manager.cc:5697] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.441928  2958 ts_tablet_manager.cc:1403] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.144s	user 0.017s	sys 0.024s
I20260709 12:21:37.448171  3253 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.449682  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:37.451124  3236 leader_election.cc:290] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:37.451354  3253 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:37.451458  2881 leader_election.cc:343] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.452271  2958 raft_consensus.cc:359] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.453044  2958 raft_consensus.cc:385] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.453166  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:37.452669  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:37.454028  2842 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.455452  2957 tablet_bootstrap.cc:492] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:37.455977  2957 ts_tablet_manager.cc:1403] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.137s	user 0.013s	sys 0.000s
I20260709 12:21:37.453364  2958 raft_consensus.cc:740] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.453224  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:37.458457  2958 consensus_queue.cc:260] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.459004  2915 raft_consensus.cc:2393] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9a10672cd53140d59ff945f75345cc06 in current term 2: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:37.461179  2958 ts_tablet_manager.cc:1434] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:21:37.465260  3026 raft_consensus.cc:493] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.465898  3026 raft_consensus.cc:515] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.468461  3026 leader_election.cc:290] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:37.465867  2957 raft_consensus.cc:359] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.471021  2957 raft_consensus.cc:385] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.471411  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:37.472113  2957 raft_consensus.cc:740] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.471241  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:37.476667  2732 leader_election.cc:304] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:37.477474  2766 raft_consensus.cc:2468] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 2.
I20260709 12:21:37.477969  3026 raft_consensus.cc:2749] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.479442  2842 raft_consensus.cc:2468] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:37.480314  2957 consensus_queue.cc:260] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.480816  3253 raft_consensus.cc:493] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.481287  3253 raft_consensus.cc:515] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.483539  3253 leader_election.cc:290] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:37.485699  2959 tablet_bootstrap.cc:654] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.485965  2807 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:37.486406  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:37.487769  3258 raft_consensus.cc:2804] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:37.488286  3258 raft_consensus.cc:697] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 2 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:37.488727  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:37.489269  3258 consensus_queue.cc:237] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:37.491187  2807 leader_election.cc:343] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:37.492151  2807 leader_election.cc:343] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.492630  2807 leader_election.cc:304] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:37.494174  2880 leader_election.cc:304] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:37.481719  2958 tablet_bootstrap.cc:492] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:37.496080  2957 ts_tablet_manager.cc:1434] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 12:21:37.497697  2957 tablet_bootstrap.cc:492] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:37.498593  3001 raft_consensus.cc:2804] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.499166  3001 raft_consensus.cc:697] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:37.499990  3001 consensus_queue.cc:237] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.493485  3154 raft_consensus.cc:2749] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.505750  2958 tablet_bootstrap.cc:654] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.505772  2957 tablet_bootstrap.cc:654] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.514501  3307 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.515240  3307 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.521507  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:37.522423  2766 raft_consensus.cc:2468] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:21:37.525878  2881 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:37.537793  3240 raft_consensus.cc:2804] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:37.538295  3240 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.538676  3240 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:37.549789  3307 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:37.548053  2664 catalog_manager.cc:5697] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 1 to 2, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.556138  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:37.564414  3253 raft_consensus.cc:493] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.565850  2663 catalog_manager.cc:5697] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.566079  3253 raft_consensus.cc:515] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.573714  2959 tablet_bootstrap.cc:492] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:37.574298  2959 ts_tablet_manager.cc:1403] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.153s	user 0.029s	sys 0.036s
I20260709 12:21:37.573721  2958 tablet_bootstrap.cc:492] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:37.575434  2958 ts_tablet_manager.cc:1403] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.094s	user 0.012s	sys 0.003s
I20260709 12:21:37.581451  2763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:37.583400  2807 leader_election.cc:343] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.586330  2958 raft_consensus.cc:359] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.587139  2958 raft_consensus.cc:385] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.587430  2958 raft_consensus.cc:740] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.588116  2958 consensus_queue.cc:260] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.588488  3253 leader_election.cc:290] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:37.590507  2958 ts_tablet_manager.cc:1434] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:21:37.590772  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:37.590901  3240 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.591883  2959 raft_consensus.cc:359] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.593155  3240 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:37.592599  2959 raft_consensus.cc:385] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.594122  2959 raft_consensus.cc:740] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.594980  2958 tablet_bootstrap.cc:492] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06: Bootstrap starting.
I20260709 12:21:37.595227  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
W20260709 12:21:37.595741  2807 leader_election.cc:343] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.596159  2807 leader_election.cc:304] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:37.596674  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:37.596969  3253 raft_consensus.cc:2749] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.597681  2766 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:37.599705  2959 consensus_queue.cc:260] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.604359  2957 tablet_bootstrap.cc:492] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:37.604943  2957 ts_tablet_manager.cc:1403] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.108s	user 0.019s	sys 0.008s
I20260709 12:21:37.604408  2958 tablet_bootstrap.cc:654] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.613857  2957 raft_consensus.cc:359] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.614717  2957 raft_consensus.cc:385] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.615082  2957 raft_consensus.cc:740] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.617444  2766 raft_consensus.cc:2468] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
I20260709 12:21:37.618952  2881 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:37.623543  2763 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.626327  2959 ts_tablet_manager.cc:1434] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.001s
I20260709 12:21:37.628371  3001 consensus_queue.cc:1048] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:37.629885  2957 consensus_queue.cc:260] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.633354  3253 raft_consensus.cc:493] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.639554  3253 raft_consensus.cc:515] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.630199  3236 raft_consensus.cc:2804] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:37.646471  3236 raft_consensus.cc:697] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:37.664932  2957 ts_tablet_manager.cc:1434] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.002s
I20260709 12:21:37.668694  3253 leader_election.cc:290] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:37.669656  3236 consensus_queue.cc:237] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.671316  3307 consensus_queue.cc:1048] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.678637  3307 raft_consensus.cc:493] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.679831  3307 raft_consensus.cc:515] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.682184  3307 leader_election.cc:290] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:37.682276  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:37.683254  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eface8ee310347d4bb08e3c5864ce5d9" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:37.683987  2840 raft_consensus.cc:2468] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:37.683941  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:37.685454  2880 leader_election.cc:304] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:37.686318  3001 raft_consensus.cc:2804] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.686692  3001 raft_consensus.cc:493] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.687021  3001 raft_consensus.cc:3060] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.687032  2957 tablet_bootstrap.cc:492] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
W20260709 12:21:37.688787  2807 leader_election.cc:343] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.677695  2959 tablet_bootstrap.cc:492] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:37.689818  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eface8ee310347d4bb08e3c5864ce5d9" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:37.691428  2881 leader_election.cc:343] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.721784  3001 raft_consensus.cc:515] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:37.727298  2807 leader_election.cc:343] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.727890  2807 leader_election.cc:304] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:37.728834  2969 raft_consensus.cc:2749] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.729803  3001 leader_election.cc:290] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:37.781276  2958 tablet_bootstrap.cc:492] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06: No bootstrap required, opened a new log
I20260709 12:21:37.781955  2958 ts_tablet_manager.cc:1403] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06: Time spent bootstrapping tablet: real 0.191s	user 0.020s	sys 0.022s
I20260709 12:21:37.787776  2958 raft_consensus.cc:359] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.788551  2958 raft_consensus.cc:385] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.788841  2958 raft_consensus.cc:740] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.788901  3252 raft_consensus.cc:493] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.789412  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eface8ee310347d4bb08e3c5864ce5d9" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:37.789511  2958 consensus_queue.cc:260] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.789608  3252 raft_consensus.cc:515] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.790074  2840 raft_consensus.cc:3060] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.792188  3252 leader_election.cc:290] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:37.793417  2958 ts_tablet_manager.cc:1434] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.005s
I20260709 12:21:37.799595  2840 raft_consensus.cc:2468] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:37.800932  2664 catalog_manager.cc:5697] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 2 to 3, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.806154  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eface8ee310347d4bb08e3c5864ce5d9" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
W20260709 12:21:37.812140  2881 leader_election.cc:343] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:37.817818  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 90545.2us
I20260709 12:21:37.825914  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
W20260709 12:21:37.834071  2807 leader_election.cc:343] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.841957  2957 tablet_bootstrap.cc:654] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.840166  2880 leader_election.cc:304] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:37.865811  2959 tablet_bootstrap.cc:654] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.874017  3152 raft_consensus.cc:2804] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.878229  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:37.879895  3152 raft_consensus.cc:697] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:37.881322  3152 consensus_queue.cc:237] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.915959  2917 raft_consensus.cc:1240] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Rejecting Update request from peer 9a10672cd53140d59ff945f75345cc06 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:37.937341  3255 raft_consensus.cc:493] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.938129  3255 raft_consensus.cc:515] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.923369  3304 consensus_queue.cc:1048] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:21:37.940304  3255 leader_election.cc:290] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:21:37.946574  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 86933.6us
I20260709 12:21:37.949347  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80bb6a87c6e403fa76954737ffda91a" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:37.950256  2909 raft_consensus.cc:2393] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 9a10672cd53140d59ff945f75345cc06 in this term.
W20260709 12:21:37.950495  2807 leader_election.cc:343] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.950294  2957 tablet_bootstrap.cc:492] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:37.951344  2807 leader_election.cc:304] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:37.951927  2957 ts_tablet_manager.cc:1403] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.265s	user 0.024s	sys 0.009s
W20260709 12:21:37.952435  2881 consensus_peers.cc:597] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:37.958475  3001 consensus_queue.cc:1048] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.959230  3254 raft_consensus.cc:2749] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:37.972199  3339 log.cc:927] Time spent T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:21:37.953637  3304 consensus_queue.cc:1059] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.015s
W20260709 12:21:37.975479  2714 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:37.978451  3302 raft_consensus.cc:493] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.979017  3302 raft_consensus.cc:515] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:37.968010  2762 raft_consensus.cc:3060] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.011674  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:38.041366  3302 leader_election.cc:290] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:38.051947  3304 consensus_queue.cc:1243] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Peer 7cc87046f1414b27928722690a5e388f log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260709 12:21:38.054414  2969 raft_consensus.cc:3055] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:38.054904  2969 raft_consensus.cc:740] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:38.055912  2969 consensus_queue.cc:260] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.059226  2666 catalog_manager.cc:5697] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:38.070699  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 55080.8us
W20260709 12:21:38.071708  2881 leader_election.cc:343] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:38.072642  2881 consensus_peers.cc:597] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:38.076753  2957 raft_consensus.cc:359] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.077427  3307 consensus_queue.cc:1048] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.077915  2957 raft_consensus.cc:385] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.078258  2957 raft_consensus.cc:740] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.078950  2957 consensus_queue.cc:260] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.081440  2957 ts_tablet_manager.cc:1434] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.129s	user 0.009s	sys 0.000s
I20260709 12:21:38.082938  2957 tablet_bootstrap.cc:492] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
W20260709 12:21:38.124753  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 179669us
I20260709 12:21:38.166536  2969 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:38.159643  3304 raft_consensus.cc:493] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:38.171092  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 97785.2us
W20260709 12:21:38.178495  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 162381us
I20260709 12:21:38.204115  2957 tablet_bootstrap.cc:654] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:38.233276  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 72419.7us
I20260709 12:21:38.243626  2766 raft_consensus.cc:1240] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Rejecting Update request from peer 9a10672cd53140d59ff945f75345cc06 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:38.178571  3304 raft_consensus.cc:515] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.246917  3304 leader_election.cc:290] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:38.248035  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:38.248765  2842 raft_consensus.cc:2468] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:38.256592  2959 tablet_bootstrap.cc:492] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:38.257210  2959 ts_tablet_manager.cc:1403] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.607s	user 0.037s	sys 0.053s
I20260709 12:21:38.264465  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:38.275974  2766 raft_consensus.cc:1275] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:38.279042  2880 leader_election.cc:304] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:38.282066  3343 consensus_queue.cc:1048] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:38.283030  3355 raft_consensus.cc:2804] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:21:38.292870  2807 leader_election.cc:343] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.287375  2959 raft_consensus.cc:359] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.293517  2959 raft_consensus.cc:385] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.293864  2959 raft_consensus.cc:740] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.295743  2959 consensus_queue.cc:260] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.294034  3258 consensus_queue.cc:1048] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:21:38.283494  3355 raft_consensus.cc:493] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:38.301646  3355 raft_consensus.cc:3060] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.318396  2959 ts_tablet_manager.cc:1434] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.061s	user 0.009s	sys 0.000s
I20260709 12:21:38.336948  2957 tablet_bootstrap.cc:492] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:38.337663  2957 ts_tablet_manager.cc:1403] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.255s	user 0.014s	sys 0.022s
I20260709 12:21:38.342157  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:38.342237  2957 raft_consensus.cc:359] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.343569  2957 raft_consensus.cc:385] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.343931  2957 raft_consensus.cc:740] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.344180  3355 raft_consensus.cc:515] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.345577  2909 raft_consensus.cc:2393] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a10672cd53140d59ff945f75345cc06 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:38.346324  2957 consensus_queue.cc:260] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.347544  3355 leader_election.cc:290] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:38.348212  2957 ts_tablet_manager.cc:1434] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:21:38.357383  2957 tablet_bootstrap.cc:492] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:38.364977  2957 tablet_bootstrap.cc:654] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:38.366022  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 190517us
I20260709 12:21:38.372416  2959 tablet_bootstrap.cc:492] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:38.381078  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80bb6a87c6e403fa76954737ffda91a" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:38.386955  2768 raft_consensus.cc:3060] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.389472  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:38.395560  2807 leader_election.cc:304] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
W20260709 12:21:38.397995  2881 leader_election.cc:343] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.402292  3355 raft_consensus.cc:493] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.404364  2731 leader_election.cc:304] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:38.401976  3258 raft_consensus.cc:3060] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.408326  3355 raft_consensus.cc:515] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.430200  3355 leader_election.cc:290] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:38.431808  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:38.433316  2881 leader_election.cc:343] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.422335  3334 raft_consensus.cc:2749] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.494320  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
W20260709 12:21:38.520452  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 99660.1us
W20260709 12:21:38.551961  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 118607us
I20260709 12:21:38.657997  2957 tablet_bootstrap.cc:492] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:38.658800  2957 ts_tablet_manager.cc:1403] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.302s	user 0.016s	sys 0.015s
I20260709 12:21:38.662882  2959 tablet_bootstrap.cc:654] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.700543  3345 raft_consensus.cc:493] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.701408  3345 raft_consensus.cc:515] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.716846  3345 leader_election.cc:290] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:38.717334  2969 raft_consensus.cc:493] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.717907  2969 raft_consensus.cc:515] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.719743  2969 leader_election.cc:290] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:38.727416  2957 raft_consensus.cc:359] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.729467  2957 raft_consensus.cc:385] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.729950  2957 raft_consensus.cc:740] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.732679  2957 consensus_queue.cc:260] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:38.733950  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 160477us
I20260709 12:21:38.669529  3258 raft_consensus.cc:2749] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.736819  2957 ts_tablet_manager.cc:1434] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.078s	user 0.022s	sys 0.000s
I20260709 12:21:38.737923  2957 tablet_bootstrap.cc:492] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:38.761027  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:38.762384  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4b953250214a6f8f3b4794fab90ce0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:38.773589  2843 raft_consensus.cc:2468] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
W20260709 12:21:38.782572  2881 consensus_peers.cc:597] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:38.786317  2807 leader_election.cc:343] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:38.787701  2807 leader_election.cc:343] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.795605  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:38.796732  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4b953250214a6f8f3b4794fab90ce0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:38.798002  3001 raft_consensus.cc:493] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.802201  2911 raft_consensus.cc:2468] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 0.
I20260709 12:21:38.802178  3001 raft_consensus.cc:515] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.804296  3001 leader_election.cc:290] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:38.807854  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:38.810190  2881 leader_election.cc:343] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:38.815486  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 56168.4us
I20260709 12:21:38.818200  2959 tablet_bootstrap.cc:492] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:38.819021  2959 ts_tablet_manager.cc:1403] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.447s	user 0.029s	sys 0.018s
I20260709 12:21:38.822023  2959 raft_consensus.cc:359] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.822856  2959 raft_consensus.cc:385] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.823462  2959 raft_consensus.cc:740] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.828881  2957 tablet_bootstrap.cc:654] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:38.827924  2807 leader_election.cc:343] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.829998  2807 leader_election.cc:304] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:38.831069  2807 leader_election.cc:304] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:38.834856  2959 consensus_queue.cc:260] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.837754  3353 raft_consensus.cc:2749] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.841696  3362 consensus_queue.cc:1048] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:38.845187  3361 raft_consensus.cc:493] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.845692  3361 raft_consensus.cc:515] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.851393  2959 ts_tablet_manager.cc:1434] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 12:21:38.852645  2959 tablet_bootstrap.cc:492] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:38.855127  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:38.855827  3361 leader_election.cc:290] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:38.866016  3353 raft_consensus.cc:2804] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:38.843206  2880 leader_election.cc:304] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:38.866498  3353 raft_consensus.cc:493] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:38.866918  3353 raft_consensus.cc:3060] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.866941  3362 raft_consensus.cc:2804] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:38.867547  3362 raft_consensus.cc:697] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:38.868508  3362 consensus_queue.cc:237] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.870184  2840 raft_consensus.cc:1275] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:21:38.870572  2840 pending_rounds.cc:85] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06: Aborting all ops after (but not including) 1
I20260709 12:21:38.872248  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:38.870867  2840 pending_rounds.cc:107] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:21:38.872893  2843 raft_consensus.cc:2468] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:38.872974  2840 raft_consensus.cc:2889] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:38.883919  3353 raft_consensus.cc:515] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.884362  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:38.896453  3353 leader_election.cc:290] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:38.898679  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:38.899471  2837 raft_consensus.cc:2468] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:38.905512  2959 tablet_bootstrap.cc:654] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.908501  2957 tablet_bootstrap.cc:492] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:38.909083  2957 ts_tablet_manager.cc:1403] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.171s	user 0.017s	sys 0.017s
I20260709 12:21:38.912249  2957 raft_consensus.cc:359] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.913038  2957 raft_consensus.cc:385] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.912837  2731 leader_election.cc:304] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:38.913367  2957 raft_consensus.cc:740] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.914268  3361 raft_consensus.cc:2749] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.917322  2957 consensus_queue.cc:260] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.919911  2957 ts_tablet_manager.cc:1434] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:21:38.920881  2880 leader_election.cc:304] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:38.922101  2957 tablet_bootstrap.cc:492] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:38.922122  3001 raft_consensus.cc:2804] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:38.922662  3001 raft_consensus.cc:493] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:38.922991  3001 raft_consensus.cc:3060] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.929035  2957 tablet_bootstrap.cc:654] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.930915  3001 raft_consensus.cc:515] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.933990  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
W20260709 12:21:38.935451  2881 leader_election.cc:343] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.935439  2664 catalog_manager.cc:5697] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:38.937728  3355 consensus_queue.cc:1048] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:38.941202  3001 leader_election.cc:290] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:38.942113  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4b953250214a6f8f3b4794fab90ce0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:38.944847  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4b953250214a6f8f3b4794fab90ce0" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:38.945466  2911 raft_consensus.cc:3060] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.949854  2880 leader_election.cc:304] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:38.955626  3355 raft_consensus.cc:2804] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:38.956076  3355 raft_consensus.cc:493] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:38.956427  3355 raft_consensus.cc:3060] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.964180  3355 raft_consensus.cc:515] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.966923  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
W20260709 12:21:38.968055  2881 leader_election.cc:343] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.971642  2959 tablet_bootstrap.cc:492] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:38.972169  2959 ts_tablet_manager.cc:1403] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.120s	user 0.010s	sys 0.002s
I20260709 12:21:38.975129  2959 raft_consensus.cc:359] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.975821  2959 raft_consensus.cc:385] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.976058  2959 raft_consensus.cc:740] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.993330  2959 consensus_queue.cc:260] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:38.999194  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:38.999900  2840 raft_consensus.cc:3060] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.007318  2911 raft_consensus.cc:2468] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 1.
I20260709 12:21:39.007493  2840 raft_consensus.cc:2468] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:39.016460  3361 raft_consensus.cc:493] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.017099  3361 raft_consensus.cc:515] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.020619  2959 ts_tablet_manager.cc:1434] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.002s
I20260709 12:21:39.021819  2959 tablet_bootstrap.cc:492] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:39.022552  3355 leader_election.cc:290] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:39.032554  3361 leader_election.cc:290] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:39.034677  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:39.035670  2911 raft_consensus.cc:2393] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:39.038014  2880 leader_election.cc:304] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:39.039619  3362 raft_consensus.cc:2804] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.040115  3362 raft_consensus.cc:697] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:39.041074  3362 consensus_queue.cc:237] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.043787  2839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:39.044572  2839 raft_consensus.cc:2468] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:39.048278  2957 tablet_bootstrap.cc:492] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:39.048941  2957 ts_tablet_manager.cc:1403] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.127s	user 0.023s	sys 0.042s
I20260709 12:21:39.031355  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:39.049947  2840 raft_consensus.cc:3060] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.058964  2957 raft_consensus.cc:359] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.059942  2957 raft_consensus.cc:385] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.060292  2957 raft_consensus.cc:740] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.061102  2957 consensus_queue.cc:260] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.061493  2807 leader_election.cc:304] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:39.063547  2957 ts_tablet_manager.cc:1434] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 12:21:39.063644  2959 tablet_bootstrap.cc:654] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.064694  2957 tablet_bootstrap.cc:492] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:39.069082  3353 raft_consensus.cc:2804] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.069710  3353 raft_consensus.cc:697] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:21:39.070590  3353 consensus_queue.cc:237] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.077018  2957 tablet_bootstrap.cc:654] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.083638  2959 tablet_bootstrap.cc:492] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:39.084153  2959 ts_tablet_manager.cc:1403] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.004s
I20260709 12:21:39.087100  2959 raft_consensus.cc:359] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.087792  2959 raft_consensus.cc:385] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.088117  2959 raft_consensus.cc:740] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.089023  2959 consensus_queue.cc:260] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.091686  2959 ts_tablet_manager.cc:1434] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:39.093580  2731 leader_election.cc:304] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 7cc87046f1414b27928722690a5e388f
I20260709 12:21:39.095978  3309 raft_consensus.cc:2804] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:39.096374  3309 raft_consensus.cc:493] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.096801  3309 raft_consensus.cc:3060] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:39.098059  2807 leader_election.cc:343] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.102535  2840 raft_consensus.cc:2468] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:39.110040  2880 leader_election.cc:304] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:39.111361  3362 raft_consensus.cc:2804] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.111918  3362 raft_consensus.cc:697] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:39.112828  3362 consensus_queue.cc:237] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.121451  2957 tablet_bootstrap.cc:492] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:39.122159  2957 ts_tablet_manager.cc:1403] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.058s	user 0.024s	sys 0.024s
I20260709 12:21:39.124852  2959 tablet_bootstrap.cc:492] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:39.125313  2957 raft_consensus.cc:359] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.126133  2957 raft_consensus.cc:385] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.126492  2957 raft_consensus.cc:740] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.127455  2957 consensus_queue.cc:260] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.130113  2957 ts_tablet_manager.cc:1434] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:21:39.131558  3353 raft_consensus.cc:493] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.135403  3353 raft_consensus.cc:515] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.124878  2663 catalog_manager.cc:5697] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:39.137009  2957 tablet_bootstrap.cc:492] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f: Bootstrap starting.
I20260709 12:21:39.137574  3353 leader_election.cc:290] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:39.139940  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6291f8f287034586b4e9c0c9220bc6e7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:39.142504  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6291f8f287034586b4e9c0c9220bc6e7" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:39.148322  2957 tablet_bootstrap.cc:654] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.126977  3309 raft_consensus.cc:515] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.152535  2959 tablet_bootstrap.cc:654] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:39.154055  2807 leader_election.cc:343] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.153360  2666 catalog_manager.cc:5697] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:39.157356  3309 leader_election.cc:290] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:39.157624  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
W20260709 12:21:39.165235  2807 leader_election.cc:343] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.165814  2807 leader_election.cc:304] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:21:39.166605  3353 raft_consensus.cc:2749] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.172400  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:39.173453  2837 raft_consensus.cc:2393] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:39.179711  2731 leader_election.cc:304] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:39.181162  3309 raft_consensus.cc:2749] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.190433  2664 catalog_manager.cc:5697] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:39.240635  2714 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:39.244617  2959 tablet_bootstrap.cc:492] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:39.245322  2959 ts_tablet_manager.cc:1403] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.121s	user 0.012s	sys 0.003s
I20260709 12:21:39.248818  2959 raft_consensus.cc:359] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.250586  2959 raft_consensus.cc:385] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.250993  2959 raft_consensus.cc:740] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.253038  2957 tablet_bootstrap.cc:492] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f: No bootstrap required, opened a new log
I20260709 12:21:39.255774  2957 ts_tablet_manager.cc:1403] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f: Time spent bootstrapping tablet: real 0.119s	user 0.045s	sys 0.035s
W20260709 12:21:39.254361  2881 consensus_peers.cc:597] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:39.258005  3309 raft_consensus.cc:493] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.258545  3309 raft_consensus.cc:515] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.259286  2957 raft_consensus.cc:359] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.260066  2957 raft_consensus.cc:385] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.260449  2957 raft_consensus.cc:740] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.261296  2957 consensus_queue.cc:260] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.264070  2957 ts_tablet_manager.cc:1434] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:39.266459  2959 consensus_queue.cc:260] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.268579  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:39.269479  2837 raft_consensus.cc:2393] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:39.270764  2959 ts_tablet_manager.cc:1434] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.002s
I20260709 12:21:39.272080  2959 tablet_bootstrap.cc:492] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:39.280651  2959 tablet_bootstrap.cc:654] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.282210  2764 raft_consensus.cc:3060] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.287683  3309 leader_election.cc:290] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:39.289002  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:39.293634  2732 leader_election.cc:304] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:39.294826  3361 raft_consensus.cc:3060] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.316826  2959 tablet_bootstrap.cc:492] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:39.317464  2959 ts_tablet_manager.cc:1403] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.007s
I20260709 12:21:39.317979  3361 raft_consensus.cc:2749] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.324760  2959 raft_consensus.cc:359] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.325649  2959 raft_consensus.cc:385] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.326849  2959 raft_consensus.cc:740] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.327721  2959 consensus_queue.cc:260] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.329941  2959 ts_tablet_manager.cc:1434] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:21:39.331120  2959 tablet_bootstrap.cc:492] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:39.345610  2959 tablet_bootstrap.cc:654] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.354090  2959 tablet_bootstrap.cc:492] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:39.354619  2959 ts_tablet_manager.cc:1403] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.000s
I20260709 12:21:39.358978  2959 raft_consensus.cc:359] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.360077  2959 raft_consensus.cc:385] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.360337  2959 raft_consensus.cc:740] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.360944  2959 consensus_queue.cc:260] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.365371  2959 ts_tablet_manager.cc:1434] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:21:39.366565  2959 tablet_bootstrap.cc:492] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:39.375483  3355 raft_consensus.cc:493] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.376000  3355 raft_consensus.cc:515] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.379627  3355 leader_election.cc:290] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:39.390460  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:39.391278  2837 raft_consensus.cc:2468] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:39.394322  2959 tablet_bootstrap.cc:654] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.394562  2880 leader_election.cc:304] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:39.395494  3362 raft_consensus.cc:2804] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:39.399089  3362 raft_consensus.cc:493] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.400882  3362 raft_consensus.cc:3060] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.401482  3361 raft_consensus.cc:493] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.402093  3361 raft_consensus.cc:515] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.407050  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:39.407768  2840 raft_consensus.cc:2468] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 0.
I20260709 12:21:39.409087  2731 leader_election.cc:304] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:21:39.410041  3340 raft_consensus.cc:2804] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:39.410439  3340 raft_consensus.cc:493] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.410899  3340 raft_consensus.cc:3060] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.411298  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:39.410241  3361 leader_election.cc:290] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:39.417912  3370 raft_consensus.cc:493] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.422464  3361 raft_consensus.cc:493] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.422845  2959 tablet_bootstrap.cc:492] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:39.423053  3361 raft_consensus.cc:515] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.426241  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4b953250214a6f8f3b4794fab90ce0" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:39.427086  2911 raft_consensus.cc:2393] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 9a10672cd53140d59ff945f75345cc06 in this term.
I20260709 12:21:39.426240  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4b953250214a6f8f3b4794fab90ce0" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:39.428709  3361 leader_election.cc:290] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:39.429623  2959 ts_tablet_manager.cc:1403] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.059s	user 0.009s	sys 0.011s
I20260709 12:21:39.429767  3370 raft_consensus.cc:515] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.430888  2731 leader_election.cc:304] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:39.417810  3362 raft_consensus.cc:515] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.433421  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
W20260709 12:21:39.435040  2881 leader_election.cc:343] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.437356  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:39.434994  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:39.436355  3309 raft_consensus.cc:3060] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.442965  2916 raft_consensus.cc:2393] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:39.444624  3362 leader_election.cc:290] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:39.442293  3370 leader_election.cc:290] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:39.447669  3355 raft_consensus.cc:493] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.447068  2959 raft_consensus.cc:359] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.449059  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:39.453464  2959 raft_consensus.cc:385] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.449918  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:39.451078  3309 raft_consensus.cc:2749] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.451951  3236 raft_consensus.cc:493] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.453517  3355 raft_consensus.cc:515] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.453869  2843 raft_consensus.cc:3060] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.452142  3340 raft_consensus.cc:515] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.456324  2764 raft_consensus.cc:2393] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cc87046f1414b27928722690a5e388f in current term 1: Already voted for candidate 442c92e5267046cd92f46e6f469570a5 in this term.
I20260709 12:21:39.456564  3355 leader_election.cc:290] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:39.454157  2837 raft_consensus.cc:2468] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:39.453823  2959 raft_consensus.cc:740] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.459425  2959 consensus_queue.cc:260] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.454208  3236 raft_consensus.cc:515] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.460323  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:21:39.461138  2911 raft_consensus.cc:2393] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:39.462118  3236 leader_election.cc:290] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:39.462785  2843 raft_consensus.cc:2468] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:39.463974  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6291f8f287034586b4e9c0c9220bc6e7" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:39.464391  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:39.464632  2837 raft_consensus.cc:2468] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:39.465049  2843 raft_consensus.cc:2393] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:39.458674  3340 leader_election.cc:290] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:39.466140  2880 leader_election.cc:304] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:39.466317  2763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:39.466544  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:39.466970  2763 raft_consensus.cc:2468] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:39.467119  2843 raft_consensus.cc:2468] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 0.
I20260709 12:21:39.467348  2880 leader_election.cc:304] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:39.468133  3236 raft_consensus.cc:2804] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:39.468195  2959 ts_tablet_manager.cc:1434] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 12:21:39.468493  3236 raft_consensus.cc:493] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.468853  3236 raft_consensus.cc:3060] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.469352  2959 tablet_bootstrap.cc:492] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:39.469937  2731 leader_election.cc:304] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:39.472028  3355 raft_consensus.cc:2804] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.472509  3355 raft_consensus.cc:697] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:39.472666  2881 leader_election.cc:304] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:39.473872  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6291f8f287034586b4e9c0c9220bc6e7" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:21:39.474617  3362 raft_consensus.cc:2804] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:39.474866  3340 raft_consensus.cc:2749] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.475026  3362 raft_consensus.cc:493] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.475450  3362 raft_consensus.cc:3060] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.473521  3355 consensus_queue.cc:237] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:21:39.481336  2881 leader_election.cc:343] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.484628  2880 leader_election.cc:304] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:39.488044  3355 raft_consensus.cc:2804] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:39.488514  3355 raft_consensus.cc:493] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.488879  3355 raft_consensus.cc:3060] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.500545  3236 raft_consensus.cc:515] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.500855  3354 consensus_queue.cc:1048] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:39.502456  3001 consensus_queue.cc:1048] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:39.507783  2959 tablet_bootstrap.cc:654] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.508596  3236 leader_election.cc:290] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:39.508951  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
W20260709 12:21:39.510336  2881 leader_election.cc:343] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.510953  3355 raft_consensus.cc:515] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.513960  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6291f8f287034586b4e9c0c9220bc6e7" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
W20260709 12:21:39.515491  2881 leader_election.cc:343] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.510960  2664 catalog_manager.cc:5697] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:39.518208  3355 leader_election.cc:290] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:39.518750  3362 raft_consensus.cc:515] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.521715  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:21:39.522244  2760 raft_consensus.cc:3060] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.528108  3362 leader_election.cc:290] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:39.530174  2760 raft_consensus.cc:2468] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:39.531582  2881 leader_election.cc:304] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:21:39.532356  3362 raft_consensus.cc:2804] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.533604  2959 tablet_bootstrap.cc:492] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:39.534097  2959 ts_tablet_manager.cc:1403] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.000s
W20260709 12:21:39.536566  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 52029.4us
I20260709 12:21:39.537142  3362 raft_consensus.cc:697] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:39.538085  3362 consensus_queue.cc:237] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.544066  3354 consensus_queue.cc:1048] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:39.550796  2959 raft_consensus.cc:359] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.551561  2959 raft_consensus.cc:385] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.551841  2959 raft_consensus.cc:740] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.552495  2959 consensus_queue.cc:260] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.555871  2959 ts_tablet_manager.cc:1434] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:21:39.556973  2959 tablet_bootstrap.cc:492] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:39.558085  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:39.558724  2837 raft_consensus.cc:3060] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.563680  2959 tablet_bootstrap.cc:654] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.584430  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6291f8f287034586b4e9c0c9220bc6e7" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:39.590790  2843 raft_consensus.cc:3060] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.584789  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:21:39.594280  2840 raft_consensus.cc:3060] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.606724  3001 consensus_queue.cc:1048] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:39.578680  2837 raft_consensus.cc:2468] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:39.607378  3397 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:39.615001  2840 raft_consensus.cc:2468] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:39.579105  3340 raft_consensus.cc:493] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.622380  2843 raft_consensus.cc:2468] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:21:39.628296  2880 leader_election.cc:304] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:39.630138  2880 leader_election.cc:304] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:21:39.629316  3397 raft_consensus.cc:2804] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.630955  3397 raft_consensus.cc:697] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:39.631362  3236 raft_consensus.cc:2804] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.626650  3340 raft_consensus.cc:515] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.638948  3340 leader_election.cc:290] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:39.631830  3397 consensus_queue.cc:237] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.631855  3236 raft_consensus.cc:697] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:21:39.641386  3355 consensus_queue.cc:1048] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:39.642050  3236 consensus_queue.cc:237] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.644630  2839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:39.647234  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:39.649832  2731 leader_election.cc:304] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:39.651583  3340 raft_consensus.cc:2749] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.685901  2666 catalog_manager.cc:5697] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:39.690469  3355 consensus_queue.cc:1048] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:39.716017  2959 tablet_bootstrap.cc:492] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:39.716549  2959 ts_tablet_manager.cc:1403] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.160s	user 0.010s	sys 0.012s
I20260709 12:21:39.723997  2959 raft_consensus.cc:359] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.725066  2959 raft_consensus.cc:385] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.725410  2959 raft_consensus.cc:740] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.726477  2959 consensus_queue.cc:260] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.728757  2959 ts_tablet_manager.cc:1434] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.003s
I20260709 12:21:39.733736  2959 tablet_bootstrap.cc:492] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:39.752283  2959 tablet_bootstrap.cc:654] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.758205  2666 catalog_manager.cc:5697] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:39.767959  2666 catalog_manager.cc:5697] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:39.786080  2764 raft_consensus.cc:3060] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.805168  2959 tablet_bootstrap.cc:492] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:39.805742  2959 ts_tablet_manager.cc:1403] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.072s	user 0.014s	sys 0.000s
I20260709 12:21:39.808682  2959 raft_consensus.cc:359] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.809427  2959 raft_consensus.cc:385] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.809747  2959 raft_consensus.cc:740] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.810487  2959 consensus_queue.cc:260] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.813340  2959 ts_tablet_manager.cc:1434] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:39.814458  2959 tablet_bootstrap.cc:492] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:39.818423  2760 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.827900  2959 tablet_bootstrap.cc:654] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.847131  2959 tablet_bootstrap.cc:492] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:39.848069  2959 ts_tablet_manager.cc:1403] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.003s
I20260709 12:21:39.850912  2959 raft_consensus.cc:359] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.851643  2959 raft_consensus.cc:385] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.851920  2959 raft_consensus.cc:740] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.852569  2959 consensus_queue.cc:260] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.854698  2959 ts_tablet_manager.cc:1434] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:21:39.856508  2959 tablet_bootstrap.cc:492] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5: Bootstrap starting.
I20260709 12:21:39.862046  2959 tablet_bootstrap.cc:654] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.868062  2959 tablet_bootstrap.cc:492] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5: No bootstrap required, opened a new log
I20260709 12:21:39.868479  2959 ts_tablet_manager.cc:1403] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 12:21:39.870995  2959 raft_consensus.cc:359] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.871670  2959 raft_consensus.cc:385] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.871920  2959 raft_consensus.cc:740] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.872536  2959 consensus_queue.cc:260] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.874682  2959 ts_tablet_manager.cc:1434] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:21:39.936779  3340 raft_consensus.cc:493] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.937295  3340 raft_consensus.cc:515] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:39.939116  3340 leader_election.cc:290] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:39.939741  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:39.940092  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:39.941668  2732 leader_election.cc:304] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:39.942338  3340 raft_consensus.cc:2749] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.959300  2764 raft_consensus.cc:3060] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.013700  3355 consensus_queue.cc:1048] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:40.026777  3397 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.027915  3404 raft_consensus.cc:493] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.028290  3236 consensus_queue.cc:1048] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.028507  3404 raft_consensus.cc:515] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:40.032399  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6291f8f287034586b4e9c0c9220bc6e7" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:40.030802  3404 leader_election.cc:290] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:40.035205  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6291f8f287034586b4e9c0c9220bc6e7" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:40.036006  2842 raft_consensus.cc:2393] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:40.038307  2732 leader_election.cc:304] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:40.040724  3404 raft_consensus.cc:3060] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.044544  3309 raft_consensus.cc:493] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.045111  3309 raft_consensus.cc:515] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:21:40.047187  3309 leader_election.cc:290] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:40.050199  3404 raft_consensus.cc:2749] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:40.050029  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:21:40.052392  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:21:40.053225  2842 raft_consensus.cc:2393] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 442c92e5267046cd92f46e6f469570a5 in current term 1: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:21:40.054522  2731 leader_election.cc:304] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:21:40.056285  3309 raft_consensus.cc:3060] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.069621  3309 raft_consensus.cc:2749] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:40.074489  3236 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:40.082551  3236 consensus_queue.cc:1048] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.094822  3236 consensus_queue.cc:1048] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.159380  3236 consensus_queue.cc:1048] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:40.175758  3370 consensus_queue.cc:1048] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.425840  2760 raft_consensus.cc:3060] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.848497  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:40.880393  3431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:40.880410  3433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:40.880792  3430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:40.883345  1517 server_base.cc:1061] running on GCE node
I20260709 12:21:40.885638  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:40.885949  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:40.886268  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599700886244 us; error 0 us; skew 500 ppm
I20260709 12:21:40.887089  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:40.890738  1517 webserver.cc:533] Webserver started at http://127.1.123.68:42907/ using document root <none> and password file <none>
I20260709 12:21:40.891327  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:40.891594  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:40.891937  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:40.893340  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-3-root/instance:
uuid: "f440bbb3ddd545819242017140636ad4"
format_stamp: "Formatted at 2026-07-09 12:21:40 on dist-test-slave-08l4"
I20260709 12:21:40.899641  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:40.905781  3438 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:40.906661  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:21:40.907001  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-3-root
uuid: "f440bbb3ddd545819242017140636ad4"
format_stamp: "Formatted at 2026-07-09 12:21:40 on dist-test-slave-08l4"
I20260709 12:21:40.907316  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:40.935083  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:40.936959  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:40.938969  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:40.941629  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:40.941855  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:40.942121  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:40.942294  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:41.057366  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.68:35397
I20260709 12:21:41.057487  3503 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.68:35397 every 8 connection(s)
I20260709 12:21:41.064471  1517 auto_leader_rebalancer-test.cc:439] add a tserver: f440bbb3ddd545819242017140636ad4
I20260709 12:21:41.127314  3504 heartbeater.cc:344] Connected to a master server at 127.1.123.126:36731
I20260709 12:21:41.127907  3504 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:41.128782  3504 heartbeater.cc:507] Master 127.1.123.126:36731 requested a full tablet report, sending...
I20260709 12:21:41.131138  2663 ts_manager.cc:194] Registered new tserver with Master: f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:21:41.132736  2663 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51706
I20260709 12:21:41.379842  2909 consensus_queue.cc:237] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:41.386730  2842 raft_consensus.cc:1275] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:41.386730  2763 raft_consensus.cc:1275] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:41.388743  3395 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:41.390218  3499 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:41.401193  3236 raft_consensus.cc:2955] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
W20260709 12:21:41.404731  2883 consensus_peers.cc:597] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2cba50a5280d4b00b9d2617ce7a22ec1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:41.405485  2760 raft_consensus.cc:2955] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.414903  2837 raft_consensus.cc:2955] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.422212  2909 consensus_queue.cc:237] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:41.429188  2663 catalog_manager.cc:5697] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:41.459038  2883 consensus_peers.cc:597] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b46ad496aaf14cbc8cf01697dceb4cf5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:41.460000  2837 raft_consensus.cc:1275] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:41.461639  3236 consensus_queue.cc:1048] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:41.467715  2760 raft_consensus.cc:1275] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:41.469899  3397 consensus_queue.cc:1048] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:41.481285  3395 raft_consensus.cc:2955] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.488399  2842 raft_consensus.cc:2955] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.492435  2760 raft_consensus.cc:2955] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.510618  2666 catalog_manager.cc:5697] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.550482  2764 consensus_queue.cc:237] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:41.566728  2909 raft_consensus.cc:1275] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:41.569056  3428 consensus_queue.cc:1048] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:41.573693  2837 raft_consensus.cc:1275] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:41.578668  3309 consensus_queue.cc:1048] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:21:41.584203  2734 consensus_peers.cc:597] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ff32c54020945d99bd67b7df9574bae. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:41.592693  3309 raft_consensus.cc:2955] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.599529  2916 raft_consensus.cc:2955] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.605765  2842 raft_consensus.cc:2955] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.623940  2664 catalog_manager.cc:5697] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.641871  2842 consensus_queue.cc:237] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:41.652320  2916 raft_consensus.cc:1275] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:41.654135  2764 raft_consensus.cc:1275] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:41.655160  3354 consensus_queue.cc:1048] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:41.656873  3354 consensus_queue.cc:1048] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:41.669796  3497 raft_consensus.cc:2955] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.671033  2764 raft_consensus.cc:2955] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.670846  2916 raft_consensus.cc:2955] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
W20260709 12:21:41.681865  2809 consensus_peers.cc:597] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb1bafa30972440499c69fb1221eae27. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:41.693132  2666 catalog_manager.cc:5697] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:41.945123  3534 ts_tablet_manager.cc:933] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:41.949601  3534 tablet_copy_client.cc:323] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:21:42.070683  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 2cba50a5280d4b00b9d2617ce7a22ec1 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:21:42.076812  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 2cba50a5280d4b00b9d2617ce7a22ec1 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-2cba50a5280d4b00b9d2617ce7a22ec1
I20260709 12:21:42.086189  2927 tablet_copy_source_session.cc:215] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:42.136868  3504 heartbeater.cc:499] Master 127.1.123.126:36731 was elected leader, sending a full tablet report...
I20260709 12:21:42.225819  3542 ts_tablet_manager.cc:933] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:42.232867  3534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cba50a5280d4b00b9d2617ce7a22ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:42.238533  3543 ts_tablet_manager.cc:933] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:42.254135  3543 tablet_copy_client.cc:323] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:21:42.254926  3542 tablet_copy_client.cc:323] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:21:42.258446  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet b46ad496aaf14cbc8cf01697dceb4cf5 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:21:42.259049  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet b46ad496aaf14cbc8cf01697dceb4cf5 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-b46ad496aaf14cbc8cf01697dceb4cf5
I20260709 12:21:42.270684  2927 tablet_copy_source_session.cc:215] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:42.276166  3534 tablet_copy_client.cc:806] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:42.280280  3534 tablet_copy_client.cc:670] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:42.282155  3543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46ad496aaf14cbc8cf01697dceb4cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:42.285697  3547 ts_tablet_manager.cc:933] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:21:42.288239  3534 tablet_copy_client.cc:538] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:42.303294  3547 tablet_copy_client.cc:323] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.65:35625
I20260709 12:21:42.314505  3543 tablet_copy_client.cc:806] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:42.315279  3543 tablet_copy_client.cc:670] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:42.319577  3543 tablet_copy_client.cc:538] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:42.324640  3534 tablet_bootstrap.cc:492] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:42.333005  2778 tablet_copy_service.cc:140] P 442c92e5267046cd92f46e6f469570a5: Received BeginTabletCopySession request for tablet 4ff32c54020945d99bd67b7df9574bae from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:33574)
I20260709 12:21:42.333527  3543 tablet_bootstrap.cc:492] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:42.333849  2778 tablet_copy_service.cc:161] P 442c92e5267046cd92f46e6f469570a5: Beginning new tablet copy session on tablet 4ff32c54020945d99bd67b7df9574bae from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:33574: session id = f440bbb3ddd545819242017140636ad4-4ff32c54020945d99bd67b7df9574bae
I20260709 12:21:42.343600  2778 tablet_copy_source_session.cc:215] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:42.347271  3547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff32c54020945d99bd67b7df9574bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:42.374058  3547 tablet_copy_client.cc:806] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:42.374742  3547 tablet_copy_client.cc:670] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:42.387322  3547 tablet_copy_client.cc:538] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:42.400928  3547 tablet_bootstrap.cc:492] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:42.404476  3534 tablet_bootstrap.cc:492] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:42.405740  3534 tablet_bootstrap.cc:492] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:42.406526  3534 ts_tablet_manager.cc:1403] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.082s	user 0.027s	sys 0.014s
I20260709 12:21:42.413945  3534 raft_consensus.cc:359] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:42.415947  3534 raft_consensus.cc:740] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:42.416724  3534 consensus_queue.cc:260] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:42.418751  2853 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet bb1bafa30972440499c69fb1221eae27 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:21:42.419427  2853 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet bb1bafa30972440499c69fb1221eae27 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-bb1bafa30972440499c69fb1221eae27
I20260709 12:21:42.432085  3534 ts_tablet_manager.cc:1434] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.025s	user 0.010s	sys 0.000s
I20260709 12:21:42.435266  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-2cba50a5280d4b00b9d2617ce7a22ec1 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:21:42.435904  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-2cba50a5280d4b00b9d2617ce7a22ec1 on tablet 2cba50a5280d4b00b9d2617ce7a22ec1 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:42.469868  3547 tablet_bootstrap.cc:492] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:42.473594  3547 tablet_bootstrap.cc:492] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:42.474792  3547 ts_tablet_manager.cc:1403] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.074s	user 0.028s	sys 0.000s
I20260709 12:21:42.527583  3475 raft_consensus.cc:1217] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:42.530622  3547 raft_consensus.cc:359] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:42.531665  3547 raft_consensus.cc:740] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
W20260709 12:21:42.534868  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 75469.2us
I20260709 12:21:42.520697  2853 tablet_copy_source_session.cc:215] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:42.532526  3547 consensus_queue.cc:260] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:42.551617  3542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1bafa30972440499c69fb1221eae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:42.567265  3547 ts_tablet_manager.cc:1434] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.092s	user 0.005s	sys 0.004s
I20260709 12:21:42.569631  2778 tablet_copy_service.cc:342] P 442c92e5267046cd92f46e6f469570a5: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-4ff32c54020945d99bd67b7df9574bae received from {username='slave'} at 127.0.0.1:33574
I20260709 12:21:42.570151  2778 tablet_copy_service.cc:434] P 442c92e5267046cd92f46e6f469570a5: ending tablet copy session f440bbb3ddd545819242017140636ad4-4ff32c54020945d99bd67b7df9574bae on tablet 4ff32c54020945d99bd67b7df9574bae with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:42.584033  3475 raft_consensus.cc:1217] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:42.591259  3543 tablet_bootstrap.cc:492] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:42.592564  3543 tablet_bootstrap.cc:492] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:42.593318  3543 ts_tablet_manager.cc:1403] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.260s	user 0.039s	sys 0.062s
I20260709 12:21:42.615250  3543 raft_consensus.cc:359] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:42.616242  3543 raft_consensus.cc:740] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:42.624791  3543 consensus_queue.cc:260] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:42.637809  3543 ts_tablet_manager.cc:1434] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 12:21:42.640882  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-b46ad496aaf14cbc8cf01697dceb4cf5 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:21:42.641405  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-b46ad496aaf14cbc8cf01697dceb4cf5 on tablet b46ad496aaf14cbc8cf01697dceb4cf5 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:42.643330  3542 tablet_copy_client.cc:806] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:42.644065  3542 tablet_copy_client.cc:670] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:42.649720  3542 tablet_copy_client.cc:538] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:42.661372  3542 tablet_bootstrap.cc:492] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:42.684891  3542 tablet_bootstrap.cc:492] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:42.685967  3542 tablet_bootstrap.cc:492] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:42.686713  3542 ts_tablet_manager.cc:1403] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.026s	user 0.022s	sys 0.005s
I20260709 12:21:42.690045  3542 raft_consensus.cc:359] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:42.690968  3542 raft_consensus.cc:740] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:42.691608  3542 consensus_queue.cc:260] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:42.694098  3542 ts_tablet_manager.cc:1434] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:42.698312  2853 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-bb1bafa30972440499c69fb1221eae27 received from {username='slave'} at 127.0.0.1:38132
I20260709 12:21:42.698812  2853 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-bb1bafa30972440499c69fb1221eae27 on tablet bb1bafa30972440499c69fb1221eae27 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:42.727727  3475 raft_consensus.cc:1217] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:42.949641  3395 raft_consensus.cc:1064] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:42.952123  3395 consensus_queue.cc:237] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:42.959339  3475 raft_consensus.cc:1275] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:42.960173  2764 raft_consensus.cc:1275] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:42.961143  2841 raft_consensus.cc:1275] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:42.961644  3539 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:42.962826  3395 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:42.964432  3556 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:42.992174  3499 raft_consensus.cc:2955] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:42.998502  3475 raft_consensus.cc:2955] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.022531  2664 catalog_manager.cc:5697] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.030519  2841 raft_consensus.cc:2955] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.039263  2764 raft_consensus.cc:2955] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:21:43.055197  2789 debug-util.cc:398] Leaking SignalData structure 0x7b08001c0280 after lost signal to thread 1520
W20260709 12:21:43.056545  2789 debug-util.cc:398] Leaking SignalData structure 0x7b08000a1620 after lost signal to thread 2700
W20260709 12:21:43.059963  2789 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5fa0 after lost signal to thread 2941
W20260709 12:21:43.061388  2789 debug-util.cc:398] Leaking SignalData structure 0x7b08002fd8a0 after lost signal to thread 3503
I20260709 12:21:43.080500  2911 consensus_queue.cc:237] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:43.119604  3475 raft_consensus.cc:1275] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:43.120388  2764 raft_consensus.cc:1275] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:43.121919  3395 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:43.125047  3499 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:43.134450  3539 raft_consensus.cc:2955] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.136092  3475 raft_consensus.cc:2955] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.153133  2764 raft_consensus.cc:2955] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.157039  3476 raft_consensus.cc:1217] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:43.162115  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2cba50a5280d4b00b9d2617ce7a22ec1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:43.180296  2664 catalog_manager.cc:5697] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.227869  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet 2cba50a5280d4b00b9d2617ce7a22ec1 with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:21:43.230356  3568 tablet_replica.cc:333] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:21:43.232249  3568 raft_consensus.cc:2243] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:43.233145  3568 raft_consensus.cc:2272] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:43.237725  3568 ts_tablet_manager.cc:1916] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:43.261606  3568 ts_tablet_manager.cc:1929] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:43.262300  3568 log.cc:1199] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/2cba50a5280d4b00b9d2617ce7a22ec1
I20260709 12:21:43.264213  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet 2cba50a5280d4b00b9d2617ce7a22ec1 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:43.266378  3544 raft_consensus.cc:1064] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:43.269750  3544 consensus_queue.cc:237] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:43.352331  3539 raft_consensus.cc:1064] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:43.355307  3539 consensus_queue.cc:237] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:43.384927  3475 raft_consensus.cc:1275] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:43.389837  3555 consensus_queue.cc:1048] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:43.393087  2911 raft_consensus.cc:1275] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:43.396505  3476 raft_consensus.cc:1275] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:43.398123  3544 consensus_queue.cc:1048] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:43.399035  2766 raft_consensus.cc:1275] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:43.400559  3499 consensus_queue.cc:1048] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:43.401808  3395 consensus_queue.cc:1048] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:43.419519  2841 raft_consensus.cc:1275] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:43.422598  3499 consensus_queue.cc:1048] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:43.449081  2837 raft_consensus.cc:1275] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:43.463349  3544 consensus_queue.cc:1048] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:43.476231  3556 raft_consensus.cc:2955] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.490548  3555 raft_consensus.cc:2955] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.500550  3475 raft_consensus.cc:2955] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.508069  2911 raft_consensus.cc:2955] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.515071  2841 raft_consensus.cc:2955] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.519724  2666 catalog_manager.cc:5697] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:43.531553  2839 raft_consensus.cc:2955] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.532435  3475 raft_consensus.cc:2955] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.551121  2766 raft_consensus.cc:2955] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.552402  2664 catalog_manager.cc:5697] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:43.554286  2763 consensus_queue.cc:237] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:43.585247  2841 raft_consensus.cc:1275] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:43.585338  3475 raft_consensus.cc:1275] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:43.589722  3555 consensus_queue.cc:1048] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:43.592522  2911 consensus_queue.cc:237] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:43.595225  3544 consensus_queue.cc:1048] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:43.595781  3557 raft_consensus.cc:1064] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:43.604036  3544 raft_consensus.cc:2955] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.606249  3476 raft_consensus.cc:1275] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:43.602640  3557 consensus_queue.cc:237] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:43.609212  3499 consensus_queue.cc:1048] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:43.621575  3397 raft_consensus.cc:2955] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.625520  3476 raft_consensus.cc:2955] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.630365  2841 raft_consensus.cc:1275] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:43.632185  3556 consensus_queue.cc:1048] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:43.639750  2839 raft_consensus.cc:2955] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.643013  2841 raft_consensus.cc:2955] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.655853  3475 raft_consensus.cc:2955] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.675043  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b46ad496aaf14cbc8cf01697dceb4cf5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:43.679934  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4ff32c54020945d99bd67b7df9574bae with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:43.706631  3475 raft_consensus.cc:1275] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:43.709851  3558 consensus_queue.cc:1048] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:43.720976  2666 catalog_manager.cc:5697] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:21:43.738034  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 50899.4us
I20260709 12:21:43.740077  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ff32c54020945d99bd67b7df9574bae with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47626
I20260709 12:21:43.744745  2664 catalog_manager.cc:5697] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.750186  2911 raft_consensus.cc:1275] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:43.751688  3583 tablet_replica.cc:333] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:21:43.759251  2764 raft_consensus.cc:1275] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:43.768342  2663 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet b46ad496aaf14cbc8cf01697dceb4cf5 with cas_config_opid_index 3: aborting the task: latest config opid_index 4; task opid_index 3
I20260709 12:21:43.768663  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet b46ad496aaf14cbc8cf01697dceb4cf5 with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:21:43.785806  3584 tablet_replica.cc:333] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:21:43.794191  3581 consensus_queue.cc:1048] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:43.812004  3353 consensus_queue.cc:1048] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:43.813356  3583 raft_consensus.cc:2243] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:43.814291  3583 raft_consensus.cc:2272] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:43.818295  3584 raft_consensus.cc:2243] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:43.819319  3584 raft_consensus.cc:2272] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:43.820618  3583 ts_tablet_manager.cc:1916] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:43.832451  3584 ts_tablet_manager.cc:1916] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:43.865094  3581 raft_consensus.cc:2955] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.875365  3583 ts_tablet_manager.cc:1929] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:43.880409  3583 log.cc:1199] T 4ff32c54020945d99bd67b7df9574bae P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/4ff32c54020945d99bd67b7df9574bae
I20260709 12:21:43.879432  2911 raft_consensus.cc:2955] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.884871  3584 ts_tablet_manager.cc:1929] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:43.883713  3475 raft_consensus.cc:2955] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.891608  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet 4ff32c54020945d99bd67b7df9574bae (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:43.894074  3584 log.cc:1199] T b46ad496aaf14cbc8cf01697dceb4cf5 P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/b46ad496aaf14cbc8cf01697dceb4cf5
I20260709 12:21:43.903817  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet b46ad496aaf14cbc8cf01697dceb4cf5 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:43.907441  2663 catalog_manager.cc:5697] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.933245  2759 raft_consensus.cc:2955] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.969744  2839 consensus_queue.cc:237] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:43.976729  3475 raft_consensus.cc:1275] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:43.978506  3581 consensus_queue.cc:1048] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:43.980002  2759 raft_consensus.cc:1275] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:43.981814  3581 consensus_queue.cc:1048] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:43.989097  3558 raft_consensus.cc:2955] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:43.996459  3476 raft_consensus.cc:2955] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:44.015632  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bb1bafa30972440499c69fb1221eae27 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:44.022486  2666 catalog_manager.cc:5697] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:44.001154  2764 raft_consensus.cc:2955] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:44.055691  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet bb1bafa30972440499c69fb1221eae27 with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47626
I20260709 12:21:44.056843  3583 tablet_replica.cc:333] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:21:44.058043  3583 raft_consensus.cc:2243] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:44.058789  3583 raft_consensus.cc:2272] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:44.068951  3583 ts_tablet_manager.cc:1916] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:44.094399  3583 ts_tablet_manager.cc:1929] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:44.095099  3583 log.cc:1199] T bb1bafa30972440499c69fb1221eae27 P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/bb1bafa30972440499c69fb1221eae27
I20260709 12:21:44.096974  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet bb1bafa30972440499c69fb1221eae27 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:45.148200  2839 consensus_queue.cc:237] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:45.157157  2911 raft_consensus.cc:1275] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:45.157295  2809 consensus_peers.cc:597] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3dbc0cbfcb67483f9f554976e140ba7f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:45.158167  2759 raft_consensus.cc:1275] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:45.158782  3581 consensus_queue.cc:1048] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:45.160646  3353 consensus_queue.cc:1048] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:21:45.182070  3353 raft_consensus.cc:2955] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.186064  2911 raft_consensus.cc:2955] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.189361  2759 raft_consensus.cc:2955] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.204566  2916 consensus_queue.cc:237] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:45.221925  2664 catalog_manager.cc:5697] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:45.245117  2883 consensus_peers.cc:597] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d500b7bb3bd042f09ffa7310e921aa24. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:45.245117  2839 raft_consensus.cc:1275] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:45.246424  2759 raft_consensus.cc:1275] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:45.247337  3539 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:45.248899  3588 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:45.267331  3588 raft_consensus.cc:2955] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.272634  2763 raft_consensus.cc:2955] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.287137  2839 raft_consensus.cc:2955] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.296497  2837 consensus_queue.cc:237] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:45.312930  2663 catalog_manager.cc:5697] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:45.325851  2809 consensus_peers.cc:597] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53752656656243c3abe78c7a9cb488c7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:45.328258  2763 raft_consensus.cc:1275] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:45.328585  2916 raft_consensus.cc:1275] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:45.330840  3353 consensus_queue.cc:1048] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:45.331923  3581 consensus_queue.cc:1048] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:45.353909  3581 raft_consensus.cc:2955] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.368248  2916 raft_consensus.cc:2955] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.370788  2763 raft_consensus.cc:2955] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.377777  2664 catalog_manager.cc:5697] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:45.373903  2764 consensus_queue.cc:237] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:45.411083  2916 raft_consensus.cc:1275] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:45.413519  2734 consensus_peers.cc:597] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e3545f99312f483eb6dc8965c5bc418b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:45.419016  2839 raft_consensus.cc:1275] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:45.418794  3309 consensus_queue.cc:1048] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:45.424669  3309 consensus_queue.cc:1048] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:45.435992  3428 raft_consensus.cc:2955] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.438006  2911 raft_consensus.cc:2955] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.445218  2837 raft_consensus.cc:2955] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.464536  2663 catalog_manager.cc:5697] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:45.663282  3608 ts_tablet_manager.cc:933] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:45.665897  3608 tablet_copy_client.cc:323] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:21:45.668088  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet d500b7bb3bd042f09ffa7310e921aa24 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:21:45.668715  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet d500b7bb3bd042f09ffa7310e921aa24 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-d500b7bb3bd042f09ffa7310e921aa24
I20260709 12:21:45.688009  2927 tablet_copy_source_session.cc:215] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:45.697058  3608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d500b7bb3bd042f09ffa7310e921aa24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:45.731307  3610 ts_tablet_manager.cc:933] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:45.738972  3608 tablet_copy_client.cc:806] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:45.739780  3608 tablet_copy_client.cc:670] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:45.744102  3608 tablet_copy_client.cc:538] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:45.746038  3610 tablet_copy_client.cc:323] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:21:45.752000  2853 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet 3dbc0cbfcb67483f9f554976e140ba7f from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:21:45.752579  2853 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet 3dbc0cbfcb67483f9f554976e140ba7f from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-3dbc0cbfcb67483f9f554976e140ba7f
I20260709 12:21:45.755208  3608 tablet_bootstrap.cc:492] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:45.770653  2853 tablet_copy_source_session.cc:215] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:45.776438  3610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dbc0cbfcb67483f9f554976e140ba7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:45.788023  3612 ts_tablet_manager.cc:933] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:45.806298  3612 tablet_copy_client.cc:323] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:21:45.808377  2853 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet 53752656656243c3abe78c7a9cb488c7 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:21:45.808947  2853 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet 53752656656243c3abe78c7a9cb488c7 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-53752656656243c3abe78c7a9cb488c7
I20260709 12:21:45.820873  2853 tablet_copy_source_session.cc:215] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:45.825416  3612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53752656656243c3abe78c7a9cb488c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:45.845008  3610 tablet_copy_client.cc:806] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:45.848687  3610 tablet_copy_client.cc:670] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:45.861832  3610 tablet_copy_client.cc:538] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:45.885057  3610 tablet_bootstrap.cc:492] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:45.896270  3612 tablet_copy_client.cc:806] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:45.897189  3612 tablet_copy_client.cc:670] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:45.904084  3612 tablet_copy_client.cc:538] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:45.922947  3612 tablet_bootstrap.cc:492] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:45.945654  3617 ts_tablet_manager.cc:933] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:21:45.954015  3617 tablet_copy_client.cc:323] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.65:35625
I20260709 12:21:45.955789  2778 tablet_copy_service.cc:140] P 442c92e5267046cd92f46e6f469570a5: Received BeginTabletCopySession request for tablet e3545f99312f483eb6dc8965c5bc418b from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:33574)
I20260709 12:21:45.956418  2778 tablet_copy_service.cc:161] P 442c92e5267046cd92f46e6f469570a5: Beginning new tablet copy session on tablet e3545f99312f483eb6dc8965c5bc418b from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:33574: session id = f440bbb3ddd545819242017140636ad4-e3545f99312f483eb6dc8965c5bc418b
I20260709 12:21:45.966440  3610 tablet_bootstrap.cc:492] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:45.970257  3610 tablet_bootstrap.cc:492] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:45.972589  3610 ts_tablet_manager.cc:1403] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.088s	user 0.017s	sys 0.012s
I20260709 12:21:45.985132  3610 raft_consensus.cc:359] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:45.986608  3610 raft_consensus.cc:740] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:45.987669  3610 consensus_queue.cc:260] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:46.011245  2778 tablet_copy_source_session.cc:215] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:46.011971  3612 tablet_bootstrap.cc:492] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:46.014073  3612 tablet_bootstrap.cc:492] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:46.015357  3617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3545f99312f483eb6dc8965c5bc418b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.016095  3612 ts_tablet_manager.cc:1403] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.094s	user 0.022s	sys 0.004s
I20260709 12:21:46.020746  3608 tablet_bootstrap.cc:492] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:46.023837  3608 tablet_bootstrap.cc:492] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:46.020768  3612 raft_consensus.cc:359] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:46.031154  3612 raft_consensus.cc:740] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:46.032153  3612 consensus_queue.cc:260] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:46.041641  3608 ts_tablet_manager.cc:1403] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.287s	user 0.032s	sys 0.029s
I20260709 12:21:46.038019  3617 tablet_copy_client.cc:806] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:46.043823  3617 tablet_copy_client.cc:670] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:46.053186  3610 ts_tablet_manager.cc:1434] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.079s	user 0.018s	sys 0.002s
I20260709 12:21:46.053853  3617 tablet_copy_client.cc:538] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:46.054337  3612 ts_tablet_manager.cc:1434] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.004s
I20260709 12:21:46.068187  3617 tablet_bootstrap.cc:492] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:46.072366  2853 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-3dbc0cbfcb67483f9f554976e140ba7f received from {username='slave'} at 127.0.0.1:38132
I20260709 12:21:46.072963  2853 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-3dbc0cbfcb67483f9f554976e140ba7f on tablet 3dbc0cbfcb67483f9f554976e140ba7f with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:46.075896  2853 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-53752656656243c3abe78c7a9cb488c7 received from {username='slave'} at 127.0.0.1:38132
I20260709 12:21:46.076318  2853 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-53752656656243c3abe78c7a9cb488c7 on tablet 53752656656243c3abe78c7a9cb488c7 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:46.136602  3608 raft_consensus.cc:359] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:46.147409  3608 raft_consensus.cc:740] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:46.148623  3608 consensus_queue.cc:260] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:46.169193  3617 tablet_bootstrap.cc:492] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:46.170660  3617 tablet_bootstrap.cc:492] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:46.171554  3617 ts_tablet_manager.cc:1403] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.104s	user 0.062s	sys 0.038s
I20260709 12:21:46.179121  3617 raft_consensus.cc:359] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:46.180526  3617 raft_consensus.cc:740] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:46.181595  3617 consensus_queue.cc:260] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:46.184366  3617 ts_tablet_manager.cc:1434] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:21:46.187741  2778 tablet_copy_service.cc:342] P 442c92e5267046cd92f46e6f469570a5: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-e3545f99312f483eb6dc8965c5bc418b received from {username='slave'} at 127.0.0.1:33574
I20260709 12:21:46.188228  2778 tablet_copy_service.cc:434] P 442c92e5267046cd92f46e6f469570a5: ending tablet copy session f440bbb3ddd545819242017140636ad4-e3545f99312f483eb6dc8965c5bc418b on tablet e3545f99312f483eb6dc8965c5bc418b with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:46.208917  3608 ts_tablet_manager.cc:1434] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.165s	user 0.005s	sys 0.003s
I20260709 12:21:46.210271  3476 raft_consensus.cc:1217] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:46.211596  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-d500b7bb3bd042f09ffa7310e921aa24 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:21:46.212090  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-d500b7bb3bd042f09ffa7310e921aa24 on tablet d500b7bb3bd042f09ffa7310e921aa24 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:46.240767  3476 raft_consensus.cc:1217] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:46.317098  3476 raft_consensus.cc:1217] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:46.513844  3476 raft_consensus.cc:1217] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:46.670349  3620 raft_consensus.cc:1064] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:46.671077  3353 raft_consensus.cc:1064] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:46.674429  3620 consensus_queue.cc:237] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:46.673612  3353 consensus_queue.cc:237] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:46.697029  3476 raft_consensus.cc:1275] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:46.703931  3539 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:46.762781  2837 raft_consensus.cc:1275] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:46.762568  2764 raft_consensus.cc:1275] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:46.766106  3588 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:46.770648  3588 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:46.784263  3615 raft_consensus.cc:1064] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:46.801708  3475 raft_consensus.cc:1275] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:46.818123  3593 consensus_queue.cc:1048] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:46.805037  2911 raft_consensus.cc:1275] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:46.805233  2763 raft_consensus.cc:1275] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:46.828130  3615 consensus_queue.cc:237] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:46.835727  3353 consensus_queue.cc:1048] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:46.839367  3581 consensus_queue.cc:1048] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:46.853295  3620 raft_consensus.cc:2955] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:46.861024  2763 raft_consensus.cc:2955] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:46.879496  3593 raft_consensus.cc:2955] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:46.894578  2666 catalog_manager.cc:5697] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:46.907428  2911 raft_consensus.cc:2955] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:46.916025  2764 raft_consensus.cc:2955] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:46.924317  3475 raft_consensus.cc:1275] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:46.927485  3581 consensus_queue.cc:1048] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:21:46.958093  2768 raft_consensus.cc:1275] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:46.960623  2916 raft_consensus.cc:1275] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:46.965386  2837 raft_consensus.cc:2955] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:46.968879  3476 raft_consensus.cc:2955] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:46.984973  3615 consensus_queue.cc:1048] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:47.004312  3475 raft_consensus.cc:2955] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.008584  3615 consensus_queue.cc:1048] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:47.027961  3625 raft_consensus.cc:1064] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:47.031512  3625 consensus_queue.cc:237] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:47.052305  2664 catalog_manager.cc:5697] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:47.059702  3472 raft_consensus.cc:1275] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:47.062071  2917 raft_consensus.cc:1275] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:47.062731  3309 consensus_queue.cc:1048] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:47.076287  3309 consensus_queue.cc:1048] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:47.086373  3615 raft_consensus.cc:2955] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.119464  2837 raft_consensus.cc:1275] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:21:47.124255  3646 log.cc:927] Time spent T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:21:47.110431  2916 raft_consensus.cc:2955] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.242153  3607 consensus_queue.cc:1048] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:21:47.246616  2842 consensus_queue.cc:237] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:47.216034  2764 raft_consensus.cc:2955] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.295068  3624 raft_consensus.cc:2955] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.318928  3472 raft_consensus.cc:2955] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:21:47.338963  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 120337us
I20260709 12:21:47.399678  2916 consensus_queue.cc:237] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:47.282585  3475 raft_consensus.cc:2955] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.466718  3476 raft_consensus.cc:1275] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:47.469178  3593 consensus_queue.cc:1048] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:47.474653  2917 raft_consensus.cc:1275] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:47.480099  3615 consensus_queue.cc:1048] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:47.486852  3615 raft_consensus.cc:2955] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.494763  2910 raft_consensus.cc:2955] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.511317  3472 raft_consensus.cc:1275] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:47.514295  2664 catalog_manager.cc:5697] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.519376  2666 catalog_manager.cc:5697] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:21:47.539218  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 108840us
W20260709 12:21:47.545637  3657 log.cc:927] Time spent T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:21:47.488816  3476 raft_consensus.cc:2955] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.551676  3640 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:21:47.492353  2917 raft_consensus.cc:2955] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.586418  3640 raft_consensus.cc:2955] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.601784  3472 raft_consensus.cc:2955] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.613790  2840 raft_consensus.cc:2955] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.633306  2663 catalog_manager.cc:5697] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:21:47.682160  2809 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.715920  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet 3dbc0cbfcb67483f9f554976e140ba7f with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:21:47.759899  2842 consensus_queue.cc:237] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:47.768324  2763 raft_consensus.cc:1275] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:47.771881  2663 catalog_manager.cc:5697] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.799101  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d500b7bb3bd042f09ffa7310e921aa24 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:47.805496  3659 tablet_replica.cc:333] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:21:47.807179  3659 raft_consensus.cc:2243] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.812351  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3dbc0cbfcb67483f9f554976e140ba7f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:47.816867  3472 raft_consensus.cc:1275] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:47.820111  2763 raft_consensus.cc:1275] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:47.822327  3353 consensus_queue.cc:1048] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:47.823526  3581 consensus_queue.cc:1048] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:47.820238  3659 raft_consensus.cc:2272] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.851392  2764 raft_consensus.cc:2955] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.856693  2759 consensus_queue.cc:237] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:47.848747  3581 raft_consensus.cc:2955] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.861581  3539 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:21:47.876171  3472 raft_consensus.cc:2955] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.900689  3476 raft_consensus.cc:1275] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:47.902357  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet d500b7bb3bd042f09ffa7310e921aa24 with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:21:47.900840  2910 raft_consensus.cc:1275] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:47.900679  3659 ts_tablet_manager.cc:1916] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:47.906687  3658 consensus_queue.cc:1048] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:47.914381  3658 consensus_queue.cc:1048] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:47.924990  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53752656656243c3abe78c7a9cb488c7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:47.932343  3607 raft_consensus.cc:2955] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.941495  3475 raft_consensus.cc:2955] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:47.947170  2662 catalog_manager.cc:5697] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:48.020118  2911 raft_consensus.cc:2955] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:48.028834  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e3545f99312f483eb6dc8965c5bc418b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:48.039718  2662 catalog_manager.cc:5697] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:48.044706  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet 53752656656243c3abe78c7a9cb488c7 with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47626
I20260709 12:21:48.055859  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet e3545f99312f483eb6dc8965c5bc418b with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:21:48.075350  3661 tablet_replica.cc:333] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:21:48.076561  3661 raft_consensus.cc:2243] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:48.080677  2764 raft_consensus.cc:2955] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:48.088539  3661 raft_consensus.cc:2272] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:48.107203  3661 ts_tablet_manager.cc:1916] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:48.110961  3659 ts_tablet_manager.cc:1929] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:48.112331  3659 log.cc:1199] T 3dbc0cbfcb67483f9f554976e140ba7f P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/3dbc0cbfcb67483f9f554976e140ba7f
I20260709 12:21:48.114044  3663 tablet_replica.cc:333] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:21:48.114734  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet 3dbc0cbfcb67483f9f554976e140ba7f (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:48.115481  3663 raft_consensus.cc:2243] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:48.116433  3663 raft_consensus.cc:2272] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:48.133204  3663 ts_tablet_manager.cc:1916] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:48.133212  3661 ts_tablet_manager.cc:1929] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:48.133999  3661 log.cc:1199] T d500b7bb3bd042f09ffa7310e921aa24 P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/d500b7bb3bd042f09ffa7310e921aa24
I20260709 12:21:48.136068  3661 tablet_replica.cc:333] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:21:48.137163  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet d500b7bb3bd042f09ffa7310e921aa24 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:48.137526  3661 raft_consensus.cc:2243] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:48.138482  3661 raft_consensus.cc:2272] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:48.142402  3661 ts_tablet_manager.cc:1916] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:48.153307  3663 ts_tablet_manager.cc:1929] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:48.153954  3663 log.cc:1199] T 53752656656243c3abe78c7a9cb488c7 P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/53752656656243c3abe78c7a9cb488c7
I20260709 12:21:48.155961  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet 53752656656243c3abe78c7a9cb488c7 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:48.164774  3661 ts_tablet_manager.cc:1929] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:48.165551  3661 log.cc:1199] T e3545f99312f483eb6dc8965c5bc418b P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/e3545f99312f483eb6dc8965c5bc418b
I20260709 12:21:48.167197  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet e3545f99312f483eb6dc8965c5bc418b (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:49.221666  2911 consensus_queue.cc:237] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:49.230247  2837 raft_consensus.cc:1275] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:49.230935  2883 consensus_peers.cc:597] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c737810e34ba42deaa41e6478e84fe87. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:49.232165  2764 raft_consensus.cc:1275] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:49.232497  3628 consensus_queue.cc:1048] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:49.233659  3665 consensus_queue.cc:1048] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:49.243456  3665 raft_consensus.cc:2955] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.247962  2837 raft_consensus.cc:2955] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.247844  2764 raft_consensus.cc:2955] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.270485  2759 consensus_queue.cc:237] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:49.276935  2663 catalog_manager.cc:5697] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.309839  2911 raft_consensus.cc:1275] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:21:49.310319  2734 consensus_peers.cc:597] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 547d5e0cde57459bbad31d06eb64d442. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:49.311764  3309 consensus_queue.cc:1048] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:49.315478  2842 raft_consensus.cc:1275] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:49.320297  3309 consensus_queue.cc:1048] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:49.332899  3607 raft_consensus.cc:2955] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.338277  2837 raft_consensus.cc:2955] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.409480  2911 raft_consensus.cc:2955] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
W20260709 12:21:49.412487  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 61247.2us
I20260709 12:21:49.438534  2911 consensus_queue.cc:237] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:49.447405  2662 catalog_manager.cc:5697] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.469331  2767 raft_consensus.cc:1275] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:49.469769  2883 consensus_peers.cc:597] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7bb611b2490b4682a0c5a45f2bbe7c94. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:49.470996  3588 consensus_queue.cc:1048] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:49.472133  2837 raft_consensus.cc:1275] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:49.473848  3588 consensus_queue.cc:1048] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:49.488135  3628 raft_consensus.cc:2955] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.489938  2764 raft_consensus.cc:2955] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.490800  2840 raft_consensus.cc:2955] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.506389  2911 consensus_queue.cc:237] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:49.513998  2665 catalog_manager.cc:5697] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.520891  2759 raft_consensus.cc:1275] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:49.523097  3641 consensus_queue.cc:1048] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:49.522249  2883 consensus_peers.cc:597] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f0df81094e2493cbe804d2dba9aa9d4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:49.526273  2840 raft_consensus.cc:1275] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:49.527736  3641 consensus_queue.cc:1048] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:49.547386  3665 raft_consensus.cc:2955] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.549363  2764 raft_consensus.cc:2955] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.576066  2662 catalog_manager.cc:5697] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.576506  2837 raft_consensus.cc:2955] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:49.700004  3689 ts_tablet_manager.cc:933] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:49.721208  3689 tablet_copy_client.cc:323] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:21:49.726337  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet c737810e34ba42deaa41e6478e84fe87 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:21:49.726995  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet c737810e34ba42deaa41e6478e84fe87 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-c737810e34ba42deaa41e6478e84fe87
I20260709 12:21:49.735126  2927 tablet_copy_source_session.cc:215] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:49.738795  3689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c737810e34ba42deaa41e6478e84fe87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.761163  3689 tablet_copy_client.cc:806] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:49.761891  3689 tablet_copy_client.cc:670] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:49.765859  3689 tablet_copy_client.cc:538] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:49.775979  3689 tablet_bootstrap.cc:492] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:49.799605  3689 tablet_bootstrap.cc:492] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:49.800707  3689 tablet_bootstrap.cc:492] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:49.801491  3689 ts_tablet_manager.cc:1403] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.005s
I20260709 12:21:49.805060  3689 raft_consensus.cc:359] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:49.806021  3689 raft_consensus.cc:740] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:49.806707  3689 consensus_queue.cc:260] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:49.810985  3689 ts_tablet_manager.cc:1434] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:21:49.813165  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-c737810e34ba42deaa41e6478e84fe87 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:21:49.813798  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-c737810e34ba42deaa41e6478e84fe87 on tablet c737810e34ba42deaa41e6478e84fe87 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:49.834983  3689 ts_tablet_manager.cc:933] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:21:49.837078  3689 tablet_copy_client.cc:323] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.65:35625
I20260709 12:21:49.839035  2778 tablet_copy_service.cc:140] P 442c92e5267046cd92f46e6f469570a5: Received BeginTabletCopySession request for tablet 547d5e0cde57459bbad31d06eb64d442 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:33574)
I20260709 12:21:49.839596  2778 tablet_copy_service.cc:161] P 442c92e5267046cd92f46e6f469570a5: Beginning new tablet copy session on tablet 547d5e0cde57459bbad31d06eb64d442 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:33574: session id = f440bbb3ddd545819242017140636ad4-547d5e0cde57459bbad31d06eb64d442
I20260709 12:21:49.850018  2778 tablet_copy_source_session.cc:215] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:49.853519  3689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 547d5e0cde57459bbad31d06eb64d442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.873098  3689 tablet_copy_client.cc:806] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:49.874228  3689 tablet_copy_client.cc:670] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:49.889146  3689 tablet_copy_client.cc:538] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:49.902019  3689 tablet_bootstrap.cc:492] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:49.926330  3689 tablet_bootstrap.cc:492] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:49.927415  3689 tablet_bootstrap.cc:492] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:49.928220  3689 ts_tablet_manager.cc:1403] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.026s	user 0.023s	sys 0.000s
I20260709 12:21:49.931919  3689 raft_consensus.cc:359] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:49.932928  3689 raft_consensus.cc:740] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:49.933713  3689 consensus_queue.cc:260] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:49.936496  3689 ts_tablet_manager.cc:1434] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 12:21:49.938851  2778 tablet_copy_service.cc:342] P 442c92e5267046cd92f46e6f469570a5: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-547d5e0cde57459bbad31d06eb64d442 received from {username='slave'} at 127.0.0.1:33574
I20260709 12:21:49.939360  2778 tablet_copy_service.cc:434] P 442c92e5267046cd92f46e6f469570a5: ending tablet copy session f440bbb3ddd545819242017140636ad4-547d5e0cde57459bbad31d06eb64d442 on tablet 547d5e0cde57459bbad31d06eb64d442 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:50.017315  3689 ts_tablet_manager.cc:933] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:50.019501  3689 tablet_copy_client.cc:323] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:21:50.021265  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 7bb611b2490b4682a0c5a45f2bbe7c94 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:21:50.022009  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 7bb611b2490b4682a0c5a45f2bbe7c94 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-7bb611b2490b4682a0c5a45f2bbe7c94
I20260709 12:21:50.040738  2927 tablet_copy_source_session.cc:215] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:50.047462  3689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bb611b2490b4682a0c5a45f2bbe7c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.062682  3696 ts_tablet_manager.cc:933] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:50.067567  3689 tablet_copy_client.cc:806] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:50.068331  3689 tablet_copy_client.cc:670] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:50.068605  3696 tablet_copy_client.cc:323] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:21:50.071425  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 1f0df81094e2493cbe804d2dba9aa9d4 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:21:50.072009  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 1f0df81094e2493cbe804d2dba9aa9d4 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-1f0df81094e2493cbe804d2dba9aa9d4
I20260709 12:21:50.079058  3689 tablet_copy_client.cc:538] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:50.085392  2927 tablet_copy_source_session.cc:215] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:50.089044  3689 tablet_bootstrap.cc:492] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:50.098721  3696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f0df81094e2493cbe804d2dba9aa9d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.115232  3689 tablet_bootstrap.cc:492] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:50.116271  3689 tablet_bootstrap.cc:492] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:50.116962  3689 ts_tablet_manager.cc:1403] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.011s
I20260709 12:21:50.119975  3689 raft_consensus.cc:359] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:50.120800  3689 raft_consensus.cc:740] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:50.121405  3689 consensus_queue.cc:260] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:50.127573  3689 ts_tablet_manager.cc:1434] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:21:50.128255  3696 tablet_copy_client.cc:806] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:50.128950  3696 tablet_copy_client.cc:670] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:50.129323  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-7bb611b2490b4682a0c5a45f2bbe7c94 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:21:50.129851  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-7bb611b2490b4682a0c5a45f2bbe7c94 on tablet 7bb611b2490b4682a0c5a45f2bbe7c94 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:50.135218  3696 tablet_copy_client.cc:538] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:50.146497  3696 tablet_bootstrap.cc:492] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:50.177796  3696 tablet_bootstrap.cc:492] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:50.178850  3696 tablet_bootstrap.cc:492] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:50.179587  3696 ts_tablet_manager.cc:1403] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.033s	user 0.019s	sys 0.003s
I20260709 12:21:50.183035  3696 raft_consensus.cc:359] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:50.183941  3696 raft_consensus.cc:740] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:50.184620  3696 consensus_queue.cc:260] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:50.187426  3696 ts_tablet_manager.cc:1434] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:50.190834  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-1f0df81094e2493cbe804d2dba9aa9d4 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:21:50.191290  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-1f0df81094e2493cbe804d2dba9aa9d4 on tablet 1f0df81094e2493cbe804d2dba9aa9d4 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:50.199797  3475 raft_consensus.cc:1217] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:50.366650  3475 raft_consensus.cc:1217] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:21:50.603401  3475 raft_consensus.cc:1217] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:50.654083  3475 raft_consensus.cc:1217] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:50.675838  3665 raft_consensus.cc:1064] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:50.679519  3665 consensus_queue.cc:237] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:50.686246  3475 raft_consensus.cc:1275] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:50.686930  2837 raft_consensus.cc:1275] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:50.686966  2767 raft_consensus.cc:1275] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:50.688452  3628 consensus_queue.cc:1048] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:50.689481  3700 consensus_queue.cc:1048] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:50.702809  3641 consensus_queue.cc:1048] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:50.745821  3665 raft_consensus.cc:2955] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:50.752897  3475 raft_consensus.cc:2955] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:50.754158  2766 raft_consensus.cc:2955] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:50.762547  2837 raft_consensus.cc:2955] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:50.771173  2663 catalog_manager.cc:5697] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:50.814842  2910 consensus_queue.cc:237] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:50.821995  3475 raft_consensus.cc:1275] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:50.823607  3628 consensus_queue.cc:1048] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:50.830971  2837 raft_consensus.cc:1275] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:50.832072  3665 raft_consensus.cc:2955] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:50.834247  3704 consensus_queue.cc:1048] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:50.841745  2837 raft_consensus.cc:2955] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:50.845824  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c737810e34ba42deaa41e6478e84fe87 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:50.844129  3475 raft_consensus.cc:2955] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:50.857096  2662 catalog_manager.cc:5697] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:50.878437  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet c737810e34ba42deaa41e6478e84fe87 with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:21:50.880383  3712 tablet_replica.cc:333] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:21:50.881610  3712 raft_consensus.cc:2243] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:50.882792  3712 raft_consensus.cc:2272] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:50.886320  3712 ts_tablet_manager.cc:1916] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:50.903846  3712 ts_tablet_manager.cc:1929] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:50.904284  3712 log.cc:1199] T c737810e34ba42deaa41e6478e84fe87 P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/c737810e34ba42deaa41e6478e84fe87
I20260709 12:21:50.905843  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet c737810e34ba42deaa41e6478e84fe87 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:50.915014  3676 raft_consensus.cc:1064] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:50.919956  3676 consensus_queue.cc:237] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:50.928404  3475 raft_consensus.cc:1275] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:50.928606  2911 raft_consensus.cc:1275] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:50.930346  3714 consensus_queue.cc:1048] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:50.931619  3678 consensus_queue.cc:1048] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:50.932358  2837 raft_consensus.cc:1275] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:50.938390  3715 consensus_queue.cc:1048] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:50.952234  3715 raft_consensus.cc:2955] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:50.954057  3475 raft_consensus.cc:2955] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:50.956983  2837 raft_consensus.cc:2955] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:50.970249  2662 catalog_manager.cc:5697] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:50.978297  2911 raft_consensus.cc:2955] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.006134  2767 consensus_queue.cc:237] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:51.041522  2840 raft_consensus.cc:1275] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:51.043352  3715 consensus_queue.cc:1048] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:51.044603  3475 raft_consensus.cc:1275] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:51.046195  3678 consensus_queue.cc:1048] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:51.051920  3715 raft_consensus.cc:2955] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.059959  2837 raft_consensus.cc:2955] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.084769  3475 raft_consensus.cc:2955] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.091143  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 547d5e0cde57459bbad31d06eb64d442 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:51.101500  2662 catalog_manager.cc:5697] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:51.118132  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet 547d5e0cde57459bbad31d06eb64d442 with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47626
I20260709 12:21:51.141183  3719 tablet_replica.cc:333] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:21:51.142179  3719 raft_consensus.cc:2243] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:51.143050  3719 raft_consensus.cc:2272] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:51.147514  3665 raft_consensus.cc:1064] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:51.148656  3719 ts_tablet_manager.cc:1916] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:51.150494  3665 consensus_queue.cc:237] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:51.167909  3475 raft_consensus.cc:1275] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:51.169950  3628 consensus_queue.cc:1048] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:51.174045  3719 ts_tablet_manager.cc:1929] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:21:51.174607  3719 log.cc:1199] T 547d5e0cde57459bbad31d06eb64d442 P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/547d5e0cde57459bbad31d06eb64d442
I20260709 12:21:51.175386  2837 raft_consensus.cc:1275] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:51.176913  3628 consensus_queue.cc:1048] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:51.179459  2764 raft_consensus.cc:1275] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:51.181876  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet 547d5e0cde57459bbad31d06eb64d442 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:51.182564  3628 consensus_queue.cc:1048] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:51.192615  3628 raft_consensus.cc:2955] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.195147  2764 raft_consensus.cc:2955] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.196307  2840 raft_consensus.cc:2955] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.206131  3475 raft_consensus.cc:2955] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.220670  3665 raft_consensus.cc:1064] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:51.223380  3665 consensus_queue.cc:237] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:51.241689  3475 raft_consensus.cc:1275] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:51.243925  3641 consensus_queue.cc:1048] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:51.248281  2766 raft_consensus.cc:1275] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:51.249797  2665 catalog_manager.cc:5697] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.253140  2837 raft_consensus.cc:1275] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:51.255018  3641 consensus_queue.cc:1048] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:51.256924  3700 consensus_queue.cc:1048] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:21:51.371922  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 68472.1us
W20260709 12:21:51.385480  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 102249us
I20260709 12:21:51.389874  2910 consensus_queue.cc:237] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:51.408838  2837 raft_consensus.cc:1275] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:51.420887  3475 raft_consensus.cc:1275] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:21:51.436434  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 149410us
I20260709 12:21:51.438181  3729 consensus_queue.cc:1048] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:51.439512  3665 raft_consensus.cc:2955] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.444605  3725 consensus_queue.cc:1048] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:51.451395  3475 raft_consensus.cc:2955] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.453892  2837 raft_consensus.cc:2955] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.449666  3729 raft_consensus.cc:2955] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.465287  2840 raft_consensus.cc:2955] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.480345  2764 raft_consensus.cc:2955] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.482124  2663 catalog_manager.cc:5697] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.484081  3476 raft_consensus.cc:2955] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.548578  2662 catalog_manager.cc:5697] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.553568  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7bb611b2490b4682a0c5a45f2bbe7c94 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:51.588723  2911 consensus_queue.cc:237] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:51.595130  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet 7bb611b2490b4682a0c5a45f2bbe7c94 with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:21:51.597688  3732 tablet_replica.cc:333] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:21:51.599257  3732 raft_consensus.cc:2243] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:51.600281  3732 raft_consensus.cc:2272] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:51.603298  2764 raft_consensus.cc:1275] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:51.605020  3725 consensus_queue.cc:1048] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:51.605786  3732 ts_tablet_manager.cc:1916] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:51.612555  3665 raft_consensus.cc:2955] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.625316  2767 raft_consensus.cc:2955] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.632263  3475 raft_consensus.cc:1275] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:51.635953  3700 consensus_queue.cc:1048] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:51.641160  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1f0df81094e2493cbe804d2dba9aa9d4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:51.643337  3475 raft_consensus.cc:2955] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:51.660847  3732 ts_tablet_manager.cc:1929] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:51.661367  3732 log.cc:1199] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/7bb611b2490b4682a0c5a45f2bbe7c94
I20260709 12:21:51.667308  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet 7bb611b2490b4682a0c5a45f2bbe7c94 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:51.671130  2665 catalog_manager.cc:5697] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:21:51.679227  2651 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 1f0df81094e2493cbe804d2dba9aa9d4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 12:21:51.695389  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f0df81094e2493cbe804d2dba9aa9d4 with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:21:51.697329  3733 tablet_replica.cc:333] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:21:51.704629  3733 raft_consensus.cc:2243] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:51.705407  3733 raft_consensus.cc:2272] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:51.708958  3733 ts_tablet_manager.cc:1916] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:51.723052  3733 ts_tablet_manager.cc:1929] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:51.723461  3733 log.cc:1199] T 1f0df81094e2493cbe804d2dba9aa9d4 P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/1f0df81094e2493cbe804d2dba9aa9d4
I20260709 12:21:51.725073  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet 1f0df81094e2493cbe804d2dba9aa9d4 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:52.775965  2916 consensus_queue.cc:237] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:52.782709  2842 raft_consensus.cc:1275] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:52.782915  2767 raft_consensus.cc:1275] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:52.783116  2883 consensus_peers.cc:597] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 63fa205fa4864ef4b19bda073c06a5d4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.784345  3729 consensus_queue.cc:1048] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:52.785351  3700 consensus_queue.cc:1048] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:52.800024  3700 raft_consensus.cc:2955] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:52.803148  2842 raft_consensus.cc:2955] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:52.803488  2767 raft_consensus.cc:2955] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:52.815758  2840 consensus_queue.cc:237] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:52.821130  2663 catalog_manager.cc:5697] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:52.827977  2809 consensus_peers.cc:597] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dc860efbfaf84207b7128fd001b59fe6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.831770  2916 raft_consensus.cc:1275] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:52.832438  2764 raft_consensus.cc:1275] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:52.833199  3701 consensus_queue.cc:1048] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:52.835079  3734 consensus_queue.cc:1048] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:52.858503  3701 raft_consensus.cc:2955] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:52.861274  2916 raft_consensus.cc:2955] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:52.862711  2767 raft_consensus.cc:2955] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:52.889483  2916 consensus_queue.cc:237] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:52.898361  2664 catalog_manager.cc:5697] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:52.910280  2764 raft_consensus.cc:1275] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:52.911670  2883 consensus_peers.cc:597] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9164313f06e045c3bdef4a46d365873c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.914319  2842 raft_consensus.cc:1275] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:52.920415  3700 consensus_queue.cc:1048] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:52.921489  3729 consensus_queue.cc:1048] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:52.930678  3729 raft_consensus.cc:2955] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:52.932679  2842 raft_consensus.cc:2955] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:52.946023  2764 raft_consensus.cc:2955] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:52.950312  2665 catalog_manager.cc:5697] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:52.950984  2916 consensus_queue.cc:237] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
W20260709 12:21:52.967653  2883 consensus_peers.cc:597] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6855fa72e5f9486bb478855f721e77ce. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.970147  2764 raft_consensus.cc:1275] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:52.971194  2839 raft_consensus.cc:1275] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:52.972630  3749 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:52.974639  3749 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:52.993570  3729 raft_consensus.cc:2955] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:53.006685  2839 raft_consensus.cc:2955] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:53.007038  2767 raft_consensus.cc:2955] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:53.035115  2664 catalog_manager.cc:5697] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:53.347065  3754 ts_tablet_manager.cc:933] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:53.350144  3755 ts_tablet_manager.cc:933] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:53.359174  3755 tablet_copy_client.cc:323] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:21:53.362632  2853 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet dc860efbfaf84207b7128fd001b59fe6 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:21:53.363276  2853 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet dc860efbfaf84207b7128fd001b59fe6 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-dc860efbfaf84207b7128fd001b59fe6
I20260709 12:21:53.365487  3754 tablet_copy_client.cc:323] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:21:53.367460  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 63fa205fa4864ef4b19bda073c06a5d4 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:21:53.368021  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 63fa205fa4864ef4b19bda073c06a5d4 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-63fa205fa4864ef4b19bda073c06a5d4
I20260709 12:21:53.373004  2853 tablet_copy_source_session.cc:215] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:53.377588  3755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc860efbfaf84207b7128fd001b59fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.379592  2927 tablet_copy_source_session.cc:215] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:53.384874  3754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63fa205fa4864ef4b19bda073c06a5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.402796  3755 tablet_copy_client.cc:806] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:53.403599  3755 tablet_copy_client.cc:670] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:53.404383  3754 tablet_copy_client.cc:806] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:53.405086  3754 tablet_copy_client.cc:670] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:53.412982  3755 tablet_copy_client.cc:538] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:53.419377  3754 tablet_copy_client.cc:538] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:53.424293  3755 tablet_bootstrap.cc:492] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:53.438055  3754 tablet_bootstrap.cc:492] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:53.468159  3755 tablet_bootstrap.cc:492] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:53.469322  3755 tablet_bootstrap.cc:492] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:53.470167  3755 ts_tablet_manager.cc:1403] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.046s	user 0.021s	sys 0.025s
I20260709 12:21:53.473506  3755 raft_consensus.cc:359] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:53.474834  3755 raft_consensus.cc:740] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:53.475620  3755 consensus_queue.cc:260] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:53.493682  3754 tablet_bootstrap.cc:492] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:53.495370  3754 tablet_bootstrap.cc:492] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:53.496426  3754 ts_tablet_manager.cc:1403] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.059s	user 0.021s	sys 0.010s
I20260709 12:21:53.499436  3760 ts_tablet_manager.cc:933] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:53.500780  3754 raft_consensus.cc:359] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:53.501965  3754 raft_consensus.cc:740] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:53.502805  3754 consensus_queue.cc:260] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:53.522379  3754 ts_tablet_manager.cc:1434] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.025s	user 0.011s	sys 0.003s
I20260709 12:21:53.530040  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-63fa205fa4864ef4b19bda073c06a5d4 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:21:53.530616  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-63fa205fa4864ef4b19bda073c06a5d4 on tablet 63fa205fa4864ef4b19bda073c06a5d4 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:53.539319  3760 tablet_copy_client.cc:323] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:21:53.541918  3755 ts_tablet_manager.cc:1434] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.071s	user 0.010s	sys 0.041s
I20260709 12:21:53.542924  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 6855fa72e5f9486bb478855f721e77ce from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:21:53.550710  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 6855fa72e5f9486bb478855f721e77ce from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-6855fa72e5f9486bb478855f721e77ce
I20260709 12:21:53.560009  2927 tablet_copy_source_session.cc:215] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:53.543411  3754 ts_tablet_manager.cc:933] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:53.569586  3754 tablet_copy_client.cc:323] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:21:53.571946  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 9164313f06e045c3bdef4a46d365873c from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:21:53.572551  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 9164313f06e045c3bdef4a46d365873c from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-9164313f06e045c3bdef4a46d365873c
I20260709 12:21:53.573894  3760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6855fa72e5f9486bb478855f721e77ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.574112  2853 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-dc860efbfaf84207b7128fd001b59fe6 received from {username='slave'} at 127.0.0.1:38132
I20260709 12:21:53.574613  2853 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-dc860efbfaf84207b7128fd001b59fe6 on tablet dc860efbfaf84207b7128fd001b59fe6 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:53.581447  2927 tablet_copy_source_session.cc:215] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:53.585300  3754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9164313f06e045c3bdef4a46d365873c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.595909  3760 tablet_copy_client.cc:806] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:53.596752  3760 tablet_copy_client.cc:670] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:53.614485  3760 tablet_copy_client.cc:538] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:53.617681  3754 tablet_copy_client.cc:806] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:53.618462  3754 tablet_copy_client.cc:670] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:53.625707  3760 tablet_bootstrap.cc:492] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:53.627200  3754 tablet_copy_client.cc:538] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:53.652504  3754 tablet_bootstrap.cc:492] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:53.671547  3760 tablet_bootstrap.cc:492] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:53.672739  3760 tablet_bootstrap.cc:492] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:53.673650  3760 ts_tablet_manager.cc:1403] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.048s	user 0.028s	sys 0.020s
I20260709 12:21:53.684880  3760 raft_consensus.cc:359] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:53.686084  3760 raft_consensus.cc:740] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:53.686800  3760 consensus_queue.cc:260] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:53.689855  3760 ts_tablet_manager.cc:1434] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 12:21:53.695921  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-6855fa72e5f9486bb478855f721e77ce received from {username='slave'} at 127.0.0.1:59582
I20260709 12:21:53.696424  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-6855fa72e5f9486bb478855f721e77ce on tablet 6855fa72e5f9486bb478855f721e77ce with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:53.707499  3754 tablet_bootstrap.cc:492] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:53.708674  3754 tablet_bootstrap.cc:492] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:53.709457  3754 ts_tablet_manager.cc:1403] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.058s	user 0.025s	sys 0.005s
I20260709 12:21:53.712900  3754 raft_consensus.cc:359] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:53.713876  3754 raft_consensus.cc:740] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:53.714527  3754 consensus_queue.cc:260] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:53.717279  3754 ts_tablet_manager.cc:1434] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:53.719451  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-9164313f06e045c3bdef4a46d365873c received from {username='slave'} at 127.0.0.1:59582
I20260709 12:21:53.719898  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-9164313f06e045c3bdef4a46d365873c on tablet 9164313f06e045c3bdef4a46d365873c with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:53.816295  3476 raft_consensus.cc:1217] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:53.906428  3476 raft_consensus.cc:1217] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:53.981616  3476 raft_consensus.cc:1217] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:54.104960  3476 raft_consensus.cc:1217] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:54.266160  3615 raft_consensus.cc:1064] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:54.268922  3615 consensus_queue.cc:237] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:54.275440  3476 raft_consensus.cc:1275] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.277213  3768 consensus_queue.cc:1048] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:54.282774  2764 raft_consensus.cc:1275] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.276563  2911 raft_consensus.cc:1275] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.291729  3615 consensus_queue.cc:1048] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:54.292835  3768 consensus_queue.cc:1048] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:54.312194  3701 raft_consensus.cc:2955] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.314028  2916 raft_consensus.cc:2955] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.316708  2763 raft_consensus.cc:2955] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.332789  3476 raft_consensus.cc:2955] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.400673  2664 catalog_manager.cc:5697] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:54.411226  3749 raft_consensus.cc:1064] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:54.414536  3749 consensus_queue.cc:237] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:54.452453  2839 raft_consensus.cc:1275] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.453004  3476 raft_consensus.cc:1275] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.454241  3665 consensus_queue.cc:1048] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:54.456574  3665 consensus_queue.cc:1048] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:54.462744  2763 raft_consensus.cc:1275] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.470309  3749 consensus_queue.cc:1048] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:54.497310  3700 raft_consensus.cc:2955] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.510484  2763 raft_consensus.cc:2955] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.621101  2841 raft_consensus.cc:2955] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.631678  3476 raft_consensus.cc:2955] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.653162  3751 raft_consensus.cc:1064] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:54.656045  3751 consensus_queue.cc:237] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:54.675673  2841 consensus_queue.cc:237] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:54.692962  3476 raft_consensus.cc:1275] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.694401  2664 catalog_manager.cc:5697] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.697010  2841 raft_consensus.cc:1275] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.697467  3476 raft_consensus.cc:1275] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:54.697284  3785 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:54.699038  3615 consensus_queue.cc:1048] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:54.703001  3789 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:54.706233  2764 raft_consensus.cc:1275] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:54.709460  3787 raft_consensus.cc:2955] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.713987  3782 consensus_queue.cc:1048] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:54.717973  2768 raft_consensus.cc:1275] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.721485  3785 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:54.720752  3475 raft_consensus.cc:2955] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.732859  3785 raft_consensus.cc:1064] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:54.736652  3785 consensus_queue.cc:237] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:54.756425  2764 raft_consensus.cc:2955] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.765767  2910 consensus_queue.cc:237] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:54.773384  3729 raft_consensus.cc:2955] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.788528  2663 catalog_manager.cc:5697] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.802276  2839 raft_consensus.cc:1275] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:54.804282  3789 consensus_queue.cc:1048] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:54.805455  3474 raft_consensus.cc:1275] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:54.809844  3751 consensus_queue.cc:1048] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:54.815416  3475 raft_consensus.cc:1275] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.816955  3785 consensus_queue.cc:1048] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:54.810510  3476 raft_consensus.cc:2955] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.826241  2764 raft_consensus.cc:1275] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.827031  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet dc860efbfaf84207b7128fd001b59fe6 with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47626
I20260709 12:21:54.828188  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dc860efbfaf84207b7128fd001b59fe6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:54.829242  2841 raft_consensus.cc:1275] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:54.833729  3665 consensus_queue.cc:1048] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:54.835608  2843 raft_consensus.cc:2955] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.837193  2837 raft_consensus.cc:2955] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.846338  3472 raft_consensus.cc:2955] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.833247  3785 raft_consensus.cc:2955] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.912483  3665 consensus_queue.cc:1048] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:54.918390  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 63fa205fa4864ef4b19bda073c06a5d4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:54.911688  2665 catalog_manager.cc:5697] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:54.932924  2664 catalog_manager.cc:5697] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.934113  2768 raft_consensus.cc:2955] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.957592  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet 63fa205fa4864ef4b19bda073c06a5d4 with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:21:54.960299  3800 tablet_replica.cc:333] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:21:54.972631  3789 raft_consensus.cc:2955] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.972705  3797 tablet_replica.cc:333] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:21:54.992561  3797 raft_consensus.cc:2243] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:54.993548  3797 raft_consensus.cc:2272] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:54.993331  3472 raft_consensus.cc:2955] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:54.998226  3797 ts_tablet_manager.cc:1916] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:55.012513  2841 raft_consensus.cc:2955] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:55.015818  2763 raft_consensus.cc:2955] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:55.036024  3800 raft_consensus.cc:2243] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.043108  2916 consensus_queue.cc:237] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:55.047039  2665 catalog_manager.cc:5697] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:55.052914  3800 raft_consensus.cc:2272] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.057973  3800 ts_tablet_manager.cc:1916] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:55.058920  2764 raft_consensus.cc:1275] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:55.060863  3785 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:55.025487  3797 ts_tablet_manager.cc:1929] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:55.062206  3797 log.cc:1199] T dc860efbfaf84207b7128fd001b59fe6 P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/dc860efbfaf84207b7128fd001b59fe6
I20260709 12:21:55.064080  3472 raft_consensus.cc:1275] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:55.068840  3785 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:55.070592  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet dc860efbfaf84207b7128fd001b59fe6 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:55.094794  2910 consensus_queue.cc:237] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:55.104765  3474 raft_consensus.cc:1275] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:55.107425  3784 consensus_queue.cc:1048] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:55.110002  3800 ts_tablet_manager.cc:1929] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:55.110589  3800 log.cc:1199] T 63fa205fa4864ef4b19bda073c06a5d4 P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/63fa205fa4864ef4b19bda073c06a5d4
I20260709 12:21:55.110301  3474 raft_consensus.cc:2955] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:55.113422  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet 63fa205fa4864ef4b19bda073c06a5d4 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:55.098896  3789 raft_consensus.cc:2955] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:55.103792  2768 raft_consensus.cc:2955] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:55.121363  3700 raft_consensus.cc:2955] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:21:55.123461  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 74874.4us
I20260709 12:21:55.126102  2767 raft_consensus.cc:1275] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:55.139523  2664 catalog_manager.cc:5697] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:55.147951  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6855fa72e5f9486bb478855f721e77ce with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:55.155206  3665 consensus_queue.cc:1048] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:55.157264  3474 raft_consensus.cc:2955] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:55.169617  2763 raft_consensus.cc:2955] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:55.173627  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9164313f06e045c3bdef4a46d365873c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:55.177178  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet 6855fa72e5f9486bb478855f721e77ce with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:21:55.195601  2665 catalog_manager.cc:5697] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:55.219584  3802 tablet_replica.cc:333] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:21:55.220989  3802 raft_consensus.cc:2243] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.221841  3802 raft_consensus.cc:2272] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.227159  3802 ts_tablet_manager.cc:1916] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:55.232512  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet 9164313f06e045c3bdef4a46d365873c with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:21:55.245087  3802 ts_tablet_manager.cc:1929] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:55.245609  3802 log.cc:1199] T 6855fa72e5f9486bb478855f721e77ce P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/6855fa72e5f9486bb478855f721e77ce
I20260709 12:21:55.247290  3802 tablet_replica.cc:333] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:21:55.247216  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet 6855fa72e5f9486bb478855f721e77ce (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:55.250408  3802 raft_consensus.cc:2243] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.251189  3802 raft_consensus.cc:2272] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.254127  3802 ts_tablet_manager.cc:1916] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:55.269889  3802 ts_tablet_manager.cc:1929] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:55.270334  3802 log.cc:1199] T 9164313f06e045c3bdef4a46d365873c P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/9164313f06e045c3bdef4a46d365873c
I20260709 12:21:55.272056  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet 9164313f06e045c3bdef4a46d365873c (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:56.308514  2841 consensus_queue.cc:237] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:56.316149  2910 raft_consensus.cc:1275] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:56.317075  2809 consensus_peers.cc:597] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 710edfdfdd7e4720b77f4ec27f5f41c0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:56.317588  2763 raft_consensus.cc:1275] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:56.318323  3615 consensus_queue.cc:1048] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:56.319157  3778 consensus_queue.cc:1048] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:56.328465  3615 raft_consensus.cc:2955] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.331281  2910 raft_consensus.cc:2955] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.332626  2763 raft_consensus.cc:2955] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.364513  2663 catalog_manager.cc:5697] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.401939  2910 consensus_queue.cc:237] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:56.410161  2841 raft_consensus.cc:1275] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:56.411103  2763 raft_consensus.cc:1275] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:56.411883  3784 consensus_queue.cc:1048] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:56.413161  2883 consensus_peers.cc:597] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a1a771f5b2742d8bd641ec0afec2cf4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:56.415084  3784 consensus_queue.cc:1048] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:56.439455  3785 raft_consensus.cc:2955] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.441133  2763 raft_consensus.cc:2955] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.442709  2843 raft_consensus.cc:2955] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.455796  2841 consensus_queue.cc:237] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:56.462492  2664 catalog_manager.cc:5697] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.469540  2910 raft_consensus.cc:1275] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:56.470250  2809 consensus_peers.cc:597] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a6389f65afb4adabe1d70b6f09e99e7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:56.469715  2763 raft_consensus.cc:1275] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:56.471400  3806 consensus_queue.cc:1048] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:56.473963  3806 consensus_queue.cc:1048] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:56.482172  3807 raft_consensus.cc:2955] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.489018  2763 raft_consensus.cc:2955] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.489450  2910 raft_consensus.cc:2955] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.504474  2916 consensus_queue.cc:237] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:56.513921  2663 catalog_manager.cc:5697] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
W20260709 12:21:56.534907  2883 consensus_peers.cc:597] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a7c3a03a1304452ebc48e986f5629037. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:56.537744  2843 raft_consensus.cc:1275] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:56.539836  2763 raft_consensus.cc:1275] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:56.541378  3784 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:56.544072  3785 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:56.554672  3784 raft_consensus.cc:2955] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.557196  2768 raft_consensus.cc:2955] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.558897  2843 raft_consensus.cc:2955] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.573808  2664 catalog_manager.cc:5697] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:21:56.807873  3822 ts_tablet_manager.cc:933] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:56.810243  3822 tablet_copy_client.cc:323] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:21:56.812179  2853 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet 710edfdfdd7e4720b77f4ec27f5f41c0 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:21:56.812778  2853 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet 710edfdfdd7e4720b77f4ec27f5f41c0 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-710edfdfdd7e4720b77f4ec27f5f41c0
I20260709 12:21:56.823695  2853 tablet_copy_source_session.cc:215] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:56.829352  3822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 710edfdfdd7e4720b77f4ec27f5f41c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:56.854890  3822 tablet_copy_client.cc:806] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:56.855720  3822 tablet_copy_client.cc:670] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:56.863562  3824 ts_tablet_manager.cc:933] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:56.864768  3822 tablet_copy_client.cc:538] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:56.874070  3824 tablet_copy_client.cc:323] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:21:56.875922  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 0a1a771f5b2742d8bd641ec0afec2cf4 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:21:56.876538  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 0a1a771f5b2742d8bd641ec0afec2cf4 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-0a1a771f5b2742d8bd641ec0afec2cf4
I20260709 12:21:56.881917  3822 tablet_bootstrap.cc:492] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:56.886124  2927 tablet_copy_source_session.cc:215] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:56.900429  3824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1a771f5b2742d8bd641ec0afec2cf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:56.929286  3824 tablet_copy_client.cc:806] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:56.930124  3824 tablet_copy_client.cc:670] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:56.936201  3822 tablet_bootstrap.cc:492] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:56.937250  3822 tablet_bootstrap.cc:492] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:56.938030  3822 ts_tablet_manager.cc:1403] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.056s	user 0.022s	sys 0.009s
I20260709 12:21:56.934505  3824 tablet_copy_client.cc:538] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:56.941437  3822 raft_consensus.cc:359] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:56.942349  3822 raft_consensus.cc:740] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:56.943044  3822 consensus_queue.cc:260] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:56.949407  3822 ts_tablet_manager.cc:1434] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 12:21:56.952353  3824 tablet_bootstrap.cc:492] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:56.961488  2853 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-710edfdfdd7e4720b77f4ec27f5f41c0 received from {username='slave'} at 127.0.0.1:38132
I20260709 12:21:56.962051  2853 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-710edfdfdd7e4720b77f4ec27f5f41c0 on tablet 710edfdfdd7e4720b77f4ec27f5f41c0 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:56.994740  3824 tablet_bootstrap.cc:492] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:56.996501  3824 tablet_bootstrap.cc:492] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:56.997304  3824 ts_tablet_manager.cc:1403] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.045s	user 0.022s	sys 0.008s
I20260709 12:21:57.000665  3824 raft_consensus.cc:359] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:57.001734  3824 raft_consensus.cc:740] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:57.002415  3824 consensus_queue.cc:260] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:57.015005  3824 ts_tablet_manager.cc:1434] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:21:57.017495  3822 ts_tablet_manager.cc:933] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:21:57.022042  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-0a1a771f5b2742d8bd641ec0afec2cf4 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:21:57.022580  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-0a1a771f5b2742d8bd641ec0afec2cf4 on tablet 0a1a771f5b2742d8bd641ec0afec2cf4 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:57.035058  3822 tablet_copy_client.cc:323] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:21:57.036991  2853 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet 2a6389f65afb4adabe1d70b6f09e99e7 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:21:57.037606  2853 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet 2a6389f65afb4adabe1d70b6f09e99e7 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-2a6389f65afb4adabe1d70b6f09e99e7
I20260709 12:21:57.044782  3824 ts_tablet_manager.cc:933] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:21:57.050071  2853 tablet_copy_source_session.cc:215] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:57.054351  3822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a6389f65afb4adabe1d70b6f09e99e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:57.074990  3824 tablet_copy_client.cc:323] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:21:57.077004  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet a7c3a03a1304452ebc48e986f5629037 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:21:57.077520  3822 tablet_copy_client.cc:806] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:57.077626  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet a7c3a03a1304452ebc48e986f5629037 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-a7c3a03a1304452ebc48e986f5629037
I20260709 12:21:57.078442  3822 tablet_copy_client.cc:670] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:57.098469  3822 tablet_copy_client.cc:538] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:57.114266  3822 tablet_bootstrap.cc:492] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:57.117677  2927 tablet_copy_source_session.cc:215] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:57.121659  3824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7c3a03a1304452ebc48e986f5629037. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:57.134485  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 133183us
I20260709 12:21:57.171073  3822 tablet_bootstrap.cc:492] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.172186  3822 tablet_bootstrap.cc:492] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:57.172920  3822 ts_tablet_manager.cc:1403] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.059s	user 0.019s	sys 0.008s
I20260709 12:21:57.174153  3824 tablet_copy_client.cc:806] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:57.174813  3824 tablet_copy_client.cc:670] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:57.176287  3822 raft_consensus.cc:359] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:57.177248  3822 raft_consensus.cc:740] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:57.177933  3822 consensus_queue.cc:260] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:57.185508  3822 ts_tablet_manager.cc:1434] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.004s
I20260709 12:21:57.193562  3824 tablet_copy_client.cc:538] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:57.200664  2853 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-2a6389f65afb4adabe1d70b6f09e99e7 received from {username='slave'} at 127.0.0.1:38132
I20260709 12:21:57.201254  2853 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-2a6389f65afb4adabe1d70b6f09e99e7 on tablet 2a6389f65afb4adabe1d70b6f09e99e7 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:57.204360  3824 tablet_bootstrap.cc:492] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:21:57.233520  3824 tablet_bootstrap.cc:492] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.234601  3824 tablet_bootstrap.cc:492] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:21:57.235306  3824 ts_tablet_manager.cc:1403] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.012s
I20260709 12:21:57.238560  3824 raft_consensus.cc:359] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:57.239444  3824 raft_consensus.cc:740] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:21:57.240114  3824 consensus_queue.cc:260] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:21:57.242897  3824 ts_tablet_manager.cc:1434] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:57.244233  3474 raft_consensus.cc:1217] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:57.248039  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-a7c3a03a1304452ebc48e986f5629037 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:21:57.248509  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-a7c3a03a1304452ebc48e986f5629037 on tablet a7c3a03a1304452ebc48e986f5629037 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:21:57.382929  3472 raft_consensus.cc:1217] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:57.438143  3472 raft_consensus.cc:1217] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:57.574824  3474 raft_consensus.cc:1217] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:57.862447  3807 raft_consensus.cc:1064] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:57.868072  3807 consensus_queue.cc:237] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:57.890662  2916 raft_consensus.cc:1275] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:57.892375  2767 raft_consensus.cc:1275] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:57.892630  3807 consensus_queue.cc:1048] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:57.899802  3807 consensus_queue.cc:1048] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:57.924451  3472 raft_consensus.cc:1275] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:57.928018  3831 raft_consensus.cc:1064] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:57.935883  3836 consensus_queue.cc:1048] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:57.990824  3831 consensus_queue.cc:237] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:58.141266  2761 raft_consensus.cc:1275] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:58.150707  3847 consensus_queue.cc:1048] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:58.167969  3476 raft_consensus.cc:1275] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:58.169636  3845 raft_consensus.cc:1064] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:58.172781  3845 consensus_queue.cc:237] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:58.175328  3789 consensus_queue.cc:1048] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:58.194605  3838 raft_consensus.cc:1064] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:21:58.205760  3838 consensus_queue.cc:237] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:58.216941  3836 raft_consensus.cc:2955] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.235678  2840 raft_consensus.cc:1275] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:58.247828  2915 raft_consensus.cc:2955] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.255813  3853 consensus_queue.cc:1048] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:58.282769  3784 raft_consensus.cc:2955] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.288558  3475 raft_consensus.cc:2955] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.290119  3476 raft_consensus.cc:2955] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.296806  3473 raft_consensus.cc:1275] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:58.299211  3807 consensus_queue.cc:1048] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:58.306824  2764 raft_consensus.cc:1275] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:58.308380  2917 raft_consensus.cc:1275] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:58.309830  2764 raft_consensus.cc:2955] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.315773  3807 consensus_queue.cc:1048] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:58.319996  2763 raft_consensus.cc:1275] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:58.321810  3844 consensus_queue.cc:1048] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:58.323262  3853 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:21:58.331527  3472 raft_consensus.cc:1275] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:58.331525  2840 raft_consensus.cc:1275] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:58.352969  3789 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:58.370200  2764 raft_consensus.cc:2955] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.384865  3838 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:21:58.420809  2663 catalog_manager.cc:5697] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:58.423151  2662 catalog_manager.cc:5697] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.441334  3844 raft_consensus.cc:2955] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.494726  2761 raft_consensus.cc:2955] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.494113  2837 raft_consensus.cc:2955] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.540962  3789 raft_consensus.cc:2955] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.543843  3474 raft_consensus.cc:2955] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.560956  2767 raft_consensus.cc:2955] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.591369  2915 raft_consensus.cc:2955] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.591918  3474 raft_consensus.cc:2955] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.601156  2837 raft_consensus.cc:2955] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.700166  2841 consensus_queue.cc:237] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:58.704869  2662 catalog_manager.cc:5697] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.706385  3475 raft_consensus.cc:1275] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:58.716349  3807 consensus_queue.cc:1048] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:58.723127  2767 raft_consensus.cc:1275] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:58.724576  3836 consensus_queue.cc:1048] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:58.719269  2662 catalog_manager.cc:5697] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.741850  3843 raft_consensus.cc:2955] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.755899  3474 raft_consensus.cc:2955] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.759292  2764 raft_consensus.cc:2955] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.777469  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 710edfdfdd7e4720b77f4ec27f5f41c0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:58.784051  2841 consensus_queue.cc:237] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:58.790450  2663 catalog_manager.cc:5697] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.795051  3474 raft_consensus.cc:1275] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:58.797502  3806 consensus_queue.cc:1048] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:58.797994  2915 consensus_queue.cc:237] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:58.798300  2913 consensus_queue.cc:237] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:21:58.804776  3475 raft_consensus.cc:1275] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:58.809096  3789 consensus_queue.cc:1048] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:58.810746  3474 raft_consensus.cc:1275] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:58.812556  3789 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:58.812022  3836 raft_consensus.cc:2955] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.813028  2767 raft_consensus.cc:1275] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:58.818394  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet 710edfdfdd7e4720b77f4ec27f5f41c0 with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47626
I20260709 12:21:58.819041  3789 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:58.823340  3475 raft_consensus.cc:2955] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.828974  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2a6389f65afb4adabe1d70b6f09e99e7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:58.826488  3789 raft_consensus.cc:2955] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.834311  2915 raft_consensus.cc:1275] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:58.838112  3470 raft_consensus.cc:2955] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.843837  3843 consensus_queue.cc:1048] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:58.851722  2915 raft_consensus.cc:2955] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.858331  2764 raft_consensus.cc:2955] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.862216  3853 raft_consensus.cc:2955] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.867185  2841 raft_consensus.cc:1275] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:58.869810  3785 consensus_queue.cc:1048] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:58.868392  2663 catalog_manager.cc:5697] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:58.874311  2837 raft_consensus.cc:2955] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.878832  3475 raft_consensus.cc:2955] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.886895  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a7c3a03a1304452ebc48e986f5629037 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:58.889696  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0a1a771f5b2742d8bd641ec0afec2cf4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:58.896409  3873 tablet_replica.cc:333] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:21:58.897840  3873 raft_consensus.cc:2243] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.898789  3873 raft_consensus.cc:2272] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.919332  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a6389f65afb4adabe1d70b6f09e99e7 with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:21:58.920989  2665 catalog_manager.cc:5697] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.925449  2666 catalog_manager.cc:5697] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:21:58.936096  3873 ts_tablet_manager.cc:1916] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:58.938920  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet a7c3a03a1304452ebc48e986f5629037 with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:21:58.943090  3874 tablet_replica.cc:333] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:21:58.960584  3875 tablet_replica.cc:333] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:21:58.962406  3875 raft_consensus.cc:2243] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.963434  3875 raft_consensus.cc:2272] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.965960  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a1a771f5b2742d8bd641ec0afec2cf4 with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:21:58.965641  3874 raft_consensus.cc:2243] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.967250  3873 ts_tablet_manager.cc:1929] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:58.967808  3873 log.cc:1199] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/710edfdfdd7e4720b77f4ec27f5f41c0
I20260709 12:21:58.967998  3875 ts_tablet_manager.cc:1916] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:58.967713  3874 raft_consensus.cc:2272] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.972095  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet 710edfdfdd7e4720b77f4ec27f5f41c0 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:58.976665  3874 ts_tablet_manager.cc:1916] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:58.988215  3875 ts_tablet_manager.cc:1929] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:58.988927  3875 log.cc:1199] T a7c3a03a1304452ebc48e986f5629037 P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/a7c3a03a1304452ebc48e986f5629037
I20260709 12:21:58.992024  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet a7c3a03a1304452ebc48e986f5629037 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:58.994668  3874 ts_tablet_manager.cc:1929] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:58.995190  3874 log.cc:1199] T 2a6389f65afb4adabe1d70b6f09e99e7 P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/2a6389f65afb4adabe1d70b6f09e99e7
I20260709 12:21:58.997028  3874 tablet_replica.cc:333] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:21:58.996903  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet 2a6389f65afb4adabe1d70b6f09e99e7 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:21:58.998132  3874 raft_consensus.cc:2243] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.998855  3874 raft_consensus.cc:2272] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:59.002382  3874 ts_tablet_manager.cc:1916] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:21:59.004467  2697 debug-util.cc:398] Leaking SignalData structure 0x7b0800038620 after lost signal to thread 1520
W20260709 12:21:59.005705  2697 debug-util.cc:398] Leaking SignalData structure 0x7b0800038d00 after lost signal to thread 2700
W20260709 12:21:59.007035  2697 debug-util.cc:398] Leaking SignalData structure 0x7b08002b3da0 after lost signal to thread 2792
W20260709 12:21:59.009747  2697 debug-util.cc:398] Leaking SignalData structure 0x7b0800256b40 after lost signal to thread 2867
W20260709 12:21:59.011025  2697 debug-util.cc:398] Leaking SignalData structure 0x7b0800320d40 after lost signal to thread 2941
W20260709 12:21:59.012315  2697 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1380 after lost signal to thread 3503
I20260709 12:21:59.027551  3874 ts_tablet_manager.cc:1929] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:59.028307  3874 log.cc:1199] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/0a1a771f5b2742d8bd641ec0afec2cf4
I20260709 12:21:59.030229  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet 0a1a771f5b2742d8bd641ec0afec2cf4 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:00.068886  2767 consensus_queue.cc:237] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
W20260709 12:22:00.080124  2734 consensus_peers.cc:597] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2594ea75a8d44503a64ea4e06916c4b3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.092656  2917 raft_consensus.cc:1275] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:00.094332  2837 raft_consensus.cc:1275] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:00.096783  3876 consensus_queue.cc:1048] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.099375  3876 consensus_queue.cc:1048] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.112272  3868 raft_consensus.cc:2955] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.114804  2836 raft_consensus.cc:2955] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.117875  2917 raft_consensus.cc:2955] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.146708  2764 consensus_queue.cc:237] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:00.147426  2663 catalog_manager.cc:5697] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.169833  2917 raft_consensus.cc:1275] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:00.170642  2836 raft_consensus.cc:1275] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:22:00.172092  2734 consensus_peers.cc:597] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 133424caefd34fa8b83015c9fe06d612. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.172942  3868 consensus_queue.cc:1048] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:00.175823  3876 consensus_queue.cc:1048] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.196730  3876 raft_consensus.cc:2955] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.210845  2913 raft_consensus.cc:2955] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.237484  2841 raft_consensus.cc:2955] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.277597  2841 consensus_queue.cc:237] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:00.284716  2664 catalog_manager.cc:5697] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
W20260709 12:22:00.326943  2809 consensus_peers.cc:597] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a2f8a42191e14f07a43032df18499a9c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.327143  2913 raft_consensus.cc:1275] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:00.328912  3806 consensus_queue.cc:1048] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.331874  2764 raft_consensus.cc:1275] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:00.333504  3806 consensus_queue.cc:1048] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.348977  3806 raft_consensus.cc:2955] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.351079  2913 raft_consensus.cc:2955] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.354447  2767 raft_consensus.cc:2955] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.392338  2663 catalog_manager.cc:5697] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.419063  2841 consensus_queue.cc:237] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
W20260709 12:22:00.491354  2809 consensus_peers.cc:597] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 51910d09518b4757be25a91c35ee2ec2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.493726  2913 raft_consensus.cc:1275] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:00.496080  3806 consensus_queue.cc:1048] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:22:00.499828  2767 raft_consensus.cc:1275] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:00.509261  3806 consensus_queue.cc:1048] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:00.519088  3893 raft_consensus.cc:2955] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.541956  2917 raft_consensus.cc:2955] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.555979  2666 catalog_manager.cc:5697] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:00.567886  2761 raft_consensus.cc:2955] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:00.603289  3897 ts_tablet_manager.cc:933] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:22:00.634845  3897 tablet_copy_client.cc:323] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.65:35625
I20260709 12:22:00.637934  2778 tablet_copy_service.cc:140] P 442c92e5267046cd92f46e6f469570a5: Received BeginTabletCopySession request for tablet 2594ea75a8d44503a64ea4e06916c4b3 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:33574)
I20260709 12:22:00.639102  2778 tablet_copy_service.cc:161] P 442c92e5267046cd92f46e6f469570a5: Beginning new tablet copy session on tablet 2594ea75a8d44503a64ea4e06916c4b3 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:33574: session id = f440bbb3ddd545819242017140636ad4-2594ea75a8d44503a64ea4e06916c4b3
I20260709 12:22:00.640903  3899 ts_tablet_manager.cc:933] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:22:00.653308  2778 tablet_copy_source_session.cc:215] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:00.659533  3897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2594ea75a8d44503a64ea4e06916c4b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:00.671094  3899 tablet_copy_client.cc:323] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.65:35625
I20260709 12:22:00.673194  2778 tablet_copy_service.cc:140] P 442c92e5267046cd92f46e6f469570a5: Received BeginTabletCopySession request for tablet 133424caefd34fa8b83015c9fe06d612 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:33574)
I20260709 12:22:00.673790  2778 tablet_copy_service.cc:161] P 442c92e5267046cd92f46e6f469570a5: Beginning new tablet copy session on tablet 133424caefd34fa8b83015c9fe06d612 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:33574: session id = f440bbb3ddd545819242017140636ad4-133424caefd34fa8b83015c9fe06d612
I20260709 12:22:00.682250  2778 tablet_copy_source_session.cc:215] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:00.686928  3897 tablet_copy_client.cc:806] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:00.697341  3899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 133424caefd34fa8b83015c9fe06d612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:00.693804  3897 tablet_copy_client.cc:670] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:00.711464  3897 tablet_copy_client.cc:538] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:00.736605  3897 tablet_bootstrap.cc:492] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:00.795023  3899 tablet_copy_client.cc:806] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:00.795733  3899 tablet_copy_client.cc:670] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:00.808423  3899 tablet_copy_client.cc:538] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:00.834370  3897 tablet_bootstrap.cc:492] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:00.835876  3903 ts_tablet_manager.cc:933] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:00.839349  3897 tablet_bootstrap.cc:492] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:00.847692  3897 ts_tablet_manager.cc:1403] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.111s	user 0.032s	sys 0.003s
I20260709 12:22:00.856233  3899 tablet_bootstrap.cc:492] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:00.870797  3903 tablet_copy_client.cc:323] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:22:00.858836  3897 raft_consensus.cc:359] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:00.873234  2853 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet a2f8a42191e14f07a43032df18499a9c from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:22:00.873651  3897 raft_consensus.cc:740] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:00.873936  2853 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet a2f8a42191e14f07a43032df18499a9c from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-a2f8a42191e14f07a43032df18499a9c
I20260709 12:22:00.874634  3897 consensus_queue.cc:260] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:00.895340  3897 ts_tablet_manager.cc:1434] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.008s
I20260709 12:22:00.897792  2778 tablet_copy_service.cc:342] P 442c92e5267046cd92f46e6f469570a5: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-2594ea75a8d44503a64ea4e06916c4b3 received from {username='slave'} at 127.0.0.1:33574
I20260709 12:22:00.898365  2778 tablet_copy_service.cc:434] P 442c92e5267046cd92f46e6f469570a5: ending tablet copy session f440bbb3ddd545819242017140636ad4-2594ea75a8d44503a64ea4e06916c4b3 on tablet 2594ea75a8d44503a64ea4e06916c4b3 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:00.948046  2853 tablet_copy_source_session.cc:215] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:00.952008  3903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f8a42191e14f07a43032df18499a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:00.976279  3903 tablet_copy_client.cc:806] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:00.977031  3903 tablet_copy_client.cc:670] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:00.984870  3903 tablet_copy_client.cc:538] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:00.991030  3899 tablet_bootstrap.cc:492] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:00.994311  3899 tablet_bootstrap.cc:492] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:00.995916  3899 ts_tablet_manager.cc:1403] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.140s	user 0.020s	sys 0.010s
I20260709 12:22:01.001556  3899 raft_consensus.cc:359] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:01.002974  3899 raft_consensus.cc:740] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:01.003816  3899 consensus_queue.cc:260] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:01.007035  3899 ts_tablet_manager.cc:1434] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 12:22:01.009359  2778 tablet_copy_service.cc:342] P 442c92e5267046cd92f46e6f469570a5: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-133424caefd34fa8b83015c9fe06d612 received from {username='slave'} at 127.0.0.1:33574
I20260709 12:22:01.009830  2778 tablet_copy_service.cc:434] P 442c92e5267046cd92f46e6f469570a5: ending tablet copy session f440bbb3ddd545819242017140636ad4-133424caefd34fa8b83015c9fe06d612 on tablet 133424caefd34fa8b83015c9fe06d612 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:01.012005  3903 tablet_bootstrap.cc:492] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:01.056473  3475 raft_consensus.cc:1217] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:01.062193  3899 ts_tablet_manager.cc:933] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:01.083328  3470 raft_consensus.cc:1217] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:01.093631  3899 tablet_copy_client.cc:323] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
W20260709 12:22:01.096416  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 94060.8us
I20260709 12:22:01.096890  2853 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet 51910d09518b4757be25a91c35ee2ec2 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:22:01.100845  2853 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet 51910d09518b4757be25a91c35ee2ec2 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-51910d09518b4757be25a91c35ee2ec2
I20260709 12:22:01.111224  2853 tablet_copy_source_session.cc:215] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:01.142670  3899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51910d09518b4757be25a91c35ee2ec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:01.166039  3903 tablet_bootstrap.cc:492] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.167227  3903 tablet_bootstrap.cc:492] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:01.168066  3903 ts_tablet_manager.cc:1403] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.156s	user 0.055s	sys 0.070s
I20260709 12:22:01.171618  3903 raft_consensus.cc:359] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:01.172554  3903 raft_consensus.cc:740] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:01.173346  3903 consensus_queue.cc:260] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:01.172034  3899 tablet_copy_client.cc:806] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:01.174892  3899 tablet_copy_client.cc:670] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:01.179185  3903 ts_tablet_manager.cc:1434] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:22:01.181787  2852 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-a2f8a42191e14f07a43032df18499a9c received from {username='slave'} at 127.0.0.1:38132
I20260709 12:22:01.182264  2852 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-a2f8a42191e14f07a43032df18499a9c on tablet a2f8a42191e14f07a43032df18499a9c with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:01.189947  3899 tablet_copy_client.cc:538] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:01.202436  3899 tablet_bootstrap.cc:492] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:01.211525  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:22:01.336351  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 65880.7us
W20260709 12:22:01.356726  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 86302.7us
I20260709 12:22:01.394037  3899 tablet_bootstrap.cc:492] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.395726  3899 tablet_bootstrap.cc:492] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:01.408978  3899 ts_tablet_manager.cc:1403] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.207s	user 0.011s	sys 0.019s
I20260709 12:22:01.413174  3899 raft_consensus.cc:359] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:01.414469  3899 raft_consensus.cc:740] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:01.415524  3899 consensus_queue.cc:260] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:01.432166  3470 raft_consensus.cc:1217] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:01.440006  3899 ts_tablet_manager.cc:1434] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:22:01.445576  2852 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-51910d09518b4757be25a91c35ee2ec2 received from {username='slave'} at 127.0.0.1:38132
I20260709 12:22:01.446177  2852 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-51910d09518b4757be25a91c35ee2ec2 on tablet 51910d09518b4757be25a91c35ee2ec2 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:01.470280  2836 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:36554
I20260709 12:22:01.471084  2836 raft_consensus.cc:606] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:01.504025  3876 raft_consensus.cc:1064] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:01.509303  3876 consensus_queue.cc:237] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:01.555655  2915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:01.556501  2915 raft_consensus.cc:606] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:01.560252  2915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:01.561020  2915 raft_consensus.cc:606] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:01.561558  3880 raft_consensus.cc:1064] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:01.564949  2915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:01.565894  2915 raft_consensus.cc:606] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:01.569361  2915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:01.569023  3880 consensus_queue.cc:237] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:01.569832  3475 raft_consensus.cc:1275] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:01.570237  2915 raft_consensus.cc:606] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:01.573761  3907 consensus_queue.cc:1048] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:01.576277  2917 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:01.577098  2917 raft_consensus.cc:606] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:01.578488  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:22:01.581068  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:01.582456  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:01.585142  2915 raft_consensus.cc:1275] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:01.603168  3916 consensus_queue.cc:1048] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:01.607250  3475 raft_consensus.cc:1217] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:01.617110  2913 raft_consensus.cc:1275] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:01.622757  2836 raft_consensus.cc:1275] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:01.632050  3922 consensus_queue.cc:1048] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:01.634930  3922 consensus_queue.cc:1048] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:22:01.641227  2734 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 70209.8us
I20260709 12:22:01.644850  3475 raft_consensus.cc:1275] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:01.678875  3819 raft_consensus.cc:993] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:01.678763  3876 consensus_queue.cc:1048] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:01.680598  3819 raft_consensus.cc:1081] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:01.692658  2838 raft_consensus.cc:1275] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:01.698170  3926 consensus_queue.cc:1048] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:01.702167  2761 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:01.702916  2761 raft_consensus.cc:493] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:01.703333  2761 raft_consensus.cc:3060] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.709786  3912 raft_consensus.cc:993] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:01.710520  3912 raft_consensus.cc:1081] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:01.715015  2764 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38956
I20260709 12:22:01.715690  2764 raft_consensus.cc:493] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:01.716064  2764 raft_consensus.cc:3060] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.717413  3913 raft_consensus.cc:993] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:01.718194  3915 raft_consensus.cc:1081] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:01.720736  3476 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:22:01.721344  3476 raft_consensus.cc:493] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:01.723130  3476 raft_consensus.cc:3060] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.723793  2761 raft_consensus.cc:515] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:01.725498  2764 raft_consensus.cc:515] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:01.735277  3928 raft_consensus.cc:2955] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:01.737207  3476 raft_consensus.cc:515] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:01.740022  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:01.740824  2837 raft_consensus.cc:3055] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:01.741004  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:01.741158  2837 raft_consensus.cc:740] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:01.741693  2767 raft_consensus.cc:3060] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.742081  2837 consensus_queue.cc:260] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:01.743038  2837 raft_consensus.cc:3060] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.728363  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:01.743839  2913 raft_consensus.cc:3055] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:01.744251  2913 raft_consensus.cc:740] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
W20260709 12:22:01.743058  2807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:01.746402  3475 raft_consensus.cc:2955] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:01.745057  2913 consensus_queue.cc:260] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:01.748721  2913 raft_consensus.cc:3060] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.730729  2836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:01.758526  2836 raft_consensus.cc:3060] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.759516  2767 raft_consensus.cc:2468] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:01.761785  3440 leader_election.cc:304] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:01.764531  2913 raft_consensus.cc:2468] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:01.768728  3905 raft_consensus.cc:2804] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:01.770406  2837 raft_consensus.cc:2468] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:01.773025  3922 raft_consensus.cc:2955] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:01.779577  2764 leader_election.cc:290] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:01.781917  3819 raft_consensus.cc:993] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:01.782605  3819 raft_consensus.cc:1081] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:01.783505  2915 raft_consensus.cc:2955] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:01.785746  3473 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:22:01.784759  3470 raft_consensus.cc:2955] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:01.786554  3473 raft_consensus.cc:493] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:01.787104  3473 raft_consensus.cc:3060] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.789590  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:01.790222  2917 raft_consensus.cc:3060] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.726269  2761 leader_election.cc:290] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:01.807047  3915 raft_consensus.cc:993] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:01.807636  3915 raft_consensus.cc:1081] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:01.810814  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9164313f06e045c3bdef4a46d365873c" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:01.811014  2764 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:01.811461  2913 raft_consensus.cc:3055] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:01.811632  2764 raft_consensus.cc:493] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:01.811801  2913 raft_consensus.cc:740] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:01.812142  2764 raft_consensus.cc:3060] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.812664  2913 consensus_queue.cc:260] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:01.813911  2913 raft_consensus.cc:3060] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.815937  3476 leader_election.cc:290] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:01.826113  2732 leader_election.cc:304] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:01.829581  3907 raft_consensus.cc:2804] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:01.830174  3907 raft_consensus.cc:697] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:01.831103  3907 consensus_queue.cc:237] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:01.823829  2836 raft_consensus.cc:2468] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:01.841789  2731 leader_election.cc:304] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:01.844748  3921 raft_consensus.cc:2804] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:01.845466  3921 raft_consensus.cc:697] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:01.850234  3921 consensus_queue.cc:237] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:01.841182  2909 raft_consensus.cc:2955] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:01.863196  2836 raft_consensus.cc:2955] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:01.872742  3905 raft_consensus.cc:697] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:01.879217  3789 raft_consensus.cc:993] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:01.879752  3789 raft_consensus.cc:1081] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:01.881834  2767 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:01.882408  2767 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:01.882819  2767 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.878943  3905 consensus_queue.cc:237] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:01.883338  2913 raft_consensus.cc:2468] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:01.871235  3473 raft_consensus.cc:515] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:01.827255  2663 catalog_manager.cc:5697] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:01.900249  2838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:01.902531  2838 raft_consensus.cc:3060] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.907002  2767 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:01.907943  2917 raft_consensus.cc:2468] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:01.909600  2767 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 4 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:01.911058  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:01.911717  2909 raft_consensus.cc:3055] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:01.912561  2909 raft_consensus.cc:740] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:01.913288  3473 leader_election.cc:290] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:01.913399  2909 consensus_queue.cc:260] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:01.918241  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:01.918855  2837 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.919689  2764 raft_consensus.cc:515] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:01.917414  2841 raft_consensus.cc:2955] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:01.929291  2909 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.934157  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ad496aaf14cbc8cf01697dceb4cf5" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:01.934894  2913 raft_consensus.cc:3055] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:01.935201  2913 raft_consensus.cc:740] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:01.936108  2913 consensus_queue.cc:260] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:01.942569  2913 raft_consensus.cc:3060] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.948625  2909 raft_consensus.cc:2468] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 4.
I20260709 12:22:01.959830  2837 raft_consensus.cc:2468] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 4.
I20260709 12:22:01.962147  2664 catalog_manager.cc:5697] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 2 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:01.969220  2764 leader_election.cc:290] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:01.971632  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:01.973385  2916 raft_consensus.cc:3055] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:01.973745  2916 raft_consensus.cc:740] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:01.974573  2916 consensus_queue.cc:260] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:01.975348  2732 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:01.975679  2916 raft_consensus.cc:3060] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.976226  3922 raft_consensus.cc:2804] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:01.976734  3922 raft_consensus.cc:697] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 4 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:01.977625  3922 consensus_queue.cc:237] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:01.980732  2913 raft_consensus.cc:2468] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:01.982038  3440 leader_election.cc:304] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:01.984354  2916 raft_consensus.cc:2468] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:01.986112  2732 leader_election.cc:304] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:01.986927  3907 raft_consensus.cc:2804] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:01.987003  3905 raft_consensus.cc:2804] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:01.987490  3907 raft_consensus.cc:697] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:01.988618  2838 raft_consensus.cc:2468] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:01.987807  3905 raft_consensus.cc:697] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:01.988447  3907 consensus_queue.cc:237] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:02.005296  3905 consensus_queue.cc:237] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:02.019816  2665 catalog_manager.cc:5697] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.042184  2836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:02.048234  2836 raft_consensus.cc:3060] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.054436  3843 raft_consensus.cc:1064] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:02.058348  3843 consensus_queue.cc:237] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:02.111099  3470 raft_consensus.cc:1275] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:02.112650  2836 raft_consensus.cc:2468] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:02.118413  3910 consensus_queue.cc:1048] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:02.130541  2664 catalog_manager.cc:5697] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 2 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:02.138255  2767 raft_consensus.cc:1275] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:02.141307  2913 raft_consensus.cc:1275] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:02.144013  3910 consensus_queue.cc:1048] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:02.147853  2663 catalog_manager.cc:5697] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:02.150601  2663 catalog_manager.cc:5697] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:02.166440  2663 catalog_manager.cc:5697] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 3 to 4, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 4 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:02.168010  3912 raft_consensus.cc:1064] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:02.168720  2663 catalog_manager.cc:5697] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:02.159855  3910 consensus_queue.cc:1048] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:02.171327  3912 consensus_queue.cc:237] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:02.199568  2764 consensus_queue.cc:237] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:02.208639  2761 consensus_queue.cc:237] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:02.217098  3473 raft_consensus.cc:1275] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:02.222843  3922 consensus_queue.cc:1048] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:02.227273  3947 log.cc:927] Time spent T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:22:02.228278  3473 raft_consensus.cc:1275] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:02.248129  3922 consensus_queue.cc:1048] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:02.254808  3928 raft_consensus.cc:2955] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.277150  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 133424caefd34fa8b83015c9fe06d612 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:02.286859  3926 raft_consensus.cc:2955] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.291122  2916 raft_consensus.cc:1275] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.298043  3921 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:02.302045  3473 raft_consensus.cc:2955] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.328217  2909 raft_consensus.cc:1275] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.327237  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2594ea75a8d44503a64ea4e06916c4b3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:02.335222  2663 catalog_manager.cc:5697] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:02.342273  2841 raft_consensus.cc:1275] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.342694  2840 raft_consensus.cc:1275] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:02.344789  2915 raft_consensus.cc:1275] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:02.347136  3470 raft_consensus.cc:2955] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.349397  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet 133424caefd34fa8b83015c9fe06d612 with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:22:02.351269  3952 tablet_replica.cc:333] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:02.353332  3952 raft_consensus.cc:2243] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:02.359045  3952 raft_consensus.cc:2272] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:02.375063  3952 ts_tablet_manager.cc:1916] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:02.378607  3928 consensus_queue.cc:1048] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:02.378875  3926 consensus_queue.cc:1048] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:02.391508  3928 consensus_queue.cc:1048] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:02.394658  3928 consensus_queue.cc:1048] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:02.394040  2666 catalog_manager.cc:5697] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:02.398583  3475 raft_consensus.cc:1275] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:02.403621  3953 consensus_queue.cc:1048] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:02.404191  2913 raft_consensus.cc:1275] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:02.410233  2913 raft_consensus.cc:1275] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:02.410251  3912 raft_consensus.cc:2955] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.411787  3905 consensus_queue.cc:1048] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:02.416682  2909 raft_consensus.cc:2955] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.433112  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet 2594ea75a8d44503a64ea4e06916c4b3 with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47626
I20260709 12:22:02.437588  3953 consensus_queue.cc:1048] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:02.443392  2913 raft_consensus.cc:2955] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.456538  2840 raft_consensus.cc:1275] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:02.461467  2764 raft_consensus.cc:1275] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:02.460716  2838 raft_consensus.cc:2955] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.468989  2840 raft_consensus.cc:1275] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.476805  2759 raft_consensus.cc:2955] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.484236  3905 consensus_queue.cc:1048] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:02.488528  2764 raft_consensus.cc:1275] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:02.517278  3921 consensus_queue.cc:1048] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:02.538698  3921 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:02.560340  2917 raft_consensus.cc:1275] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:02.591284  3921 consensus_queue.cc:1048] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:02.592621  3470 raft_consensus.cc:2955] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.611717  2840 raft_consensus.cc:1275] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:02.586570  3953 consensus_queue.cc:1048] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:22:02.628172  3952 ts_tablet_manager.cc:1929] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:02.628888  3952 log.cc:1199] T 133424caefd34fa8b83015c9fe06d612 P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/133424caefd34fa8b83015c9fe06d612
I20260709 12:22:02.624532  3905 consensus_queue.cc:1048] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:22:02.726370  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet 133424caefd34fa8b83015c9fe06d612 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
W20260709 12:22:02.755575  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 193662us
W20260709 12:22:02.780117  2809 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 215712us
I20260709 12:22:02.851089  2663 catalog_manager.cc:5697] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.885455  2842 raft_consensus.cc:1275] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.894380  2909 raft_consensus.cc:1275] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.898437  3958 tablet_replica.cc:333] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:02.899893  3958 raft_consensus.cc:2243] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:02.900818  3958 raft_consensus.cc:2272] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:02.925369  3958 ts_tablet_manager.cc:1916] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:02.928120  3926 consensus_queue.cc:1048] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:02.929915  3946 raft_consensus.cc:2955] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.954504  2917 raft_consensus.cc:2955] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:02.986286  3922 consensus_queue.cc:1048] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:02.999725  2916 raft_consensus.cc:1275] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:03.006963  3905 consensus_queue.cc:1048] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:03.125586  2764 raft_consensus.cc:2955] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:03.136632  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:03.136686  2666 catalog_manager.cc:5697] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:03.149808  3474 raft_consensus.cc:2955] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:03.195853  3958 ts_tablet_manager.cc:1929] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:03.212869  3958 log.cc:1199] T 2594ea75a8d44503a64ea4e06916c4b3 P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/2594ea75a8d44503a64ea4e06916c4b3
I20260709 12:22:03.248854  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet 2594ea75a8d44503a64ea4e06916c4b3 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:03.281970  2843 consensus_queue.cc:237] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:03.291986  2915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15476716416b4e729148408ed7846297"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:03.292709  2915 raft_consensus.cc:606] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:03.295825  3473 raft_consensus.cc:1275] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:03.296965  2840 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:36554
I20260709 12:22:03.297595  2840 raft_consensus.cc:606] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:03.298293  3979 consensus_queue.cc:1048] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:03.302695  2915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:03.303315  2915 raft_consensus.cc:606] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:03.306700  2915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:03.307266  2915 raft_consensus.cc:606] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:03.310551  2915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7bb611b2490b4682a0c5a45f2bbe7c94"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:03.311067  2915 raft_consensus.cc:606] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:03.315840  2763 raft_consensus.cc:1275] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:03.315862  2915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:03.317634  2915 raft_consensus.cc:606] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:03.319460  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:22:03.321768  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:03.322739  3953 raft_consensus.cc:2955] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:22:03.325145  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 122345us
I20260709 12:22:03.327991  3789 raft_consensus.cc:993] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:03.331516  3474 raft_consensus.cc:2955] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:03.340986  3914 raft_consensus.cc:1081] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:03.341301  3789 raft_consensus.cc:993] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:03.342362  3789 raft_consensus.cc:1081] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:03.345058  2763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15476716416b4e729148408ed7846297"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:03.345605  3475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7bb611b2490b4682a0c5a45f2bbe7c94"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:22:03.345870  2763 raft_consensus.cc:493] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:03.346181  3475 raft_consensus.cc:493] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:03.346351  2763 raft_consensus.cc:3060] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.346598  3475 raft_consensus.cc:3060] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.354753  3475 raft_consensus.cc:515] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:03.358345  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:03.360327  2840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb611b2490b4682a0c5a45f2bbe7c94" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:03.360976  2840 raft_consensus.cc:3060] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.362706  2843 consensus_queue.cc:237] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:03.367293  3912 consensus_queue.cc:1048] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:22:03.373018  2840 raft_consensus.cc:2468] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:03.379686  3439 leader_election.cc:304] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:03.378278  3475 leader_election.cc:290] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:03.384253  3962 raft_consensus.cc:2804] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:03.384859  3962 raft_consensus.cc:697] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:03.385926  3962 consensus_queue.cc:237] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:03.389009  2665 catalog_manager.cc:5697] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:03.390538  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a2f8a42191e14f07a43032df18499a9c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:03.372920  2763 raft_consensus.cc:515] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.397106  2767 raft_consensus.cc:2955] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:03.401850  2763 leader_election.cc:290] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:03.402447  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:03.403049  2841 raft_consensus.cc:3060] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.403393  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:03.403980  2915 raft_consensus.cc:3055] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:03.404309  2915 raft_consensus.cc:740] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:03.405049  2915 consensus_queue.cc:260] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.412107  2917 raft_consensus.cc:1275] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:03.414474  3979 consensus_queue.cc:1048] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:03.414983  2915 raft_consensus.cc:3060] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.417680  3475 raft_consensus.cc:1275] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:03.422194  3953 consensus_queue.cc:1048] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:03.425470  3980 raft_consensus.cc:2955] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:03.428960  2841 raft_consensus.cc:2468] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:03.439254  2909 raft_consensus.cc:2955] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:03.447409  3475 raft_consensus.cc:2955] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:03.450282  2731 leader_election.cc:304] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:03.451157  3922 raft_consensus.cc:2804] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:03.451649  3922 raft_consensus.cc:697] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:03.452536  3922 consensus_queue.cc:237] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.455207  2664 catalog_manager.cc:5697] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 2 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:03.459690  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet a2f8a42191e14f07a43032df18499a9c with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47626
I20260709 12:22:03.442404  2915 raft_consensus.cc:2468] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:03.460799  3958 tablet_replica.cc:333] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:03.469127  3958 raft_consensus.cc:2243] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:03.470625  3958 raft_consensus.cc:2272] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:03.474695  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb611b2490b4682a0c5a45f2bbe7c94" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:03.475392  2917 raft_consensus.cc:3055] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:03.475694  2917 raft_consensus.cc:740] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:03.476688  2917 consensus_queue.cc:260] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:03.475750  3958 ts_tablet_manager.cc:1916] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:03.480746  2917 raft_consensus.cc:3060] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.486588  3789 raft_consensus.cc:993] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:03.487151  3789 raft_consensus.cc:1081] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:03.498706  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 51910d09518b4757be25a91c35ee2ec2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:03.508674  2763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:03.509418  2763 raft_consensus.cc:493] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:03.509825  2763 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.512604  3915 raft_consensus.cc:993] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:03.513103  3915 raft_consensus.cc:1081] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:03.523463  3958 ts_tablet_manager.cc:1929] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:03.524010  3958 log.cc:1199] T a2f8a42191e14f07a43032df18499a9c P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/a2f8a42191e14f07a43032df18499a9c
I20260709 12:22:03.529443  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet a2f8a42191e14f07a43032df18499a9c (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:03.531587  2767 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:03.532243  2767 raft_consensus.cc:493] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:03.532600  2767 raft_consensus.cc:3060] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.505055  2664 catalog_manager.cc:5697] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:03.548471  3819 raft_consensus.cc:993] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:03.549070  3990 raft_consensus.cc:1081] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:03.550973  2764 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:03.551548  2764 raft_consensus.cc:493] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:03.551926  2764 raft_consensus.cc:3060] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.554512  2665 catalog_manager.cc:5697] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:03.560339  2764 raft_consensus.cc:515] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.575671  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:03.576383  2841 raft_consensus.cc:3060] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.578083  2767 raft_consensus.cc:515] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.579169  2763 raft_consensus.cc:515] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.581033  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:03.588443  2843 raft_consensus.cc:3060] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.588734  2841 raft_consensus.cc:2468] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:03.591585  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:03.592185  2837 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.599961  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet 51910d09518b4757be25a91c35ee2ec2 with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:22:03.600195  2731 leader_election.cc:304] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:03.601114  3926 raft_consensus.cc:2804] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:03.601675  3926 raft_consensus.cc:697] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:03.602552  3926 consensus_queue.cc:237] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.613727  2764 leader_election.cc:290] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:03.614554  2917 raft_consensus.cc:2468] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:03.615643  2837 raft_consensus.cc:2468] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:03.620595  2767 leader_election.cc:290] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:03.621260  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:03.615516  2843 raft_consensus.cc:2468] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:03.621994  2917 raft_consensus.cc:3055] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:03.622309  2841 raft_consensus.cc:1240] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Rejecting Update request from peer 7cc87046f1414b27928722690a5e388f for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:03.623334  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:03.624187  2915 raft_consensus.cc:3055] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:03.624544  2915 raft_consensus.cc:740] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:03.622345  2917 raft_consensus.cc:740] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:03.627151  2915 consensus_queue.cc:260] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.628145  2915 raft_consensus.cc:3060] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.630455  3993 tablet_replica.cc:333] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:03.631747  3993 raft_consensus.cc:2243] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:03.632694  3993 raft_consensus.cc:2272] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:03.632756  2731 leader_election.cc:304] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:03.633930  2917 consensus_queue.cc:260] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.633020  2664 catalog_manager.cc:5697] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:03.635488  2731 leader_election.cc:304] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:03.636804  2917 raft_consensus.cc:3060] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.634027  3926 raft_consensus.cc:2804] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:03.638082  3973 raft_consensus.cc:2804] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:03.638546  3926 raft_consensus.cc:697] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:03.638691  3973 raft_consensus.cc:697] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:03.639793  3926 consensus_queue.cc:237] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.640172  3973 consensus_queue.cc:237] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.644423  2917 raft_consensus.cc:2468] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:03.660390  2915 raft_consensus.cc:2468] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:03.661172  2763 leader_election.cc:290] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:03.664240  3993 ts_tablet_manager.cc:1916] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:03.670377  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:03.671061  2915 raft_consensus.cc:3055] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:03.671594  2915 raft_consensus.cc:740] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:03.672477  2915 consensus_queue.cc:260] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:03.673839  2915 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.688609  2664 catalog_manager.cc:5697] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:03.690562  2664 catalog_manager.cc:5697] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:03.697937  2915 raft_consensus.cc:2468] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:03.710453  3993 ts_tablet_manager.cc:1929] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:03.711046  3993 log.cc:1199] T 51910d09518b4757be25a91c35ee2ec2 P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/51910d09518b4757be25a91c35ee2ec2
I20260709 12:22:03.713083  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet 51910d09518b4757be25a91c35ee2ec2 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:03.793067  2843 raft_consensus.cc:1275] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:03.794678  3962 consensus_queue.cc:1048] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:03.799729  3967 raft_consensus.cc:993] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:03.800343  3967 raft_consensus.cc:1081] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:03.802553  3474 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38432
I20260709 12:22:03.803277  3474 raft_consensus.cc:493] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:03.803722  3474 raft_consensus.cc:3060] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.820598  2915 raft_consensus.cc:1275] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:03.820346  3474 raft_consensus.cc:515] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:03.823256  3905 consensus_queue.cc:1048] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:03.824173  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:03.824854  2843 raft_consensus.cc:3055] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:03.825176  2843 raft_consensus.cc:740] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:03.826090  2843 consensus_queue.cc:260] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:03.827628  2843 raft_consensus.cc:3060] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.824188  3474 leader_election.cc:290] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:03.836918  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbc0cbfcb67483f9f554976e140ba7f" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:03.837929  2909 raft_consensus.cc:3060] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.850879  2909 raft_consensus.cc:2468] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:03.852118  2843 raft_consensus.cc:2468] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:03.852149  3440 leader_election.cc:304] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:03.853451  3905 raft_consensus.cc:2804] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:03.854022  3905 raft_consensus.cc:697] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:03.854938  3905 consensus_queue.cc:237] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:03.876011  2909 raft_consensus.cc:1275] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:03.877506  3973 consensus_queue.cc:1048] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:03.894486  2666 catalog_manager.cc:5697] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 1 to 2, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 2 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:03.902606  2843 raft_consensus.cc:1275] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:03.907851  3921 consensus_queue.cc:1048] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:04.056089  2841 raft_consensus.cc:1275] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:04.057706  3926 consensus_queue.cc:1048] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:04.077685  2915 raft_consensus.cc:1275] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:04.080359  3921 consensus_queue.cc:1048] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:04.110608  2841 raft_consensus.cc:1275] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:04.112371  3921 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:04.143119  2843 raft_consensus.cc:1275] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:04.145946  2909 raft_consensus.cc:1275] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:04.146185  2915 raft_consensus.cc:1275] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:04.152998  3922 consensus_queue.cc:1048] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:04.158516  3924 consensus_queue.cc:1048] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:04.160003  3924 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:04.263619  2841 raft_consensus.cc:1275] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:04.265182  3905 consensus_queue.cc:1048] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:04.278208  2909 raft_consensus.cc:1275] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:04.290206  3962 consensus_queue.cc:1048] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:04.692708  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:04.695626  2763 consensus_queue.cc:237] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
W20260709 12:22:04.712647  2734 consensus_peers.cc:597] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fe7ee185c0114c079f53469226dee8bf. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:04.717368  2909 raft_consensus.cc:1275] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:04.719197  2841 raft_consensus.cc:1275] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:04.720927  3973 consensus_queue.cc:1048] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:04.722028  3926 consensus_queue.cc:1048] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:04.761088  3926 raft_consensus.cc:2955] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:04.768303  2843 raft_consensus.cc:2955] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:04.810787  2917 raft_consensus.cc:2955] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:04.808277  2666 catalog_manager.cc:5697] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:04.812323  2843 consensus_queue.cc:237] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:04.911129  2917 raft_consensus.cc:1275] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:04.911814  2763 raft_consensus.cc:1275] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:04.912233  2767 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "133424caefd34fa8b83015c9fe06d612"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:04.912884  2767 raft_consensus.cc:606] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:04.914582  3967 consensus_queue.cc:1048] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:04.916968  3967 consensus_queue.cc:1048] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
W20260709 12:22:04.928397  2809 consensus_peers.cc:597] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d436a6cfd22640bea76c0256f1d01684. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:04.994153  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 97578.6us
W20260709 12:22:05.075845  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 145744us
I20260709 12:22:05.099166  2909 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:05.099839  2909 raft_consensus.cc:606] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:05.122190  2843 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:36554
I20260709 12:22:05.122921  2843 raft_consensus.cc:606] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:05.127635  4019 raft_consensus.cc:993] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:05.128360  3973 raft_consensus.cc:1081] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:05.147269  3475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "133424caefd34fa8b83015c9fe06d612"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38416
I20260709 12:22:05.148121  3475 raft_consensus.cc:493] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.148473  3475 raft_consensus.cc:3060] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.150614  4027 raft_consensus.cc:993] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:05.151125  4027 raft_consensus.cc:1081] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:05.151978  2763 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:05.152648  2763 raft_consensus.cc:606] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:05.157946  3475 raft_consensus.cc:515] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:05.160346  4042 ts_tablet_manager.cc:933] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:22:05.164606  2767 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:05.181977  2763 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:05.182381  2767 raft_consensus.cc:493] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.182685  2763 raft_consensus.cc:606] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Received request to transfer leadership to TS 7cc87046f1414b27928722690a5e388f
I20260709 12:22:05.182781  2767 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.169975  3475 leader_election.cc:290] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:05.188565  2763 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:05.189172  2763 raft_consensus.cc:606] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Received request to transfer leadership to TS 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:05.190689  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:22:05.192628  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:05.169299  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133424caefd34fa8b83015c9fe06d612" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:05.193420  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:05.193706  2917 raft_consensus.cc:3060] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.174271  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133424caefd34fa8b83015c9fe06d612" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:05.196225  2764 raft_consensus.cc:3055] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:05.196578  2764 raft_consensus.cc:740] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:05.197747  2764 consensus_queue.cc:260] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:05.199146  2764 raft_consensus.cc:3060] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.201077  2767 raft_consensus.cc:515] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.206606  4040 raft_consensus.cc:2955] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:05.220129  2767 leader_election.cc:290] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:05.226042  4042 tablet_copy_client.cc:323] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.65:35625
I20260709 12:22:05.229905  2909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:05.230584  2909 raft_consensus.cc:3055] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:05.230883  2909 raft_consensus.cc:740] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:05.231673  2909 consensus_queue.cc:260] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.233853  2909 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.236496  2778 tablet_copy_service.cc:140] P 442c92e5267046cd92f46e6f469570a5: Received BeginTabletCopySession request for tablet fe7ee185c0114c079f53469226dee8bf from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:33574)
I20260709 12:22:05.237136  2778 tablet_copy_service.cc:161] P 442c92e5267046cd92f46e6f469570a5: Beginning new tablet copy session on tablet fe7ee185c0114c079f53469226dee8bf from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:33574: session id = f440bbb3ddd545819242017140636ad4-fe7ee185c0114c079f53469226dee8bf
I20260709 12:22:05.239769  3973 raft_consensus.cc:993] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:05.240314  4019 raft_consensus.cc:1081] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Signalling peer 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:05.208067  2917 raft_consensus.cc:2468] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:05.242002  2915 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:47638
I20260709 12:22:05.242561  2915 raft_consensus.cc:493] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.242877  2915 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.242905  3440 leader_election.cc:304] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:05.243922  3905 raft_consensus.cc:2804] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:05.244277  2909 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:05.244405  3905 raft_consensus.cc:697] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:05.245411  3905 consensus_queue.cc:237] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:05.254616  2778 tablet_copy_source_session.cc:215] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:05.256960  2767 raft_consensus.cc:2955] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:05.259330  2666 catalog_manager.cc:5697] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:05.263149  2664 catalog_manager.cc:5697] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 1 to 2, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 2 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:05.265997  2732 leader_election.cc:304] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:05.267235  3973 raft_consensus.cc:2804] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:05.267766  3973 raft_consensus.cc:697] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:05.268707  3973 consensus_queue.cc:237] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.273814  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:05.274489  2841 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.266664  2916 raft_consensus.cc:2955] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:05.287863  2841 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:05.290866  2915 raft_consensus.cc:515] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.305546  2843 consensus_queue.cc:237] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:05.311446  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:05.312289  2767 raft_consensus.cc:3055] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:05.313498  2767 raft_consensus.cc:740] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:05.315017  2767 consensus_queue.cc:260] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.313105  2915 leader_election.cc:290] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:05.317842  2767 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:05.320206  2809 consensus_peers.cc:597] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 659c2c3a61024d07a2296bc83d87a52d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:05.324442  4042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe7ee185c0114c079f53469226dee8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:05.327280  2767 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:05.341234  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:05.291711  2764 raft_consensus.cc:2468] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:05.342600  2881 leader_election.cc:304] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:05.343847  4030 raft_consensus.cc:2804] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:05.344340  4030 raft_consensus.cc:697] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:05.345264  4030 consensus_queue.cc:237] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.349491  4042 tablet_copy_client.cc:806] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:05.350258  4042 tablet_copy_client.cc:670] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:05.348263  2917 raft_consensus.cc:1275] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:05.352152  2759 raft_consensus.cc:1275] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:05.357124  4042 tablet_copy_client.cc:538] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:05.360544  4035 consensus_queue.cc:1048] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:05.365588  3967 consensus_queue.cc:1048] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:05.367100  4049 ts_tablet_manager.cc:933] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:05.371104  4042 tablet_bootstrap.cc:492] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:05.378649  4049 tablet_copy_client.cc:323] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:22:05.381858  2852 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet d436a6cfd22640bea76c0256f1d01684 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:22:05.381248  2666 catalog_manager.cc:5697] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 1 to 2, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 2 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:05.382381  2852 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet d436a6cfd22640bea76c0256f1d01684 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-d436a6cfd22640bea76c0256f1d01684
I20260709 12:22:05.398612  2852 tablet_copy_source_session.cc:215] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:05.342021  2841 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.400082  4041 raft_consensus.cc:993] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:05.400668  4035 raft_consensus.cc:1081] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:05.402768  4049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d436a6cfd22640bea76c0256f1d01684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:05.406726  4042 tablet_bootstrap.cc:492] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:05.407788  4042 tablet_bootstrap.cc:492] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:05.408533  4042 ts_tablet_manager.cc:1403] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.038s	user 0.027s	sys 0.004s
I20260709 12:22:05.411765  4042 raft_consensus.cc:359] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:05.412727  4042 raft_consensus.cc:740] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:05.413414  4042 consensus_queue.cc:260] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:05.422441  4049 tablet_copy_client.cc:806] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:05.423127  4049 tablet_copy_client.cc:670] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:05.423180  4035 raft_consensus.cc:2955] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:05.425698  4042 ts_tablet_manager.cc:1434] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 12:22:05.430647  4049 tablet_copy_client.cc:538] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:05.437103  2778 tablet_copy_service.cc:342] P 442c92e5267046cd92f46e6f469570a5: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-fe7ee185c0114c079f53469226dee8bf received from {username='slave'} at 127.0.0.1:33574
I20260709 12:22:05.430449  2841 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:05.437724  2778 tablet_copy_service.cc:434] P 442c92e5267046cd92f46e6f469570a5: ending tablet copy session f440bbb3ddd545819242017140636ad4-fe7ee185c0114c079f53469226dee8bf on tablet fe7ee185c0114c079f53469226dee8bf with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:05.438282  2764 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38956
I20260709 12:22:05.438898  2764 raft_consensus.cc:493] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.439271  2764 raft_consensus.cc:3060] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.446345  2759 raft_consensus.cc:2955] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:05.448635  2666 catalog_manager.cc:5697] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:05.452991  2714 auto_rebalancer.cc:629] Failed to schedule move for tablet 22aad5981ec94ff2a74fdf45111c0692: Illegal state: Replica 7cc87046f1414b27928722690a5e388f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } }
I20260709 12:22:05.453236  2917 raft_consensus.cc:2955] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:05.460346  2764 raft_consensus.cc:515] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.463049  2664 catalog_manager.cc:5697] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:05.465477  4049 tablet_bootstrap.cc:492] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:05.474913  2764 leader_election.cc:290] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:05.475520  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:05.479770  2916 raft_consensus.cc:3060] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.482607  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:05.483305  2841 raft_consensus.cc:3055] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:05.483603  2841 raft_consensus.cc:740] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:05.484372  2841 consensus_queue.cc:260] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.485692  2841 raft_consensus.cc:3060] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.525048  2916 raft_consensus.cc:2468] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:05.528874  4019 raft_consensus.cc:993] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:05.532604  4019 raft_consensus.cc:1081] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:05.534830  2841 raft_consensus.cc:2468] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:05.536703  3474 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38416
I20260709 12:22:05.538295  2732 leader_election.cc:304] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:05.539384  4039 raft_consensus.cc:2804] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:05.539947  4039 raft_consensus.cc:697] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:05.537336  3474 raft_consensus.cc:493] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.547472  4049 tablet_bootstrap.cc:492] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:05.548758  4049 tablet_bootstrap.cc:492] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:05.549612  4049 ts_tablet_manager.cc:1403] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.084s	user 0.038s	sys 0.020s
I20260709 12:22:05.543260  3474 raft_consensus.cc:3060] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:05.553107  4049 raft_consensus.cc:359] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:05.554371  4049 raft_consensus.cc:740] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:05.555141  4049 consensus_queue.cc:260] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:05.559460  4049 ts_tablet_manager.cc:1434] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:05.541033  4039 consensus_queue.cc:237] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.564808  2852 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-d436a6cfd22640bea76c0256f1d01684 received from {username='slave'} at 127.0.0.1:38132
I20260709 12:22:05.565317  2852 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-d436a6cfd22640bea76c0256f1d01684 on tablet d436a6cfd22640bea76c0256f1d01684 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:05.579495  3973 raft_consensus.cc:993] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:05.580092  3973 raft_consensus.cc:1081] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Signalling peer 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:05.583880  2841 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:42536
I20260709 12:22:05.584488  2841 raft_consensus.cc:493] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.584837  2841 raft_consensus.cc:3060] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:05.592653  2841 raft_consensus.cc:515] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.594750  2841 leader_election.cc:290] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:05.598694  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:05.599462  2917 raft_consensus.cc:3060] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:05.598749  2666 catalog_manager.cc:5697] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:05.587783  3474 raft_consensus.cc:515] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:05.605809  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627b3359300e44fc9d2b83ed88272a9f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:05.606549  2759 raft_consensus.cc:3055] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:05.606876  2759 raft_consensus.cc:740] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:05.607460  3474 leader_election.cc:290] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:05.607674  2759 consensus_queue.cc:260] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.609086  2759 raft_consensus.cc:3060] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:05.609907  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:05.610504  2764 raft_consensus.cc:3055] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:05.610821  2764 raft_consensus.cc:740] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:05.611615  2764 consensus_queue.cc:260] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:05.612911  2764 raft_consensus.cc:3060] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:05.625952  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d5e0cde57459bbad31d06eb64d442" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:05.626956  2841 raft_consensus.cc:3060] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:05.629667  2917 raft_consensus.cc:2468] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 3.
I20260709 12:22:05.630883  2807 leader_election.cc:304] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:05.631665  3979 raft_consensus.cc:2804] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:05.632174  3979 raft_consensus.cc:697] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:05.633054  3979 consensus_queue.cc:237] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:05.639075  2841 raft_consensus.cc:2468] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 3.
I20260709 12:22:05.639904  2759 raft_consensus.cc:2468] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 3.
I20260709 12:22:05.640791  3439 leader_election.cc:304] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:05.641736  4046 raft_consensus.cc:2804] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:05.642233  4046 raft_consensus.cc:697] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 3 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:05.639482  2764 raft_consensus.cc:2468] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 3.
I20260709 12:22:05.643226  4046 consensus_queue.cc:237] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:05.662731  2666 catalog_manager.cc:5697] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:05.671008  3475 raft_consensus.cc:1217] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:05.680169  2841 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:05.683574  4019 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:05.683806  2664 catalog_manager.cc:5697] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 3 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:05.745926  4049 ts_tablet_manager.cc:933] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:05.773339  2916 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:05.782789  4049 tablet_copy_client.cc:323] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:22:05.787613  2852 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet 659c2c3a61024d07a2296bc83d87a52d from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:22:05.788241  2852 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet 659c2c3a61024d07a2296bc83d87a52d from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-659c2c3a61024d07a2296bc83d87a52d
I20260709 12:22:05.797291  2837 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:05.799479  4022 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:05.780752  4019 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:22:05.810583  2764 raft_consensus.cc:1275] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:05.812752  4046 consensus_queue.cc:1048] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:05.814424  2852 tablet_copy_source_session.cc:215] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:05.827036  3474 raft_consensus.cc:1217] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:05.839820  4049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 659c2c3a61024d07a2296bc83d87a52d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:05.848564  2764 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:05.850569  4027 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:22:05.879904  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 67538us
I20260709 12:22:05.882979  2915 raft_consensus.cc:1275] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:05.922472  3963 consensus_queue.cc:1048] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:05.938587  4049 tablet_copy_client.cc:806] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:05.940270  4049 tablet_copy_client.cc:670] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:05.948019  4049 tablet_copy_client.cc:538] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:05.960959  4049 tablet_bootstrap.cc:492] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:05.998874  4049 tablet_bootstrap.cc:492] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:05.999924  4049 tablet_bootstrap.cc:492] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:06.000676  4049 ts_tablet_manager.cc:1403] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.009s
I20260709 12:22:06.004204  4049 raft_consensus.cc:359] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:06.005122  4049 raft_consensus.cc:740] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:06.005867  4049 consensus_queue.cc:260] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:06.033520  2759 raft_consensus.cc:1275] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:06.035246  3963 consensus_queue.cc:1048] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:06.052119  2841 raft_consensus.cc:1275] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:06.053938  4046 consensus_queue.cc:1048] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:06.059083  4049 ts_tablet_manager.cc:1434] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.058s	user 0.000s	sys 0.009s
I20260709 12:22:06.061208  2852 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-659c2c3a61024d07a2296bc83d87a52d received from {username='slave'} at 127.0.0.1:38132
I20260709 12:22:06.061748  2852 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-659c2c3a61024d07a2296bc83d87a52d on tablet 659c2c3a61024d07a2296bc83d87a52d with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:06.069345  4036 raft_consensus.cc:1064] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:06.072292  4036 consensus_queue.cc:237] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:06.115424  3474 raft_consensus.cc:1275] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:06.117316  4039 consensus_queue.cc:1048] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:06.120790  2917 raft_consensus.cc:1275] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:06.124981  4019 consensus_queue.cc:1048] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:06.128710  2841 raft_consensus.cc:1275] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:06.130412  3973 consensus_queue.cc:1048] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:06.158023  2915 raft_consensus.cc:1275] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:06.162096  3973 consensus_queue.cc:1048] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:06.191608  2843 raft_consensus.cc:1275] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:06.193392  3973 consensus_queue.cc:1048] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:06.200654  2763 raft_consensus.cc:1275] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:06.209321  4040 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:06.263314  4019 raft_consensus.cc:2955] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.273640  2915 raft_consensus.cc:1275] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:06.276885  3979 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:06.280479  3474 raft_consensus.cc:1217] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:06.359946  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:06.396394  3474 raft_consensus.cc:2955] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.405865  4026 raft_consensus.cc:1064] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:06.405973  2843 raft_consensus.cc:2955] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.409037  4026 consensus_queue.cc:237] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:06.436997  3474 raft_consensus.cc:1275] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:06.439365  4035 consensus_queue.cc:1048] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:06.463796  2915 raft_consensus.cc:2955] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.513795  2767 raft_consensus.cc:1275] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:06.513904  2915 raft_consensus.cc:1275] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:06.610759  4040 consensus_queue.cc:1048] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:06.636677  2767 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2594ea75a8d44503a64ea4e06916c4b3"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:06.637342  2767 raft_consensus.cc:606] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:06.635792  2664 catalog_manager.cc:5697] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:06.645776  2837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41f5e1f4763f4d978619c2696e7aaa71"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
new_leader_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:36554
I20260709 12:22:06.646469  2837 raft_consensus.cc:606] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Received request to transfer leadership to TS 7cc87046f1414b27928722690a5e388f
I20260709 12:22:06.665217  4035 consensus_queue.cc:1048] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:06.674909  2915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:06.675846  2915 raft_consensus.cc:606] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:06.688230  2764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:06.688917  2764 raft_consensus.cc:606] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:06.693850  2917 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa220c2195724c24b522d44636d98970"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:06.694487  2917 raft_consensus.cc:606] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:06.695854  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:06.697909  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:06.698750  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:06.700201  4078 raft_consensus.cc:993] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06: : Instructing follower 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:06.700729  4078 raft_consensus.cc:1081] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Signalling peer 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:06.707887  2917 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41f5e1f4763f4d978619c2696e7aaa71"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:47632
I20260709 12:22:06.708544  2917 raft_consensus.cc:493] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:06.708928  2917 raft_consensus.cc:3060] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:06.716676  2917 raft_consensus.cc:515] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:06.727880  2917 leader_election.cc:290] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:06.731104  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41f5e1f4763f4d978619c2696e7aaa71" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:06.732002  2767 raft_consensus.cc:3060] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:06.740886  2767 raft_consensus.cc:2468] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:06.742959  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41f5e1f4763f4d978619c2696e7aaa71" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:06.744596  2841 raft_consensus.cc:3055] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:06.745067  2841 raft_consensus.cc:740] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:06.744509  4078 raft_consensus.cc:2955] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.745997  2841 consensus_queue.cc:260] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:06.747426  2841 raft_consensus.cc:3060] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:06.755292  2841 raft_consensus.cc:2468] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:06.758065  2881 leader_election.cc:304] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:06.763751  4027 raft_consensus.cc:2804] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:06.764359  4027 raft_consensus.cc:697] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:06.763928  2767 raft_consensus.cc:2955] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.765241  4027 consensus_queue.cc:237] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:06.770834  3473 raft_consensus.cc:2955] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.786087  4026 raft_consensus.cc:1064] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:06.789669  4026 consensus_queue.cc:237] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:06.814703  4030 raft_consensus.cc:993] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:06.815335  4030 raft_consensus.cc:1081] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:06.817374  2767 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa220c2195724c24b522d44636d98970"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:06.817986  2767 raft_consensus.cc:493] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:06.818413  2767 raft_consensus.cc:3060] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:06.826843  2767 raft_consensus.cc:515] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:06.828912  2767 leader_election.cc:290] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:06.829272  2917 raft_consensus.cc:1275] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:06.833384  2767 raft_consensus.cc:1275] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:06.833765  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:06.834383  2917 raft_consensus.cc:3055] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:06.834699  2917 raft_consensus.cc:740] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:06.835464  2917 consensus_queue.cc:260] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:06.836769  2917 raft_consensus.cc:3060] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:06.837849  4078 consensus_queue.cc:1048] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:06.838547  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:06.839155  2841 raft_consensus.cc:3060] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:06.840052  4078 consensus_queue.cc:1048] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:06.854385  3473 raft_consensus.cc:1275] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:06.866853  4078 consensus_queue.cc:1048] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:06.876047  2841 raft_consensus.cc:2468] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:06.877017  2916 raft_consensus.cc:2955] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.878958  2731 leader_election.cc:304] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:06.880168  4019 raft_consensus.cc:2804] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:06.880656  4019 raft_consensus.cc:697] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:06.881505  4019 consensus_queue.cc:237] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:06.882308  2917 raft_consensus.cc:2468] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:06.890734  3979 raft_consensus.cc:2955] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.894382  2915 raft_consensus.cc:2955] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.901667  3474 raft_consensus.cc:2955] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.916330  2767 raft_consensus.cc:2955] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.956787  4084 raft_consensus.cc:993] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:06.957325  4084 raft_consensus.cc:1081] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:06.965327  2665 catalog_manager.cc:5697] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.967202  2665 catalog_manager.cc:5697] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:06.969772  3473 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2594ea75a8d44503a64ea4e06916c4b3"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38416
I20260709 12:22:06.970458  3473 raft_consensus.cc:493] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:06.970834  3473 raft_consensus.cc:3060] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:06.994957  3473 raft_consensus.cc:515] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:06.994493  2666 catalog_manager.cc:5697] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:06.999342  3473 leader_election.cc:290] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:06.999708  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2594ea75a8d44503a64ea4e06916c4b3" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:07.000349  2841 raft_consensus.cc:3060] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.001925  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2594ea75a8d44503a64ea4e06916c4b3" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:07.002552  2767 raft_consensus.cc:3055] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:07.002835  2767 raft_consensus.cc:740] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:07.003587  2767 consensus_queue.cc:260] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:07.004846  2767 raft_consensus.cc:3060] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.008147  2841 raft_consensus.cc:2468] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:07.009630  3439 leader_election.cc:304] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:07.010483  4046 raft_consensus.cc:2804] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:07.012732  2767 raft_consensus.cc:2468] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:07.013267  4046 raft_consensus.cc:697] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:07.012892  2663 catalog_manager.cc:5697] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 1 to 2, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 2 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:07.014251  4046 consensus_queue.cc:237] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:07.035334  2665 catalog_manager.cc:5697] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 1 to 2, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 2 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:07.043942  2767 consensus_queue.cc:237] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:07.047719  2843 consensus_queue.cc:237] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:07.055205  2917 raft_consensus.cc:1275] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:07.057500  3473 raft_consensus.cc:1275] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:07.058223  4036 raft_consensus.cc:993] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:07.058750  4036 raft_consensus.cc:1081] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:07.061196  3473 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38416
I20260709 12:22:07.062407  3473 raft_consensus.cc:493] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:07.062898  3473 raft_consensus.cc:3060] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.061036  3973 consensus_queue.cc:1048] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:22:07.066812  4084 consensus_queue.cc:1048] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:07.067564  4030 raft_consensus.cc:993] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:07.068094  4030 raft_consensus.cc:1081] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:07.070318  3474 raft_consensus.cc:1275] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:07.072082  4026 consensus_queue.cc:1048] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:07.072522  3473 raft_consensus.cc:515] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:07.073674  2841 consensus_queue.cc:237] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:07.076546  2761 raft_consensus.cc:1275] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:07.076797  2764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:07.077171  3473 leader_election.cc:290] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:07.077478  2764 raft_consensus.cc:3055] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:07.077874  2764 raft_consensus.cc:740] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:07.078845  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:07.078660  2764 consensus_queue.cc:260] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:07.079484  2917 raft_consensus.cc:3060] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.079849  2764 raft_consensus.cc:3060] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.083124  2915 raft_consensus.cc:1275] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:07.084983  3474 raft_consensus.cc:1275] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:07.084497  3979 consensus_queue.cc:1048] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:07.087417  2917 raft_consensus.cc:2468] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:07.088609  3440 leader_election.cc:304] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:07.089432  4046 raft_consensus.cc:2804] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:07.089973  4046 raft_consensus.cc:697] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:07.090902  4046 consensus_queue.cc:237] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:07.091481  2766 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:07.092067  2766 raft_consensus.cc:493] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:07.092409  2766 raft_consensus.cc:3060] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.097949  4036 raft_consensus.cc:2955] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:07.084913  4026 consensus_queue.cc:1048] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:22:07.106793  4078 raft_consensus.cc:2955] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:07.109194  2766 raft_consensus.cc:515] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:07.109900  4040 consensus_queue.cc:1048] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:07.115604  2917 raft_consensus.cc:2955] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:07.119038  2764 raft_consensus.cc:2468] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:07.121357  3474 raft_consensus.cc:2955] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:07.127589  2766 leader_election.cc:290] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:07.132292  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:07.133237  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fe7ee185c0114c079f53469226dee8bf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:07.133312  2915 raft_consensus.cc:3055] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:07.133934  2915 raft_consensus.cc:740] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:07.135111  2915 consensus_queue.cc:260] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:07.136595  2915 raft_consensus.cc:3060] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.136106  3473 raft_consensus.cc:2955] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:07.126524  4041 raft_consensus.cc:2955] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:07.139088  2665 catalog_manager.cc:5697] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:07.146888  2915 raft_consensus.cc:2468] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:07.147524  2764 raft_consensus.cc:2955] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:07.148636  2732 leader_election.cc:304] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:07.153776  4036 raft_consensus.cc:2804] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:07.154347  4036 raft_consensus.cc:697] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:07.155153  4036 consensus_queue.cc:237] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:07.169004  3475 raft_consensus.cc:2955] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:07.171175  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:07.171820  2841 raft_consensus.cc:3060] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.175346  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet fe7ee185c0114c079f53469226dee8bf with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:22:07.200789  2664 catalog_manager.cc:5697] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:07.212666  2764 raft_consensus.cc:1275] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:07.220314  3819 consensus_queue.cc:1048] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:07.215444  2843 raft_consensus.cc:1275] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:07.228332  4022 consensus_queue.cc:1048] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:07.253919  2666 catalog_manager.cc:5697] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 1 to 2, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 2 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:07.279327  2841 raft_consensus.cc:2468] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:07.284360  4104 tablet_replica.cc:333] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:07.298008  2915 raft_consensus.cc:2955] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:07.324509  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 659c2c3a61024d07a2296bc83d87a52d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:07.325359  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d436a6cfd22640bea76c0256f1d01684 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:07.336793  2663 catalog_manager.cc:5697] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:07.338790  2663 catalog_manager.cc:5697] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:07.354573  4104 raft_consensus.cc:2243] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:07.355485  4104 raft_consensus.cc:2272] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:07.358873  4104 ts_tablet_manager.cc:1916] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:07.368721  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet d436a6cfd22640bea76c0256f1d01684 with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:22:07.373847  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet 659c2c3a61024d07a2296bc83d87a52d with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47626
I20260709 12:22:07.386993  4110 tablet_replica.cc:333] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:07.391433  4110 raft_consensus.cc:2243] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:07.392511  4110 raft_consensus.cc:2272] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:07.399919  4112 tablet_replica.cc:333] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:07.401219  4112 raft_consensus.cc:2243] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:07.402408  4112 raft_consensus.cc:2272] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:07.399857  4110 ts_tablet_manager.cc:1916] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:07.405007  4104 ts_tablet_manager.cc:1929] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:07.405570  4104 log.cc:1199] T fe7ee185c0114c079f53469226dee8bf P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/fe7ee185c0114c079f53469226dee8bf
I20260709 12:22:07.406623  4112 ts_tablet_manager.cc:1916] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:07.464741  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet fe7ee185c0114c079f53469226dee8bf (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:07.466733  2843 raft_consensus.cc:1275] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:07.490056  2916 raft_consensus.cc:1275] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:07.490624  4064 consensus_queue.cc:1048] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:07.506001  4112 ts_tablet_manager.cc:1929] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:07.506445  2843 raft_consensus.cc:1275] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:07.508450  4112 log.cc:1199] T 659c2c3a61024d07a2296bc83d87a52d P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/659c2c3a61024d07a2296bc83d87a52d
I20260709 12:22:07.514173  4084 consensus_queue.cc:1048] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:07.518275  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet 659c2c3a61024d07a2296bc83d87a52d (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:07.527882  4019 consensus_queue.cc:1048] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:07.533571  2764 raft_consensus.cc:1275] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:07.567747  2767 raft_consensus.cc:1275] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:07.576591  2916 raft_consensus.cc:1275] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:07.586189  4102 consensus_queue.cc:1048] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:22:07.590500  4064 consensus_queue.cc:1048] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:22:07.595531  4110 ts_tablet_manager.cc:1929] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:07.599745  4110 log.cc:1199] T d436a6cfd22640bea76c0256f1d01684 P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/d436a6cfd22640bea76c0256f1d01684
I20260709 12:22:07.614539  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet d436a6cfd22640bea76c0256f1d01684 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:07.624490  2916 raft_consensus.cc:1275] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:07.628851  4046 consensus_queue.cc:1048] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:07.633525  4113 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:22:07.647228  2841 raft_consensus.cc:1275] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:07.656729  4036 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
W20260709 12:22:07.890594  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 104100us
I20260709 12:22:07.875831  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:07.971513  2916 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:07.972613  2916 raft_consensus.cc:606] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:07.976202  2843 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
new_leader_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:36554
I20260709 12:22:07.976854  2843 raft_consensus.cc:606] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Received request to transfer leadership to TS 7cc87046f1414b27928722690a5e388f
I20260709 12:22:07.982280  2767 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:07.984349  3819 raft_consensus.cc:993] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:07.984807  2767 raft_consensus.cc:606] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:07.985085  4022 raft_consensus.cc:1081] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:07.987460  2767 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:07.988617  2916 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:07.988947  2767 raft_consensus.cc:493] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:07.989233  2916 raft_consensus.cc:606] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:07.989411  2767 raft_consensus.cc:3060] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.992226  2766 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:07.992750  2766 raft_consensus.cc:606] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Received request to transfer leadership to TS 7cc87046f1414b27928722690a5e388f
I20260709 12:22:07.994091  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:07.996179  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:07.997268  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:07.998780  2767 raft_consensus.cc:515] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:08.001911  2767 leader_election.cc:290] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:08.002329  3475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:08.003198  3475 raft_consensus.cc:3060] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.001879  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:08.003754  2916 raft_consensus.cc:3055] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:08.003945  2916 raft_consensus.cc:740] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:08.004588  2916 consensus_queue.cc:260] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:08.005409  2916 raft_consensus.cc:3060] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.012887  3475 raft_consensus.cc:2468] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:08.014111  2916 raft_consensus.cc:2468] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:08.015014  2734 leader_election.cc:304] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:08.015880  4113 raft_consensus.cc:2804] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:08.016362  4113 raft_consensus.cc:697] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:08.017290  4113 consensus_queue.cc:237] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:08.030753  2665 catalog_manager.cc:5697] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:08.319788  4041 raft_consensus.cc:993] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06: : Instructing follower 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:08.320382  4026 raft_consensus.cc:1081] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Signalling peer 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:08.323573  2913 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:47632
I20260709 12:22:08.324186  2913 raft_consensus.cc:493] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.324568  2913 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.331386  2913 raft_consensus.cc:515] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:08.334045  2913 leader_election.cc:290] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:08.334123  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:08.334867  2767 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.335196  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:08.335836  2843 raft_consensus.cc:3055] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:08.336203  2843 raft_consensus.cc:740] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:08.337050  2843 consensus_queue.cc:260] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:08.338208  2843 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.342569  2767 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:08.344096  2881 leader_election.cc:304] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:08.345046  3819 raft_consensus.cc:2804] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:08.345506  2843 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:08.345609  3819 raft_consensus.cc:697] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:08.346799  3819 consensus_queue.cc:237] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:08.361500  2665 catalog_manager.cc:5697] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 1 to 2, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 2 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:08.387579  3973 raft_consensus.cc:993] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:08.388139  3973 raft_consensus.cc:1081] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:08.390041  3473 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38416
I20260709 12:22:08.390600  3473 raft_consensus.cc:493] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.390996  3473 raft_consensus.cc:3060] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.399292  3473 raft_consensus.cc:515] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:08.402956  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:08.403594  2767 raft_consensus.cc:3055] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:08.403929  2767 raft_consensus.cc:740] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:08.404881  2767 consensus_queue.cc:260] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:08.405987  2767 raft_consensus.cc:3060] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.412477  3473 leader_election.cc:290] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:08.413772  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:08.414541  2843 raft_consensus.cc:3060] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.414685  2767 raft_consensus.cc:2468] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:08.416050  3440 leader_election.cc:304] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:08.417023  4036 raft_consensus.cc:993] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:08.417567  4036 raft_consensus.cc:1081] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Signalling peer 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:08.421224  2843 raft_consensus.cc:2468] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:08.422220  4102 raft_consensus.cc:2804] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:08.422784  4102 raft_consensus.cc:697] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:08.423666  4102 consensus_queue.cc:237] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:08.426288  2913 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:47638
I20260709 12:22:08.426885  2913 raft_consensus.cc:493] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.427238  2913 raft_consensus.cc:3060] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.435442  2913 raft_consensus.cc:515] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:08.438282  2913 leader_election.cc:290] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:08.438637  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:08.439781  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:08.439946  2767 raft_consensus.cc:3055] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:08.440438  2843 raft_consensus.cc:3060] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.449247  2665 catalog_manager.cc:5697] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 1 to 2, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 2 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:08.440433  2767 raft_consensus.cc:740] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:08.453390  2767 consensus_queue.cc:260] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:08.460237  2767 raft_consensus.cc:3060] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.463718  2843 raft_consensus.cc:2468] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:08.466603  2880 leader_election.cc:304] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:08.467542  4030 raft_consensus.cc:2804] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:08.468072  4030 raft_consensus.cc:697] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:08.469081  4030 consensus_queue.cc:237] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:08.471208  2767 raft_consensus.cc:2468] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:08.483560  2665 catalog_manager.cc:5697] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 1 to 2, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 2 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:08.499833  2767 consensus_queue.cc:237] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:08.508155  3473 raft_consensus.cc:1275] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:08.508517  2837 raft_consensus.cc:1275] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:22:08.509284  2734 consensus_peers.cc:597] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 532f0b78131648af8fd4930674a8bce0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:08.510650  2916 raft_consensus.cc:1275] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:08.510492  4036 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:08.511799  4132 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:08.512888  4113 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:08.535804  4036 raft_consensus.cc:2955] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.542217  2916 raft_consensus.cc:2955] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.549708  2913 raft_consensus.cc:1275] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:08.557368  4139 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:08.570729  4030 raft_consensus.cc:993] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:08.571375  4030 raft_consensus.cc:1081] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:08.573179  2766 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:08.577477  2764 consensus_queue.cc:237] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:08.574357  2766 raft_consensus.cc:493] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.581759  2766 raft_consensus.cc:3060] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.586058  2665 catalog_manager.cc:5697] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.636092  2766 raft_consensus.cc:515] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:08.640302  2766 leader_election.cc:290] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:08.644346  2838 raft_consensus.cc:2955] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.659166  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:08.659958  2915 raft_consensus.cc:3055] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:08.660303  2915 raft_consensus.cc:740] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:08.662869  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:08.663630  2837 raft_consensus.cc:3060] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.674220  2910 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:08.661119  2915 consensus_queue.cc:260] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:08.676034  2915 raft_consensus.cc:3060] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.681123  4113 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:08.713292  2841 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:08.721033  4132 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:22:08.727715  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 75224.8us
I20260709 12:22:08.734476  2915 raft_consensus.cc:2468] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:08.736629  2837 raft_consensus.cc:2468] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:08.741798  2732 leader_election.cc:304] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:08.743916  4146 raft_consensus.cc:2804] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:08.744448  4146 raft_consensus.cc:697] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:08.745379  4146 consensus_queue.cc:237] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:08.756493  2764 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.758872  3819 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:22:08.772250  2734 consensus_peers.cc:597] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 22aad5981ec94ff2a74fdf45111c0692. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:08.784832  2841 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.783684  4139 raft_consensus.cc:2955] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.787202  2837 raft_consensus.cc:2955] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.788789  3819 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.798416  2910 raft_consensus.cc:2955] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.809933  2764 raft_consensus.cc:1275] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:08.813199  2663 catalog_manager.cc:5697] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:08.816406  4148 consensus_queue.cc:1048] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:08.847832  2837 consensus_queue.cc:237] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:08.856556  2663 catalog_manager.cc:5697] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.873229  2910 raft_consensus.cc:1275] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:22:08.874994  2809 consensus_peers.cc:597] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e80bb6a87c6e403fa76954737ffda91a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:08.878368  4026 consensus_queue.cc:1048] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:08.880067  2766 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.882241  2838 raft_consensus.cc:1275] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:08.883770  4030 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.890087  4102 consensus_queue.cc:1048] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:08.891113  2764 raft_consensus.cc:1275] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:08.894199  4026 consensus_queue.cc:1048] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:08.905428  4040 raft_consensus.cc:2955] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.911604  2915 raft_consensus.cc:2955] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.941146  2838 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.942868  4022 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.955917  4163 ts_tablet_manager.cc:933] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:22:08.964907  2763 raft_consensus.cc:2955] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.972255  2663 catalog_manager.cc:5697] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:08.980782  2915 consensus_queue.cc:237] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:08.996104  4163 tablet_copy_client.cc:323] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.65:35625
W20260709 12:22:08.999871  2883 consensus_peers.cc:597] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 386a0166b36244f789cc8361a1cb0a46. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:09.005636  2763 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:09.002039  2778 tablet_copy_service.cc:140] P 442c92e5267046cd92f46e6f469570a5: Received BeginTabletCopySession request for tablet 532f0b78131648af8fd4930674a8bce0 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:33574)
I20260709 12:22:09.026436  2778 tablet_copy_service.cc:161] P 442c92e5267046cd92f46e6f469570a5: Beginning new tablet copy session on tablet 532f0b78131648af8fd4930674a8bce0 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:33574: session id = f440bbb3ddd545819242017140636ad4-532f0b78131648af8fd4930674a8bce0
I20260709 12:22:09.000471  2837 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:09.027251  3819 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:09.030918  3819 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:09.034087  4030 raft_consensus.cc:2955] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:09.035598  2763 raft_consensus.cc:2955] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:09.044241  2843 raft_consensus.cc:2955] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:09.057113  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:09.065825  2778 tablet_copy_source_session.cc:215] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:09.070501  4163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 532f0b78131648af8fd4930674a8bce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:09.081836  2665 catalog_manager.cc:5697] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 2 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:09.091113  4168 ts_tablet_manager.cc:933] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:22:09.097827  4163 tablet_copy_client.cc:806] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:09.098618  4163 tablet_copy_client.cc:670] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:09.107833  4163 tablet_copy_client.cc:538] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:09.123852  4163 tablet_bootstrap.cc:492] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:09.139248  4168 tablet_copy_client.cc:323] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.65:35625
I20260709 12:22:09.141662  2778 tablet_copy_service.cc:140] P 442c92e5267046cd92f46e6f469570a5: Received BeginTabletCopySession request for tablet 22aad5981ec94ff2a74fdf45111c0692 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:33574)
I20260709 12:22:09.142323  2778 tablet_copy_service.cc:161] P 442c92e5267046cd92f46e6f469570a5: Beginning new tablet copy session on tablet 22aad5981ec94ff2a74fdf45111c0692 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:33574: session id = f440bbb3ddd545819242017140636ad4-22aad5981ec94ff2a74fdf45111c0692
I20260709 12:22:09.151496  2778 tablet_copy_source_session.cc:215] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:09.153522  2843 raft_consensus.cc:1275] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.156127  4146 consensus_queue.cc:1048] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:09.156723  4168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22aad5981ec94ff2a74fdf45111c0692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:09.193130  4163 tablet_bootstrap.cc:492] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:09.196058  4163 tablet_bootstrap.cc:492] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:09.208667  4163 ts_tablet_manager.cc:1403] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.017s
I20260709 12:22:09.212448  4163 raft_consensus.cc:359] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:09.213557  4163 raft_consensus.cc:740] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:09.214315  4163 consensus_queue.cc:260] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:09.217218  4163 ts_tablet_manager.cc:1434] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:22:09.218528  2916 raft_consensus.cc:1275] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.225898  2778 tablet_copy_service.cc:342] P 442c92e5267046cd92f46e6f469570a5: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-532f0b78131648af8fd4930674a8bce0 received from {username='slave'} at 127.0.0.1:33574
I20260709 12:22:09.226552  2778 tablet_copy_service.cc:434] P 442c92e5267046cd92f46e6f469570a5: ending tablet copy session f440bbb3ddd545819242017140636ad4-532f0b78131648af8fd4930674a8bce0 on tablet 532f0b78131648af8fd4930674a8bce0 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:09.232985  4139 consensus_queue.cc:1048] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:09.246521  4168 tablet_copy_client.cc:806] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:09.249488  4168 tablet_copy_client.cc:670] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:09.257591  4168 tablet_copy_client.cc:538] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:09.267190  2766 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:09.268204  2766 raft_consensus.cc:606] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Received request to transfer leadership to TS 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:09.272802  4168 tablet_bootstrap.cc:492] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:09.275710  2915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:09.276399  2915 raft_consensus.cc:606] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:09.284070  2843 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d436a6cfd22640bea76c0256f1d01684"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
new_leader_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:36554
I20260709 12:22:09.284735  2843 raft_consensus.cc:606] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Received request to transfer leadership to TS 7cc87046f1414b27928722690a5e388f
I20260709 12:22:09.293552  4163 ts_tablet_manager.cc:933] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:09.306397  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:22:09.316589  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:09.323348  4163 tablet_copy_client.cc:323] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:22:09.318463  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:09.325426  2852 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet e80bb6a87c6e403fa76954737ffda91a from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:22:09.326081  2852 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet e80bb6a87c6e403fa76954737ffda91a from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-e80bb6a87c6e403fa76954737ffda91a
I20260709 12:22:09.334754  2852 tablet_copy_source_session.cc:215] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:09.338192  4163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e80bb6a87c6e403fa76954737ffda91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:09.347728  4168 tablet_bootstrap.cc:492] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:09.348845  4168 tablet_bootstrap.cc:492] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:09.349849  4168 ts_tablet_manager.cc:1403] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.077s	user 0.033s	sys 0.021s
I20260709 12:22:09.359091  4168 raft_consensus.cc:359] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:09.360060  4168 raft_consensus.cc:740] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:09.360787  4168 consensus_queue.cc:260] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:09.363843  4168 ts_tablet_manager.cc:1434] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:22:09.366731  4163 tablet_copy_client.cc:806] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:09.367331  4163 tablet_copy_client.cc:670] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:09.370306  2778 tablet_copy_service.cc:342] P 442c92e5267046cd92f46e6f469570a5: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-22aad5981ec94ff2a74fdf45111c0692 received from {username='slave'} at 127.0.0.1:33574
I20260709 12:22:09.370774  2778 tablet_copy_service.cc:434] P 442c92e5267046cd92f46e6f469570a5: ending tablet copy session f440bbb3ddd545819242017140636ad4-22aad5981ec94ff2a74fdf45111c0692 on tablet 22aad5981ec94ff2a74fdf45111c0692 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:09.373303  4163 tablet_copy_client.cc:538] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:09.389914  4163 tablet_bootstrap.cc:492] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:09.419569  4163 tablet_bootstrap.cc:492] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:09.420622  4163 tablet_bootstrap.cc:492] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:09.421375  4163 ts_tablet_manager.cc:1403] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.008s
I20260709 12:22:09.424640  4163 raft_consensus.cc:359] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:09.425515  4163 raft_consensus.cc:740] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:09.426229  4163 consensus_queue.cc:260] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:09.428891  4163 ts_tablet_manager.cc:1434] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 12:22:09.430773  2852 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-e80bb6a87c6e403fa76954737ffda91a received from {username='slave'} at 127.0.0.1:38132
I20260709 12:22:09.431226  2852 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-e80bb6a87c6e403fa76954737ffda91a on tablet e80bb6a87c6e403fa76954737ffda91a with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:09.458866  3470 raft_consensus.cc:1217] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:22:09.468454  4163 ts_tablet_manager.cc:933] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:09.474834  4163 tablet_copy_client.cc:323] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:22:09.476588  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 386a0166b36244f789cc8361a1cb0a46 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:22:09.477167  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 386a0166b36244f789cc8361a1cb0a46 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-386a0166b36244f789cc8361a1cb0a46
I20260709 12:22:09.488138  2927 tablet_copy_source_session.cc:215] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:09.494247  4163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 386a0166b36244f789cc8361a1cb0a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:09.518837  3470 raft_consensus.cc:1217] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:22:09.528147  4163 tablet_copy_client.cc:806] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:09.529173  4163 tablet_copy_client.cc:670] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:09.533974  4163 tablet_copy_client.cc:538] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:09.543512  4163 tablet_bootstrap.cc:492] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:09.560881  4163 tablet_bootstrap.cc:492] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:09.561841  4163 tablet_bootstrap.cc:492] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:09.562501  4163 ts_tablet_manager.cc:1403] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.004s
I20260709 12:22:09.565218  4163 raft_consensus.cc:359] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:09.566876  4163 raft_consensus.cc:740] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:09.567468  4163 consensus_queue.cc:260] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:09.569665  4163 ts_tablet_manager.cc:1434] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:22:09.571812  4030 raft_consensus.cc:993] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:09.572398  4030 raft_consensus.cc:1081] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:09.575068  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-386a0166b36244f789cc8361a1cb0a46 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:22:09.575676  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-386a0166b36244f789cc8361a1cb0a46 on tablet 386a0166b36244f789cc8361a1cb0a46 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:09.605324  2766 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:09.606362  2766 raft_consensus.cc:493] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:09.606832  2766 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:09.623283  2766 raft_consensus.cc:515] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:09.626210  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:09.626420  2766 leader_election.cc:290] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:09.626753  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:09.626876  2843 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:09.627322  2916 raft_consensus.cc:3055] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:09.627641  2916 raft_consensus.cc:740] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:09.628456  2916 consensus_queue.cc:260] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:09.629765  2916 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:09.638144  2843 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 3.
I20260709 12:22:09.640136  2731 leader_election.cc:304] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:09.641373  4139 raft_consensus.cc:2804] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:09.642007  4139 raft_consensus.cc:697] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:09.642911  4139 consensus_queue.cc:237] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:09.648777  2916 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 3.
I20260709 12:22:09.659783  2663 catalog_manager.cc:5697] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 2 to 3, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:09.700536  3470 raft_consensus.cc:1217] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:09.705863  4165 raft_consensus.cc:993] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06: : Instructing follower 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:09.706451  4165 raft_consensus.cc:1081] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Signalling peer 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:09.708094  2915 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d436a6cfd22640bea76c0256f1d01684"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:47632
I20260709 12:22:09.708688  2915 raft_consensus.cc:493] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:09.709075  2915 raft_consensus.cc:3060] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:09.716629  2915 raft_consensus.cc:515] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:09.719431  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d436a6cfd22640bea76c0256f1d01684" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:09.720185  2843 raft_consensus.cc:3055] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:09.720551  2843 raft_consensus.cc:740] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:09.721393  3473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d436a6cfd22640bea76c0256f1d01684" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:09.719880  2915 leader_election.cc:290] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:09.721817  2843 consensus_queue.cc:260] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:09.722066  3473 raft_consensus.cc:3060] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:09.723064  2843 raft_consensus.cc:3060] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:09.730918  3473 raft_consensus.cc:2468] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:09.732252  2883 leader_election.cc:304] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:09.733445  3819 raft_consensus.cc:2804] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:09.733973  3819 raft_consensus.cc:697] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:09.734833  3819 consensus_queue.cc:237] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:09.736603  2843 raft_consensus.cc:2468] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:09.755929  2663 catalog_manager.cc:5697] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 1 to 2, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 2 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:09.789366  4132 raft_consensus.cc:993] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:09.790026  4139 raft_consensus.cc:1081] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Signalling peer 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:09.791596  2843 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:42536
I20260709 12:22:09.792095  2843 raft_consensus.cc:493] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:09.792416  2843 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:09.798362  2843 raft_consensus.cc:515] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:09.800087  2843 leader_election.cc:290] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:09.801760  2763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:09.802415  2763 raft_consensus.cc:3055] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:09.801760  2915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:09.802819  2763 raft_consensus.cc:740] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:09.803025  2915 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:09.803689  2763 consensus_queue.cc:260] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:09.805356  2763 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:09.813038  2763 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 3.
I20260709 12:22:09.813675  2915 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 3.
I20260709 12:22:09.815172  2807 leader_election.cc:304] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:09.816471  4165 raft_consensus.cc:2804] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:09.817011  4165 raft_consensus.cc:697] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:09.818019  4165 consensus_queue.cc:237] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:09.848176  2663 catalog_manager.cc:5697] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:09.861115  2843 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42516
I20260709 12:22:09.861828  2843 raft_consensus.cc:606] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Received request to transfer leadership
I20260709 12:22:09.873983  2843 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42516
I20260709 12:22:09.874640  2843 raft_consensus.cc:606] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Received request to transfer leadership
W20260709 12:22:09.877660  2650 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 22aad5981ec94ff2a74fdf45111c0692 already in progress [suppressed 4 similar messages]
I20260709 12:22:09.883173  2843 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42516
I20260709 12:22:09.883773  2843 raft_consensus.cc:606] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Received request to transfer leadership
W20260709 12:22:09.889071  2714 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:22:09.889588  2714 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 22aad5981ec94ff2a74fdf45111c0692 already in progress
W20260709 12:22:09.889861  2714 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 22aad5981ec94ff2a74fdf45111c0692 already in progress
I20260709 12:22:09.940744  3473 raft_consensus.cc:1217] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:22:10.012246  4139 raft_consensus.cc:1064] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:10.014923  4139 consensus_queue.cc:237] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:10.023087  3473 raft_consensus.cc:1275] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:22:10.109223  2843 raft_consensus.cc:1275] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:22:10.111428  2843 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:10.117153  4146 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:10.120651  2915 raft_consensus.cc:1275] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:22:10.124854  4146 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:10.126117  4147 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:10.130016  4139 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:22:10.185894  2916 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:10.187716  4190 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:10.192929  4178 raft_consensus.cc:1064] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:10.196460  3470 raft_consensus.cc:1275] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:10.196254  4178 consensus_queue.cc:237] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:10.198750  4169 consensus_queue.cc:1048] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:10.211984  3474 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Advancing to term 3
I20260709 12:22:10.248651  3474 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:10.250690  4189 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:10.274993  2916 raft_consensus.cc:2955] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:10.284703  2910 raft_consensus.cc:1275] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:10.285234  3470 raft_consensus.cc:1275] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:10.286252  2763 raft_consensus.cc:1275] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:10.259883  4190 raft_consensus.cc:2955] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:10.291097  4165 consensus_queue.cc:1048] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:10.293391  4189 consensus_queue.cc:1048] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:10.298319  4165 consensus_queue.cc:1048] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:22:10.310742  2837 raft_consensus.cc:1275] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:10.318354  4030 consensus_queue.cc:1048] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:22:10.409197  2916 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 12:22:10.418929  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 94809.3us
I20260709 12:22:10.416425  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:10.430974  4207 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:10.436019  2763 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:10.447831  4207 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:22:10.513032  3473 raft_consensus.cc:2955] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:10.545146  4030 raft_consensus.cc:1064] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:10.547988  4030 consensus_queue.cc:237] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:10.548599  2666 catalog_manager.cc:5697] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:22:10.587811  2809 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 123936us
W20260709 12:22:10.617125  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 169566us
I20260709 12:22:10.631403  4165 raft_consensus.cc:1064] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:10.641969  4223 raft_consensus.cc:993] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: : Instructing follower 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:10.645277  4223 raft_consensus.cc:2955] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:10.649310  3473 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 12:22:10.651011  4165 raft_consensus.cc:1068] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: Unable to promote non-voter f440bbb3ddd545819242017140636ad4: Service unavailable: leader transfer in progress
I20260709 12:22:10.652070  4220 raft_consensus.cc:1081] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Signalling peer 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:10.659304  2841 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:22:10.664862  2767 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:22:10.662281  3819 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:10.685102  4169 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:10.708317  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 44853.1us
I20260709 12:22:10.744885  4230 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:22:10.753314  2916 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:47632
I20260709 12:22:10.754326  2916 raft_consensus.cc:493] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:22:10.755125  2809 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 51855.6us
I20260709 12:22:10.754814  2916 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:10.772924  2910 raft_consensus.cc:2955] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:10.775810  2916 raft_consensus.cc:515] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:10.770888  3473 raft_consensus.cc:2955] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:10.792361  4220 raft_consensus.cc:1064] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
W20260709 12:22:10.797941  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 171839us
W20260709 12:22:10.798691  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 179491us
I20260709 12:22:10.811683  2916 leader_election.cc:290] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 4 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:10.808493  4220 raft_consensus.cc:1068] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: Unable to promote non-voter f440bbb3ddd545819242017140636ad4: Service unavailable: leader transfer in progress
I20260709 12:22:10.876595  2767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:10.877386  2767 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:10.903069  4203 log.cc:927] Time spent T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:22:10.962013  4217 log.cc:927] Time spent T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:22:10.989279  2838 raft_consensus.cc:2955] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.004652  2767 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 4.
I20260709 12:22:11.019634  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:11.020393  2843 raft_consensus.cc:3055] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:11.020736  2843 raft_consensus.cc:740] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:11.026067  2843 consensus_queue.cc:260] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
W20260709 12:22:11.036026  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 229928us
I20260709 12:22:11.036952  2843 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:11.057160  2881 leader_election.cc:304] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:11.080296  4169 raft_consensus.cc:2804] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:11.088621  2843 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 4.
I20260709 12:22:11.092005  2913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:11.092763  2913 raft_consensus.cc:606] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:11.086654  4169 raft_consensus.cc:697] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 4 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:11.110563  4169 consensus_queue.cc:237] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:11.147405  2766 raft_consensus.cc:2955] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:22:11.151211  2807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:11.157294  2761 raft_consensus.cc:1240] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Rejecting Update request from peer 9a10672cd53140d59ff945f75345cc06 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 12:22:11.186575  2666 catalog_manager.cc:5697] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.265017  4214 raft_consensus.cc:2955] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:22:11.291071  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 196940us
I20260709 12:22:11.297968  2837 raft_consensus.cc:2955] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.317509  2761 consensus_queue.cc:237] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:11.335538  3474 raft_consensus.cc:2955] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.344260  3475 raft_consensus.cc:1275] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:22:11.344192  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 51395.8us
I20260709 12:22:11.354353  2837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:36554
I20260709 12:22:11.355404  2837 raft_consensus.cc:606] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:11.358721  2838 raft_consensus.cc:1275] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:11.361272  4244 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:11.362967  4214 raft_consensus.cc:993] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:11.363502  4229 raft_consensus.cc:1081] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:11.364461  2768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa220c2195724c24b522d44636d98970"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:11.370797  2765 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:11.371548  2765 raft_consensus.cc:493] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:11.371940  2765 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:11.365303  2768 raft_consensus.cc:606] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Received request to transfer leadership to TS 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:11.379057  4244 raft_consensus.cc:2955] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.383934  4219 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:11.385099  2838 raft_consensus.cc:2955] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.390901  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:22:11.393198  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:11.400318  3475 raft_consensus.cc:2955] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.410825  2765 raft_consensus.cc:515] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:11.416419  2766 raft_consensus.cc:1240] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Rejecting Update request from peer 7cc87046f1414b27928722690a5e388f for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:11.418460  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:11.419132  2837 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:11.425515  4229 consensus_queue.cc:1059] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:11.427048  4229 raft_consensus.cc:3055] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:11.428902  4229 raft_consensus.cc:740] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:11.430111  4229 consensus_queue.cc:260] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:11.431730  4229 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:11.433077  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:11.436144  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:11.428936  2837 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 3.
I20260709 12:22:11.442584  3475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:11.443396  3475 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:11.446213  2765 leader_election.cc:290] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:11.472879  2917 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 3.
I20260709 12:22:11.483276  2732 leader_election.cc:304] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:11.484398  4202 raft_consensus.cc:2804] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:11.484925  4202 raft_consensus.cc:697] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:11.485929  4202 consensus_queue.cc:237] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:11.491982  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 532f0b78131648af8fd4930674a8bce0 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:11.487007  2837 consensus_queue.cc:237] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:11.499310  3475 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 3.
I20260709 12:22:11.507761  2664 catalog_manager.cc:5697] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 4 to 5, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.509296  3474 raft_consensus.cc:1275] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:11.509580  2664 catalog_manager.cc:5697] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 3 to 4, term changed from 2 to 3, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to <none>, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.511229  4207 consensus_queue.cc:1048] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:11.526337  3474 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Advancing to term 4
I20260709 12:22:11.532972  2917 raft_consensus.cc:1275] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:11.535300  4236 consensus_queue.cc:1048] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:11.539840  2666 catalog_manager.cc:5697] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 3 to 4, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 4 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.548480  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet 532f0b78131648af8fd4930674a8bce0 with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47626
I20260709 12:22:11.549505  4189 raft_consensus.cc:2955] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.579174  3473 raft_consensus.cc:2955] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.579883  2663 catalog_manager.cc:5697] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 4 to 3, leader changed from <none> to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65), f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from VOTER to NON_VOTER, now has a pending config: VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65), VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66), VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67), VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 3 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.595395  2917 raft_consensus.cc:2955] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:11.613766  4252 tablet_replica.cc:333] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:11.621807  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e80bb6a87c6e403fa76954737ffda91a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:11.626924  4252 raft_consensus.cc:2243] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:11.630637  3474 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 4 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:22:11.629778  2663 catalog_manager.cc:5697] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:11.633958  4229 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:11.637431  4252 raft_consensus.cc:2272] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:11.637454  2768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38926
I20260709 12:22:11.641848  2768 raft_consensus.cc:606] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Received request to transfer leadership
I20260709 12:22:11.650924  4252 ts_tablet_manager.cc:1916] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:11.653261  4229 raft_consensus.cc:1064] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
W20260709 12:22:11.657011  4229 raft_consensus.cc:1068] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f: Unable to promote non-voter f440bbb3ddd545819242017140636ad4: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:22:11.661468  2768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38926
I20260709 12:22:11.662227  2768 raft_consensus.cc:606] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Received request to transfer leadership
I20260709 12:22:11.688545  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet e80bb6a87c6e403fa76954737ffda91a with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:22:11.688727  2768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38926
I20260709 12:22:11.689461  2768 raft_consensus.cc:606] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Received request to transfer leadership
I20260709 12:22:11.695640  2837 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:22:11.697153  4229 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:11.709815  4254 tablet_replica.cc:333] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
W20260709 12:22:11.712299  2714 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:22:11.714730  2714 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 386a0166b36244f789cc8361a1cb0a46 already in progress
W20260709 12:22:11.715071  2714 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 386a0166b36244f789cc8361a1cb0a46 already in progress
I20260709 12:22:11.712857  4252 ts_tablet_manager.cc:1929] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:22:11.717180  4252 log.cc:1199] T 532f0b78131648af8fd4930674a8bce0 P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/532f0b78131648af8fd4930674a8bce0
I20260709 12:22:11.719004  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet 532f0b78131648af8fd4930674a8bce0 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:11.727067  2768 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:22:11.729158  4249 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:11.736786  4244 raft_consensus.cc:993] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:11.737746  4219 raft_consensus.cc:1081] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Signalling peer 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:11.740777  4254 raft_consensus.cc:2243] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:11.741969  4254 raft_consensus.cc:2272] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:11.746150  4254 ts_tablet_manager.cc:1916] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:11.754612  2837 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa220c2195724c24b522d44636d98970"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:42536
I20260709 12:22:11.755615  2837 raft_consensus.cc:493] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:11.756137  2837 raft_consensus.cc:3060] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:11.904398  2838 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:11.905179  4254 ts_tablet_manager.cc:1929] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:11.905853  4254 log.cc:1199] T e80bb6a87c6e403fa76954737ffda91a P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/e80bb6a87c6e403fa76954737ffda91a
I20260709 12:22:11.906840  4204 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:11.909636  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet e80bb6a87c6e403fa76954737ffda91a (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:11.909174  2837 raft_consensus.cc:515] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:11.915174  2837 leader_election.cc:290] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:11.920583  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:11.921454  2765 raft_consensus.cc:3055] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:11.921897  2765 raft_consensus.cc:740] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:11.922873  2765 consensus_queue.cc:260] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:11.924604  2765 raft_consensus.cc:3060] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:11.962517  2765 raft_consensus.cc:2468] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 3.
I20260709 12:22:11.964393  2807 leader_election.cc:304] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:11.994853  3474 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:11.984122  4222 raft_consensus.cc:2804] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:22:12.096081  4255 log.cc:927] Time spent T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: Append to log took a long time: real 0.095s	user 0.003s	sys 0.000s
I20260709 12:22:12.094894  4222 raft_consensus.cc:697] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
W20260709 12:22:12.125504  2734 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 111816us
W20260709 12:22:12.207350  2734 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 80615.5us
I20260709 12:22:12.166697  4146 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:22:12.180759  4222 consensus_queue.cc:237] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:12.227804  4204 raft_consensus.cc:993] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:12.228690  4204 raft_consensus.cc:1081] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Signalling peer 9a10672cd53140d59ff945f75345cc06 to start an election
W20260709 12:22:12.223894  4258 log.cc:927] Time spent T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 12:22:12.275255  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 106177us
I20260709 12:22:12.243192  2663 catalog_manager.cc:5697] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65), VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66), VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67), VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 3 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:22:12.331398  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 115190us
W20260709 12:22:12.377655  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 43872.1us
I20260709 12:22:12.419238  4268 raft_consensus.cc:993] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:12.421833  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:12.422560  2910 raft_consensus.cc:3060] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:12.427587  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 48860.5us
W20260709 12:22:12.428026  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 47980.5us
W20260709 12:22:12.489971  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 59357.8us
I20260709 12:22:12.495834  2910 raft_consensus.cc:2468] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 3.
I20260709 12:22:12.446344  4268 raft_consensus.cc:1081] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
W20260709 12:22:12.527899  2734 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 45104.5us
I20260709 12:22:12.532956  2913 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:22:12.508427  4258 log.cc:927] Time spent T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 12:22:12.609501  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 57824.4us
W20260709 12:22:12.613668  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 58870.7us
W20260709 12:22:12.615821  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 68083.8us
W20260709 12:22:12.616577  2809 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 53557.8us
I20260709 12:22:12.615630  4300 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
W20260709 12:22:12.634164  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 87923.9us
I20260709 12:22:12.635469  3476 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38432
I20260709 12:22:12.637292  3476 raft_consensus.cc:493] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:22:12.638124  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 190395us
I20260709 12:22:12.638341  3476 raft_consensus.cc:3060] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:12.632906  3939 log.cc:927] Time spent T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:22:12.684844  4277 raft_consensus.cc:1064] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:12.665861  4146 raft_consensus.cc:2955] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:12.691480  4277 consensus_queue.cc:237] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:12.696635  3476 raft_consensus.cc:515] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:12.730384  2910 raft_consensus.cc:1275] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:12.731703  3470 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 4 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:12.732661  3476 leader_election.cc:290] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:12.733965  2838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:12.734597  2838 raft_consensus.cc:3055] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:12.734938  2838 raft_consensus.cc:740] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:12.737640  2841 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:12.739550  2838 consensus_queue.cc:260] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:12.740774  2838 raft_consensus.cc:3060] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.741614  2765 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:12.717455  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:12.754762  4277 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:12.755791  4262 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:12.765763  4261 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:12.806942  2843 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:42536
I20260709 12:22:12.807595  2843 raft_consensus.cc:493] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:12.807976  2843 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:12.832674  2843 raft_consensus.cc:515] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:12.834450  2842 raft_consensus.cc:1240] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Rejecting Update request from peer 442c92e5267046cd92f46e6f469570a5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:12.838960  2843 leader_election.cc:290] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 4 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:12.845274  4276 raft_consensus.cc:2955] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 3 to 6, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:12.849901  3476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:12.850606  3476 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:12.851492  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:12.852210  2768 raft_consensus.cc:3060] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.852403  4222 consensus_queue.cc:1048] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:12.858374  2765 raft_consensus.cc:2955] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 3 to 6, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:12.862460  4202 consensus_queue.cc:1059] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:12.864248  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:12.864883  2911 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:12.868443  2663 catalog_manager.cc:5697] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:12.875751  2768 raft_consensus.cc:2468] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:12.887773  3476 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 4.
I20260709 12:22:12.899142  2838 raft_consensus.cc:2468] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:12.900851  3439 leader_election.cc:304] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:12.903239  4272 raft_consensus.cc:2804] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:12.903798  4272 raft_consensus.cc:697] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:12.904829  4272 consensus_queue.cc:237] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:12.900134  2911 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 4.
I20260709 12:22:12.922891  2807 leader_election.cc:304] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:12.924312  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:12.928787  2911 raft_consensus.cc:1240] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Rejecting Update request from peer 442c92e5267046cd92f46e6f469570a5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:12.930843  4222 raft_consensus.cc:2804] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:12.931449  4222 raft_consensus.cc:697] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:12.932463  4222 consensus_queue.cc:237] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:12.930370  3473 raft_consensus.cc:2955] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 3 to 6, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:12.923337  2666 catalog_manager.cc:5697] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 1 to 2, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 2 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:12.940583  4146 raft_consensus.cc:3055] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:12.941092  4146 raft_consensus.cc:740] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:12.942018  4146 consensus_queue.cc:260] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:12.962905  2665 catalog_manager.cc:5697] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 3 to 6, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:12.967620  4146 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:12.974364  2766 raft_consensus.cc:1275] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:12.937036  2843 raft_consensus.cc:2955] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 3 to 6, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:12.978524  2664 catalog_manager.cc:5697] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 3 to 4, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 4 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:12.987028  2762 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 4.
I20260709 12:22:13.003870  2841 raft_consensus.cc:1240] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Rejecting Update request from peer 442c92e5267046cd92f46e6f469570a5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:13.012627  4302 consensus_queue.cc:1048] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:13.036406  2916 consensus_queue.cc:237] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.052405  3473 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:22:13.054330  2762 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:22:13.054805  4262 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:13.057422  4296 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:13.083845  4296 raft_consensus.cc:2955] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 4 LEADER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:13.096654  3473 raft_consensus.cc:2955] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:13.093675  2768 raft_consensus.cc:2955] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:13.174945  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 22aad5981ec94ff2a74fdf45111c0692 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:13.181722  2663 catalog_manager.cc:5697] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 6 to 7, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 4 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:13.208698  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet 22aad5981ec94ff2a74fdf45111c0692 with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:42516
I20260709 12:22:13.217808  2843 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
new_leader_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:36554
I20260709 12:22:13.218569  2843 raft_consensus.cc:606] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Received request to transfer leadership to TS 7cc87046f1414b27928722690a5e388f
I20260709 12:22:13.237783  2768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:13.238478  2768 raft_consensus.cc:606] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:13.244637  2910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:13.245075  4313 tablet_replica.cc:333] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:13.245283  2910 raft_consensus.cc:606] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:13.248636  2910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:13.249207  2910 raft_consensus.cc:606] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:13.249856  4313 raft_consensus.cc:2243] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.250921  4313 raft_consensus.cc:2272] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.253279  2842 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:36554
I20260709 12:22:13.253976  2842 raft_consensus.cc:606] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:13.255267  4313 ts_tablet_manager.cc:1916] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:13.255672  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:13.257417  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:13.258239  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:13.272586  2768 consensus_queue.cc:237] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:13.283316  4313 ts_tablet_manager.cc:1929] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.6
I20260709 12:22:13.283628  2910 raft_consensus.cc:1275] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:13.284104  4313 log.cc:1199] T 22aad5981ec94ff2a74fdf45111c0692 P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/22aad5981ec94ff2a74fdf45111c0692
I20260709 12:22:13.284188  2843 raft_consensus.cc:1275] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:22:13.285926  2734 consensus_peers.cc:597] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e4cde589c024472b0a4cec236ea69a3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.286439  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet 22aad5981ec94ff2a74fdf45111c0692 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:13.286550  4291 consensus_queue.cc:1048] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:13.287549  4146 consensus_queue.cc:1048] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:13.302405  4204 raft_consensus.cc:2955] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:13.306911  2842 raft_consensus.cc:2955] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:13.311105  2768 raft_consensus.cc:1275] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:13.315858  4272 consensus_queue.cc:1048] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:13.326455  2910 raft_consensus.cc:2955] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:13.334254  2666 catalog_manager.cc:5697] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:13.336572  2916 consensus_queue.cc:237] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
W20260709 12:22:13.353606  2883 consensus_peers.cc:597] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eface8ee310347d4bb08e3c5864ce5d9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.356078  2842 raft_consensus.cc:1275] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:13.357089  2766 raft_consensus.cc:1275] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:13.361253  4264 consensus_queue.cc:1048] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:13.365445  4269 raft_consensus.cc:993] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:13.365980  4269 raft_consensus.cc:1081] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:13.367676  2843 raft_consensus.cc:1275] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:13.368661  4296 consensus_queue.cc:1048] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:13.378129  3476 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38432
I20260709 12:22:13.378847  3476 raft_consensus.cc:493] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.379326  3476 raft_consensus.cc:3060] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.384527  4319 consensus_queue.cc:1048] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:13.391414  3476 raft_consensus.cc:515] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.396687  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:13.397475  2765 raft_consensus.cc:3060] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.396628  2842 raft_consensus.cc:2955] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:13.392473  4296 raft_consensus.cc:2955] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:13.417083  3476 leader_election.cc:290] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:13.417272  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:13.418695  2765 raft_consensus.cc:2468] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:13.421092  3440 leader_election.cc:304] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:13.422067  4320 raft_consensus.cc:2804] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.422653  4320 raft_consensus.cc:697] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:13.423478  4320 consensus_queue.cc:237] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.436872  2836 consensus_queue.cc:237] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:13.447197  2768 raft_consensus.cc:2955] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:13.418111  2843 raft_consensus.cc:3055] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:13.450371  2843 raft_consensus.cc:740] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:13.466012  4219 raft_consensus.cc:993] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:13.466619  4219 raft_consensus.cc:1081] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:13.468772  3476 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38416
I20260709 12:22:13.469305  3476 raft_consensus.cc:493] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.469655  3476 raft_consensus.cc:3060] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.475814  2766 raft_consensus.cc:1240] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Rejecting Update request from peer 9a10672cd53140d59ff945f75345cc06 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:13.477392  4222 consensus_queue.cc:1059] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:13.482456  4264 raft_consensus.cc:993] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:13.482554  3476 raft_consensus.cc:515] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.483179  2766 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:13.484059  4277 raft_consensus.cc:1081] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:13.485766  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:13.486495  2766 raft_consensus.cc:3055] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:13.486860  2766 raft_consensus.cc:740] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:13.484704  4222 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:13.487818  2766 consensus_queue.cc:260] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.491075  2910 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:13.494228  2762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:13.495826  2762 raft_consensus.cc:493] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.496287  2762 raft_consensus.cc:3060] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.496169  4268 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:13.491987  3476 leader_election.cc:290] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:13.509475  4262 raft_consensus.cc:993] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:13.510082  4262 raft_consensus.cc:1081] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:13.493502  2766 raft_consensus.cc:3060] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.512727  2761 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:13.513286  2761 raft_consensus.cc:493] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.513331  2838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532f0b78131648af8fd4930674a8bce0" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:13.513720  2761 raft_consensus.cc:3060] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.513998  2838 raft_consensus.cc:3060] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.523361  2761 raft_consensus.cc:515] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.524521  3473 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:13.526520  4293 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:13.527555  2916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:13.528201  2916 raft_consensus.cc:3055] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:13.528529  2916 raft_consensus.cc:740] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:13.528456  3473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:13.529376  3473 raft_consensus.cc:3060] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.529702  2916 consensus_queue.cc:260] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.530735  2916 raft_consensus.cc:3060] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.545810  2843 consensus_queue.cc:260] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.553118  2843 raft_consensus.cc:3060] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.565502  2916 raft_consensus.cc:2468] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:13.528225  2761 leader_election.cc:290] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:13.602981  2732 leader_election.cc:304] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:13.604521  2766 raft_consensus.cc:2468] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 3.
I20260709 12:22:13.460547  2663 catalog_manager.cc:5697] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:13.613965  2809 consensus_peers.cc:597] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 627b3359300e44fc9d2b83ed88272a9f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.605527  2762 raft_consensus.cc:515] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:13.621771  2765 raft_consensus.cc:1275] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:13.611498  2838 raft_consensus.cc:2468] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 3.
I20260709 12:22:13.622496  4293 raft_consensus.cc:993] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06: : Instructing follower 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:13.632445  4293 raft_consensus.cc:1081] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Signalling peer 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:13.632887  2762 leader_election.cc:290] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:13.607095  2664 catalog_manager.cc:5697] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 1 to 2, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 2 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:13.617972  4291 raft_consensus.cc:2804] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.635643  4291 raft_consensus.cc:697] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:13.636543  4291 consensus_queue.cc:237] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.617985  2916 raft_consensus.cc:1275] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:13.640245  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:13.640848  2910 raft_consensus.cc:3055] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:13.641120  2910 raft_consensus.cc:740] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:13.642117  4290 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:13.642457  2915 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:47632
I20260709 12:22:13.643517  2910 consensus_queue.cc:260] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:13.644109  4290 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:13.644969  2910 raft_consensus.cc:3060] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.623683  3440 leader_election.cc:304] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:13.649243  4234 raft_consensus.cc:2804] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:13.651463  4234 raft_consensus.cc:697] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 3 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:13.652493  4234 consensus_queue.cc:237] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.660779  2910 raft_consensus.cc:2468] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 3.
I20260709 12:22:13.668726  2838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:13.669387  2915 raft_consensus.cc:493] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.670183  2915 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.682449  2732 leader_election.cc:304] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:13.683502  4204 raft_consensus.cc:2804] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:13.684029  4204 raft_consensus.cc:697] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:13.684921  4204 consensus_queue.cc:237] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:13.691876  2838 raft_consensus.cc:3060] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.695462  4279 raft_consensus.cc:2955] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:13.698421  2915 raft_consensus.cc:515] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.709385  3473 raft_consensus.cc:2468] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:13.710614  3476 raft_consensus.cc:1240] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Rejecting Update request from peer 7cc87046f1414b27928722690a5e388f for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:13.705364  2910 raft_consensus.cc:2955] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
W20260709 12:22:13.715114  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 46450.1us
W20260709 12:22:13.722757  2650 catalog_manager.cc:4755] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): ChangeConfig:REMOVE_PEER RPC failed for tablet 386a0166b36244f789cc8361a1cb0a46: Remote error: Service unavailable: leader transfer in progress
I20260709 12:22:13.735936  2913 consensus_queue.cc:237] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:13.746088  2836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:13.745972  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:13.746966  2762 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.747089  2915 leader_election.cc:290] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:13.747838  2662 catalog_manager.cc:5697] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 3 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:13.754195  2838 raft_consensus.cc:2468] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 3.
I20260709 12:22:13.759308  2843 raft_consensus.cc:2468] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
W20260709 12:22:13.760157  4269 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:22:13.760320  2836 raft_consensus.cc:3055] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:13.760753  2836 raft_consensus.cc:740] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:13.769170  2836 consensus_queue.cc:260] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.771248  2762 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 5.
I20260709 12:22:13.771994  2836 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.781126  2836 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 5.
I20260709 12:22:13.782423  2880 leader_election.cc:304] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:13.783345  3474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:13.783988  3474 raft_consensus.cc:3060] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.786154  4296 raft_consensus.cc:2804] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:13.786754  4296 raft_consensus.cc:697] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 5 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:13.787915  4296 consensus_queue.cc:237] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:13.792055  3474 raft_consensus.cc:2468] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 5.
I20260709 12:22:13.791585  2664 catalog_manager.cc:5697] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:13.803170  2650 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 386a0166b36244f789cc8361a1cb0a46 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9a10672cd53140d59ff945f75345cc06 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:13.806651  2666 catalog_manager.cc:5697] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 3 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:13.816052  2765 raft_consensus.cc:2955] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
W20260709 12:22:13.829916  2883 consensus_peers.cc:597] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 42af6798b62746449070b7b2bc4e0699. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.831703  2836 raft_consensus.cc:1275] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:13.832374  2766 raft_consensus.cc:1275] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:13.833592  4335 ts_tablet_manager.cc:933] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:22:13.834309  4277 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:13.834120  2666 catalog_manager.cc:5697] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 2 to 3, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:13.837164  4277 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:13.848486  2664 catalog_manager.cc:5697] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 4 to 5, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 5 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:13.859405  4335 tablet_copy_client.cc:323] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.65:35625
I20260709 12:22:13.861733  2778 tablet_copy_service.cc:140] P 442c92e5267046cd92f46e6f469570a5: Received BeginTabletCopySession request for tablet 7e4cde589c024472b0a4cec236ea69a3 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:33574)
I20260709 12:22:13.862330  2778 tablet_copy_service.cc:161] P 442c92e5267046cd92f46e6f469570a5: Beginning new tablet copy session on tablet 7e4cde589c024472b0a4cec236ea69a3 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:33574: session id = f440bbb3ddd545819242017140636ad4-7e4cde589c024472b0a4cec236ea69a3
I20260709 12:22:13.862649  2765 raft_consensus.cc:1275] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:13.864113  4234 consensus_queue.cc:1048] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:13.866186  4264 raft_consensus.cc:2955] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:13.870877  2778 tablet_copy_source_session.cc:215] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:13.876216  2836 raft_consensus.cc:2955] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:13.900264  2843 raft_consensus.cc:1275] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:13.917042  4320 consensus_queue.cc:1048] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:13.919332  4335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e4cde589c024472b0a4cec236ea69a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:13.900122  2666 catalog_manager.cc:5697] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:13.946834  2766 raft_consensus.cc:2955] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:13.982661  4341 ts_tablet_manager.cc:933] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:14.006486  4335 tablet_copy_client.cc:806] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:14.016126  4335 tablet_copy_client.cc:670] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:14.019433  4341 tablet_copy_client.cc:323] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:22:14.021970  2927 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet eface8ee310347d4bb08e3c5864ce5d9 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:22:14.022516  2927 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet eface8ee310347d4bb08e3c5864ce5d9 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-eface8ee310347d4bb08e3c5864ce5d9
I20260709 12:22:14.030550  2927 tablet_copy_source_session.cc:215] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:14.033890  4341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eface8ee310347d4bb08e3c5864ce5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.038201  4335 tablet_copy_client.cc:538] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:14.052148  4335 tablet_bootstrap.cc:492] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:14.052368  4341 tablet_copy_client.cc:806] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:14.053182  4341 tablet_copy_client.cc:670] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:14.058831  4341 tablet_copy_client.cc:538] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:14.069440  4341 tablet_bootstrap.cc:492] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:14.084407  2836 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:14.085446  4335 tablet_bootstrap.cc:492] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:14.085911  4204 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:14.086547  4335 tablet_bootstrap.cc:492] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:14.087479  4335 ts_tablet_manager.cc:1403] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.036s	user 0.024s	sys 0.006s
I20260709 12:22:14.091207  4335 raft_consensus.cc:359] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:14.092329  4335 raft_consensus.cc:740] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:14.093082  4335 consensus_queue.cc:260] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:14.096618  4335 ts_tablet_manager.cc:1434] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 12:22:14.108127  2915 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:14.110421  4219 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:14.243769  4341 tablet_bootstrap.cc:492] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:14.244954  4341 tablet_bootstrap.cc:492] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:14.245807  4341 ts_tablet_manager.cc:1403] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.177s	user 0.037s	sys 0.022s
I20260709 12:22:14.293640  3473 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:14.302618  4262 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:14.318588  3474 raft_consensus.cc:1275] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:14.249307  4341 raft_consensus.cc:359] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:14.344882  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:14.347569  2766 raft_consensus.cc:1275] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:14.346903  2778 tablet_copy_service.cc:342] P 442c92e5267046cd92f46e6f469570a5: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-7e4cde589c024472b0a4cec236ea69a3 received from {username='slave'} at 127.0.0.1:33574
I20260709 12:22:14.354363  2778 tablet_copy_service.cc:434] P 442c92e5267046cd92f46e6f469570a5: ending tablet copy session f440bbb3ddd545819242017140636ad4-7e4cde589c024472b0a4cec236ea69a3 on tablet 7e4cde589c024472b0a4cec236ea69a3 with peer f440bbb3ddd545819242017140636ad4
W20260709 12:22:14.357458  2809 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 71019.8us
I20260709 12:22:14.345415  4291 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:22:14.378652  2843 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:14.356926  4341 raft_consensus.cc:740] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:14.390836  2917 raft_consensus.cc:1275] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:14.405117  4341 consensus_queue.cc:260] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:14.407095  2843 raft_consensus.cc:1275] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:22:14.410130  4346 log.cc:927] Time spent T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:22:14.363221  4319 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:22:14.413890  4356 ts_tablet_manager.cc:933] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:14.407169  4351 ts_tablet_manager.cc:933] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:14.442092  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 83869us
I20260709 12:22:14.471184  4353 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:14.475907  3439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 59531.1us
I20260709 12:22:14.497330  2765 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:22:14.503400  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 52845.8us
I20260709 12:22:14.490316  4319 consensus_queue.cc:1048] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:22:14.526127  3473 raft_consensus.cc:1217] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:22:14.572316  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 65833.7us
I20260709 12:22:14.602970  4351 tablet_copy_client.cc:323] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:22:14.673502  4341 ts_tablet_manager.cc:1434] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.427s	user 0.028s	sys 0.011s
W20260709 12:22:14.824447  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 237380us
I20260709 12:22:14.812371  4356 tablet_copy_client.cc:323] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
W20260709 12:22:14.877416  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 118489us
I20260709 12:22:14.880084  3470 raft_consensus.cc:1217] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:14.879248  2852 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet 627b3359300e44fc9d2b83ed88272a9f from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
W20260709 12:22:14.937209  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 440233us
I20260709 12:22:14.865237  4348 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 12:22:14.988729  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 107472us
I20260709 12:22:14.982244  2852 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet 627b3359300e44fc9d2b83ed88272a9f from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-627b3359300e44fc9d2b83ed88272a9f
I20260709 12:22:15.041548  2915 consensus_queue.cc:237] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:15.042922  2916 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:15.043699  2916 raft_consensus.cc:606] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
W20260709 12:22:15.114475  2882 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:15.149695  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 132235us
W20260709 12:22:15.164049  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 157190us
W20260709 12:22:15.187044  2734 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 150621us
W20260709 12:22:15.181592  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 118412us
I20260709 12:22:15.182842  2927 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-eface8ee310347d4bb08e3c5864ce5d9 received from {username='slave'} at 127.0.0.1:59582
W20260709 12:22:15.190754  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 140505us
W20260709 12:22:15.348708  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 72344us
W20260709 12:22:15.241861  2734 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 49258.2us
I20260709 12:22:15.241489  2926 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 42af6798b62746449070b7b2bc4e0699 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
W20260709 12:22:15.352924  4395 log.cc:927] Time spent T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:22:15.421993  3472 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:22:15.377939  2852 tablet_copy_source_session.cc:215] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:15.429060  2838 raft_consensus.cc:1275] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:22:15.352572  4347 raft_consensus.cc:493] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:15.243479  4377 raft_consensus.cc:493] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:15.413687  2927 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-eface8ee310347d4bb08e3c5864ce5d9 on tablet eface8ee310347d4bb08e3c5864ce5d9 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:15.437038  4377 raft_consensus.cc:515] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:15.438426  4351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 627b3359300e44fc9d2b83ed88272a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.434651  4379 raft_consensus.cc:493] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f440bbb3ddd545819242017140636ad4)
I20260709 12:22:15.440374  4379 raft_consensus.cc:515] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:15.442675  4377 leader_election.cc:290] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:15.448678  2926 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 42af6798b62746449070b7b2bc4e0699 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-42af6798b62746449070b7b2bc4e0699
I20260709 12:22:15.450543  4379 leader_election.cc:290] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:15.453410  4401 raft_consensus.cc:493] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f440bbb3ddd545819242017140636ad4)
I20260709 12:22:15.454416  3471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:15.455598  3470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:15.444864  4347 raft_consensus.cc:515] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:15.454975  4401 raft_consensus.cc:515] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:15.463060  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc860efbfaf84207b7128fd001b59fe6" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:15.461971  4277 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:22:15.470427  4401 leader_election.cc:290] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:15.472746  2731 leader_election.cc:304] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4
I20260709 12:22:15.474571  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:15.477856  4401 raft_consensus.cc:2749] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:15.477936  4389 log.cc:927] Time spent T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: Append to log took a long time: real 0.487s	user 0.006s	sys 0.010s
I20260709 12:22:15.478108  3472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:15.479749  2734 leader_election.cc:304] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4
I20260709 12:22:15.480786  4401 raft_consensus.cc:2749] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:15.488720  4347 leader_election.cc:290] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:15.495173  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:15.496981  4401 raft_consensus.cc:493] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f440bbb3ddd545819242017140636ad4)
I20260709 12:22:15.498003  4401 raft_consensus.cc:515] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:15.517961  3476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:15.517874  4402 raft_consensus.cc:493] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:15.515925  4347 raft_consensus.cc:493] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f440bbb3ddd545819242017140636ad4)
I20260709 12:22:15.516616  4401 leader_election.cc:290] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:15.518945  4402 raft_consensus.cc:515] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:15.527756  4347 raft_consensus.cc:515] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:15.547839  4276 raft_consensus.cc:493] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:15.560755  4276 raft_consensus.cc:515] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:15.564251  4276 leader_election.cc:290] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:15.565981  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "442c92e5267046cd92f46e6f469570a5" is_pre_election: true
I20260709 12:22:15.566236  2838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:15.559974  4402 leader_election.cc:290] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:15.568495  4357 raft_consensus.cc:1064] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:15.570883  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:15.570828  4357 consensus_queue.cc:237] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:15.588117  4347 leader_election.cc:290] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
W20260709 12:22:15.589512  4405 log.cc:927] Time spent T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:22:15.589779  3476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:15.593624  4372 raft_consensus.cc:993] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:15.594204  4372 raft_consensus.cc:1081] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:15.596249  2926 tablet_copy_source_session.cc:215] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:15.599575  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff32c54020945d99bd67b7df9574bae" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:15.612783  2731 leader_election.cc:304] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4
I20260709 12:22:15.637804  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:15.630427  2880 leader_election.cc:304] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:15.649428  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:15.658635  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3545f99312f483eb6dc8965c5bc418b" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:15.630246  4219 raft_consensus.cc:2749] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:15.668170  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9961074150e45dbb36db8e49d840c5d" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:15.617372  4368 consensus_queue.cc:1048] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:22:15.662024  4372 raft_consensus.cc:2749] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:15.681815  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 48985.7us
W20260709 12:22:15.685887  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 47425.2us
I20260709 12:22:15.690448  2766 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:15.691082  2766 raft_consensus.cc:493] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.691448  2766 raft_consensus.cc:3060] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.708159  4353 raft_consensus.cc:1064] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:15.714386  2766 raft_consensus.cc:515] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:15.724134  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:15.725190  2766 leader_election.cc:290] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:15.726552  2732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.729454  4382 raft_consensus.cc:493] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.730175  4382 raft_consensus.cc:515] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:15.731648  2842 raft_consensus.cc:3060] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.757910  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:15.763226  2768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:15.763974  2768 raft_consensus.cc:606] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Received request to transfer leadership to TS 7cc87046f1414b27928722690a5e388f
I20260709 12:22:15.775241  2732 leader_election.cc:304] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:15.785466  2732 leader_election.cc:304] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4
I20260709 12:22:15.794060  4382 leader_election.cc:290] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:15.756525  3469 raft_consensus.cc:2955] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Committing config change with OpId 5.8: config changed from index 4 to 8, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:15.812649  4262 raft_consensus.cc:2955] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 5 LEADER]: Committing config change with OpId 5.8: config changed from index 4 to 8, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:15.830188  2836 raft_consensus.cc:1275] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:15.833354  2842 raft_consensus.cc:2468] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:15.837126  2841 raft_consensus.cc:2955] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Committing config change with OpId 5.8: config changed from index 4 to 8, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:15.850435  4378 raft_consensus.cc:2749] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:15.851346  2762 raft_consensus.cc:1240] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Rejecting Update request from peer 7cc87046f1414b27928722690a5e388f for earlier term 1. Current term is 2. Ops: []
W20260709 12:22:15.852123  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 66772.8us
I20260709 12:22:15.853130  2732 leader_election.cc:304] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4
I20260709 12:22:15.856756  2731 leader_election.cc:304] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:15.857007  2732 leader_election.cc:304] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:15.859977  4381 raft_consensus.cc:2749] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:15.861467  2913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:15.862097  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:15.862277  2913 raft_consensus.cc:3055] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:15.862795  2913 raft_consensus.cc:740] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:15.866470  2911 raft_consensus.cc:1275] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:15.866873  4291 consensus_queue.cc:1048] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:15.860765  4401 raft_consensus.cc:2749] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:15.868911  4219 raft_consensus.cc:2804] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:15.869436  4219 raft_consensus.cc:697] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:15.876536  2765 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:15.879781  2732 leader_election.cc:304] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:15.875239  2913 consensus_queue.cc:260] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:15.878633  2765 raft_consensus.cc:606] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Received request to transfer leadership to TS 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:15.852427  4353 consensus_queue.cc:237] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:15.860914  4357 raft_consensus.cc:2749] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:15.901233  2913 raft_consensus.cc:3060] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.907563  4380 raft_consensus.cc:2749] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:15.908541  4401 consensus_queue.cc:1048] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:22:15.913094  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 219930us
I20260709 12:22:15.915323  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:22:15.917342  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:15.918236  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:15.927027  4351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38432 (request call id 416) took 1663 ms. Trace:
I20260709 12:22:15.927619  4351 rpcz_store.cc:276] 0709 12:22:14.263913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.267200 (+  3287us) service_pool.cc:224] Handling call
0709 12:22:14.427981 (+160781us) ts_tablet_manager.cc:934] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
0709 12:22:15.436962 (+1008981us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:22:15.514323 (+ 77361us) tablet_metadata.cc:665] Metadata flushed
0709 12:22:15.514474 (+   151us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:22:15.876654 (+362180us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:22:15.905855 (+ 29201us) tablet_replica.cc:206] Creating consensus instance
0709 12:22:15.926870 (+ 21015us) ts_tablet_manager.cc:989] Replica registered.
0709 12:22:15.926933 (+    63us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":340,"tablet-copy.queue_time_us":97017,"thread_start_us":157920,"threads_started":2}
I20260709 12:22:15.870486  4219 consensus_queue.cc:237] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:15.933085  2913 raft_consensus.cc:2468] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:15.946830  3469 raft_consensus.cc:1275] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:15.953330  2841 raft_consensus.cc:1275] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:15.961786  4353 consensus_queue.cc:1048] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:22:15.958009  4351 tablet_copy_client.cc:806] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:15.964187  4351 tablet_copy_client.cc:670] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:15.983050  3473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "386a0166b36244f789cc8361a1cb0a46" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:16.002104  3476 raft_consensus.cc:1275] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:16.019264  4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42af6798b62746449070b7b2bc4e0699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.046414  2766 raft_consensus.cc:1275] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:16.008512  4362 consensus_queue.cc:1048] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 12:22:16.038548  2666 catalog_manager.cc:5697] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 4 to 8, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 5 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.102000  4351 tablet_copy_client.cc:538] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:16.102207  4404 consensus_queue.cc:1048] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:22:16.117877  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 121891us
I20260709 12:22:16.106324  4262 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:47626 (request call id 85) took 1055 ms. Trace:
I20260709 12:22:16.134919  4402 consensus_queue.cc:1048] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:22:16.137313  4262 rpcz_store.cc:276] 0709 12:22:15.037785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.038080 (+   295us) service_pool.cc:224] Handling call
0709 12:22:15.048321 (+ 10241us) log.cc:844] Serialized 494 byte log entry
0709 12:22:15.370746 (+322425us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b1800342d60. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:22:16.093084 (+722338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1274,"raft.queue_time_us":14268,"raft.run_cpu_time_us":1525,"raft.run_wall_time_us":57800,"spinlock_wait_cycles":5510400}
I20260709 12:22:16.160012  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 386a0166b36244f789cc8361a1cb0a46 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
W20260709 12:22:16.216241  4417 log.cc:927] Time spent T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f: Append to log took a long time: real 0.193s	user 0.000s	sys 0.002s
I20260709 12:22:16.217826  4351 tablet_bootstrap.cc:492] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:16.229739  4353 raft_consensus.cc:2955] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.231580  2837 raft_consensus.cc:2955] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.234750  2917 raft_consensus.cc:2955] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.225425  4380 raft_consensus.cc:2955] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.260300  4219 raft_consensus.cc:993] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:16.260938  4219 raft_consensus.cc:1081] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Signalling peer 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:16.251799  2762 raft_consensus.cc:2955] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.271945  4356 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38410 (request call id 536) took 1965 ms. Trace:
I20260709 12:22:16.272384  4356 rpcz_store.cc:276] 0709 12:22:14.306367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.306549 (+   182us) service_pool.cc:224] Handling call
0709 12:22:14.414281 (+107732us) ts_tablet_manager.cc:934] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
0709 12:22:15.994679 (+1580398us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:22:16.168261 (+173582us) tablet_metadata.cc:665] Metadata flushed
0709 12:22:16.168362 (+   101us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:22:16.271179 (+102817us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:22:16.271485 (+   306us) tablet_replica.cc:206] Creating consensus instance
0709 12:22:16.271816 (+   331us) ts_tablet_manager.cc:989] Replica registered.
0709 12:22:16.271880 (+    64us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":109477,"tablet-copy.queue_time_us":101551,"thread_start_us":339518,"threads_started":2}
I20260709 12:22:16.275206  2841 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:42536
I20260709 12:22:16.293915  4356 tablet_copy_client.cc:806] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:16.293993  2841 raft_consensus.cc:493] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:16.294633  2841 raft_consensus.cc:3060] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.296378  4356 tablet_copy_client.cc:670] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:16.276103  4383 raft_consensus.cc:993] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:16.301993  4383 raft_consensus.cc:1081] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Signalling peer 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:16.302812  4356 tablet_copy_client.cc:538] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:16.285209  2663 catalog_manager.cc:5697] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:16.306090  2916 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:47638
I20260709 12:22:16.306761  2916 raft_consensus.cc:493] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:16.307170  2916 raft_consensus.cc:3060] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:16.312608  2841 raft_consensus.cc:515] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:22:16.316298  2841 leader_election.cc:290] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:16.321070  2916 raft_consensus.cc:515] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:16.323729  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:16.324339  2765 raft_consensus.cc:3055] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:16.324626  2765 raft_consensus.cc:740] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:16.325451  2765 consensus_queue.cc:260] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:22:16.326365  2765 raft_consensus.cc:3060] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.327929  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26ca468f5384641ad5bd05af362aef9" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:16.328576  2917 raft_consensus.cc:3060] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.341781  2916 leader_election.cc:290] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:16.343381  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:16.344091  2768 raft_consensus.cc:3055] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:16.344420  2768 raft_consensus.cc:740] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:16.345319  2765 raft_consensus.cc:2468] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 2.
I20260709 12:22:16.345194  2768 consensus_queue.cc:260] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:16.346700  2768 raft_consensus.cc:3060] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:16.351935  4356 tablet_bootstrap.cc:492] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:16.372944  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet 386a0166b36244f789cc8361a1cb0a46 with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:38926
I20260709 12:22:16.375092  2666 catalog_manager.cc:5697] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.381733  2807 leader_election.cc:304] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:16.382356  3473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cba50a5280d4b00b9d2617ce7a22ec1" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:16.383026  3473 raft_consensus.cc:3060] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:16.383262  4279 raft_consensus.cc:2804] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:16.383750  4279 raft_consensus.cc:697] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 2 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:16.384611  4279 consensus_queue.cc:237] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } }
I20260709 12:22:16.375108  4430 tablet_replica.cc:333] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:16.386713  4430 raft_consensus.cc:2243] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:16.387578  4430 raft_consensus.cc:2272] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:16.390514  4430 ts_tablet_manager.cc:1916] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:16.392824  3473 raft_consensus.cc:2468] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
I20260709 12:22:16.407361  2883 leader_election.cc:304] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:16.408370  4372 raft_consensus.cc:2804] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:16.408898  4372 raft_consensus.cc:697] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:16.409840  4372 consensus_queue.cc:237] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:16.405138  4351 tablet_bootstrap.cc:492] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:16.411749  4351 tablet_bootstrap.cc:492] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:16.418938  4351 ts_tablet_manager.cc:1403] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.203s	user 0.023s	sys 0.012s
I20260709 12:22:16.430809  4351 raft_consensus.cc:359] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:16.431969  2910 raft_consensus.cc:1275] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:16.433792  3473 raft_consensus.cc:2955] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.435128  4351 raft_consensus.cc:740] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:16.436054  4351 consensus_queue.cc:260] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:16.439085  4351 ts_tablet_manager.cc:1434] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.020s	user 0.001s	sys 0.005s
I20260709 12:22:16.442770  2664 catalog_manager.cc:5697] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.431510  3476 raft_consensus.cc:2955] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.447294  4357 consensus_queue.cc:1048] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:16.458688  2841 raft_consensus.cc:2955] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.477784  4356 tablet_bootstrap.cc:492] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:16.486650  4356 tablet_bootstrap.cc:492] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:16.487425  2852 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-627b3359300e44fc9d2b83ed88272a9f received from {username='slave'} at 127.0.0.1:38132
I20260709 12:22:16.497025  2852 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-627b3359300e44fc9d2b83ed88272a9f on tablet 627b3359300e44fc9d2b83ed88272a9f with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:16.491048  4430 ts_tablet_manager.cc:1929] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.6
I20260709 12:22:16.491110  2768 raft_consensus.cc:2468] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
I20260709 12:22:16.506004  4430 log.cc:1199] T 386a0166b36244f789cc8361a1cb0a46 P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/386a0166b36244f789cc8361a1cb0a46
I20260709 12:22:16.506904  2663 catalog_manager.cc:5697] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 1 to 2, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:16.510835  2837 raft_consensus.cc:1275] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:16.511528  4356 ts_tablet_manager.cc:1403] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.160s	user 0.025s	sys 0.008s
I20260709 12:22:16.514096  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet 386a0166b36244f789cc8361a1cb0a46 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:16.517437  4356 raft_consensus.cc:359] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:16.518406  4356 raft_consensus.cc:740] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:16.519142  4356 consensus_queue.cc:260] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:16.522737  4357 consensus_queue.cc:1048] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:16.546492  2917 raft_consensus.cc:2468] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 2.
I20260709 12:22:16.562590  2766 consensus_queue.cc:237] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:16.551338  4356 ts_tablet_manager.cc:1434] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.038s	user 0.001s	sys 0.007s
I20260709 12:22:16.566931  2926 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-42af6798b62746449070b7b2bc4e0699 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:22:16.567387  2926 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-42af6798b62746449070b7b2bc4e0699 on tablet 42af6798b62746449070b7b2bc4e0699 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:16.568471  3469 raft_consensus.cc:1275] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:16.571242  4401 consensus_queue.cc:1048] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:16.593619  2917 raft_consensus.cc:1275] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:16.597033  4380 consensus_queue.cc:1048] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:16.620059  2663 catalog_manager.cc:5697] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:16.634438  4357 raft_consensus.cc:2955] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.639312  2917 raft_consensus.cc:2955] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.659991  3473 raft_consensus.cc:2955] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.661281  2917 consensus_queue.cc:237] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:16.672668  2843 raft_consensus.cc:1275] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:16.676868  3474 raft_consensus.cc:1217] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 12:22:16.685700  4277 consensus_queue.cc:1048] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:22:16.717072  2841 raft_consensus.cc:2955] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.722285  3469 raft_consensus.cc:1275] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:16.730357  2666 catalog_manager.cc:5697] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.733328  4353 consensus_queue.cc:1048] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:22:16.723655  4372 raft_consensus.cc:2955] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.743487  3469 raft_consensus.cc:2955] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.787178  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e4cde589c024472b0a4cec236ea69a3 with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:22:16.805209  2662 catalog_manager.cc:5697] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:16.826649  4441 tablet_replica.cc:333] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:16.826840  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7e4cde589c024472b0a4cec236ea69a3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:16.847611  4441 raft_consensus.cc:2243] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:16.848587  4441 raft_consensus.cc:2272] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:16.863196  2768 raft_consensus.cc:1275] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:16.868485  4277 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:16.863196  3469 raft_consensus.cc:1275] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:16.874542  4441 ts_tablet_manager.cc:1916] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:16.876039  4372 consensus_queue.cc:1048] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:16.879487  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet eface8ee310347d4bb08e3c5864ce5d9 with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:22:16.880514  4430 tablet_replica.cc:333] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:16.881587  4430 raft_consensus.cc:2243] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:16.882404  4430 raft_consensus.cc:2272] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:16.881593  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eface8ee310347d4bb08e3c5864ce5d9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:16.885908  4430 ts_tablet_manager.cc:1916] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:16.909946  2917 raft_consensus.cc:1275] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:16.915495  4400 consensus_queue.cc:1048] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:16.975339  2766 raft_consensus.cc:1275] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:16.981215  4400 consensus_queue.cc:1048] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:17.006165  4441 ts_tablet_manager.cc:1929] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:17.006768  4441 log.cc:1199] T 7e4cde589c024472b0a4cec236ea69a3 P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/7e4cde589c024472b0a4cec236ea69a3
I20260709 12:22:17.008788  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet 7e4cde589c024472b0a4cec236ea69a3 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:17.023181  4430 ts_tablet_manager.cc:1929] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:17.037715  4430 log.cc:1199] T eface8ee310347d4bb08e3c5864ce5d9 P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/eface8ee310347d4bb08e3c5864ce5d9
I20260709 12:22:17.039678  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet eface8ee310347d4bb08e3c5864ce5d9 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:17.062930  3469 raft_consensus.cc:1217] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:17.235324  4439 raft_consensus.cc:1064] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:17.238456  4439 consensus_queue.cc:237] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:17.247939  3469 raft_consensus.cc:1275] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:17.249104  2910 raft_consensus.cc:1275] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:17.249639  4392 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:17.249936  2766 raft_consensus.cc:1275] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:17.251037  4400 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:22:17.255823  4439 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:17.274477  4400 raft_consensus.cc:2955] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:17.281852  2766 raft_consensus.cc:2955] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:17.287168  2910 raft_consensus.cc:2955] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:17.298235  3469 raft_consensus.cc:2955] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:17.311538  2663 catalog_manager.cc:5697] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:17.384157  2841 consensus_queue.cc:237] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:17.399241  2766 raft_consensus.cc:1275] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:17.402097  4399 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:17.410912  3469 raft_consensus.cc:1275] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:17.414956  4399 consensus_queue.cc:1048] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:17.426182  4392 raft_consensus.cc:2955] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:17.433558  2766 raft_consensus.cc:2955] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:17.438457  3469 raft_consensus.cc:2955] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:17.441763  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 627b3359300e44fc9d2b83ed88272a9f with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:17.465476  2662 catalog_manager.cc:5697] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 5 to 6, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 3 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:17.477923  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet 627b3359300e44fc9d2b83ed88272a9f with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47626
I20260709 12:22:17.491222  4453 tablet_replica.cc:333] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:17.496498  4453 raft_consensus.cc:2243] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:17.497620  4453 raft_consensus.cc:2272] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:17.500785  4453 ts_tablet_manager.cc:1916] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:17.523628  4453 ts_tablet_manager.cc:1929] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:22:17.524103  4453 log.cc:1199] T 627b3359300e44fc9d2b83ed88272a9f P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/627b3359300e44fc9d2b83ed88272a9f
I20260709 12:22:17.561695  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet 627b3359300e44fc9d2b83ed88272a9f (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:17.580227  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:22:17.852775  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 97044.9us
I20260709 12:22:17.860330  4372 raft_consensus.cc:1064] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:17.868140  4372 consensus_queue.cc:237] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:17.911238  3473 raft_consensus.cc:1275] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:17.912320  2762 raft_consensus.cc:1275] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:17.914353  4460 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:17.916734  4462 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:17.930402  2841 raft_consensus.cc:1275] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:17.943964  4462 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:17.983122  4443 raft_consensus.cc:2955] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:17.985159  2841 raft_consensus.cc:2955] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:17.991091  2910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:17.991900  2910 raft_consensus.cc:606] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 4 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:17.998178  2768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:17.998986  2768 raft_consensus.cc:606] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Received request to transfer leadership to TS 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:18.001258  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:18.004742  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:18.003904  2762 raft_consensus.cc:2955] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:18.006341  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:18.026945  3473 raft_consensus.cc:2955] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:18.052086  2663 catalog_manager.cc:5697] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:18.107486  2910 consensus_queue.cc:237] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:18.122861  3473 raft_consensus.cc:1275] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:18.125576  4459 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:18.127084  2762 raft_consensus.cc:1275] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:18.140576  4459 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:22:18.146447  4443 raft_consensus.cc:2955] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:18.151659  3469 raft_consensus.cc:2955] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:18.156165  2768 raft_consensus.cc:2955] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:18.168493  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 42af6798b62746449070b7b2bc4e0699 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:18.184268  2662 catalog_manager.cc:5697] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:18.199568  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet 42af6798b62746449070b7b2bc4e0699 with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:22:18.206580  4470 tablet_replica.cc:333] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:18.215152  4470 raft_consensus.cc:2243] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:18.215873  4470 raft_consensus.cc:2272] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:18.219043  4470 ts_tablet_manager.cc:1916] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:18.233868  4470 ts_tablet_manager.cc:1929] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:18.234393  4470 log.cc:1199] T 42af6798b62746449070b7b2bc4e0699 P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/42af6798b62746449070b7b2bc4e0699
I20260709 12:22:18.236421  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet 42af6798b62746449070b7b2bc4e0699 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:18.243927  4462 raft_consensus.cc:993] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:18.244549  4443 raft_consensus.cc:1081] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 4 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:18.246213  2768 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:18.246791  2768 raft_consensus.cc:493] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.247104  2768 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:18.255748  2768 raft_consensus.cc:515] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:18.258769  2768 leader_election.cc:290] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 5 election: Requested vote from peers 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:18.259073  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:18.259572  3469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:18.260005  2917 raft_consensus.cc:3055] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:18.260308  3469 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:18.260419  2917 raft_consensus.cc:740] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:18.261504  2917 consensus_queue.cc:260] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:18.262859  2917 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:18.269368  3469 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 5.
I20260709 12:22:18.271466  2734 leader_election.cc:304] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:18.272646  4456 raft_consensus.cc:2804] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:18.273221  4456 raft_consensus.cc:697] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 5 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:18.274334  4456 consensus_queue.cc:237] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:18.273191  2917 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 5.
I20260709 12:22:18.288210  2664 catalog_manager.cc:5697] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 4 to 5, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 5 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.309137  4383 raft_consensus.cc:993] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:18.309783  4383 raft_consensus.cc:1081] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Signalling peer 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:18.311398  2841 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:42536
I20260709 12:22:18.311985  2841 raft_consensus.cc:493] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.312260  2841 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.317790  2841 raft_consensus.cc:515] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:18.320245  2841 leader_election.cc:290] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:18.321208  2768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:18.321473  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:18.321950  2768 raft_consensus.cc:3055] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:18.322098  2917 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.322337  2768 raft_consensus.cc:740] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:18.323387  2768 consensus_queue.cc:260] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:18.324741  2768 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.332485  2917 raft_consensus.cc:2468] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 3.
I20260709 12:22:18.333907  2807 leader_election.cc:304] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:18.334347  2768 raft_consensus.cc:2468] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 3.
I20260709 12:22:18.334820  4399 raft_consensus.cc:2804] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:18.335358  4399 raft_consensus.cc:697] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:18.336248  4399 consensus_queue.cc:237] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:18.348384  2664 catalog_manager.cc:5697] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.769498  2762 raft_consensus.cc:1275] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:18.771082  4400 consensus_queue.cc:1048] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:18.815716  2910 raft_consensus.cc:1275] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:18.819746  4399 consensus_queue.cc:1048] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:18.821388  3473 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:22:18.822103  2910 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:22:18.822911  4383 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:18.824541  4456 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:22:19.130023  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:19.173055  2762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:19.173684  2762 raft_consensus.cc:606] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Received request to transfer leadership to TS 7cc87046f1414b27928722690a5e388f
I20260709 12:22:19.176532  2843 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:36554
I20260709 12:22:19.177172  2843 raft_consensus.cc:606] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:19.178640  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:19.180696  4456 raft_consensus.cc:993] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:19.180792  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:19.181380  4383 raft_consensus.cc:1081] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Signalling peer 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:19.182166  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:19.183050  2917 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:47638
I20260709 12:22:19.183583  2917 raft_consensus.cc:493] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:19.183888  2917 raft_consensus.cc:3060] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.190850  2917 raft_consensus.cc:515] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:19.194024  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:19.194710  2762 raft_consensus.cc:3055] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.195044  2762 raft_consensus.cc:740] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:19.195816  2762 consensus_queue.cc:260] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:19.196993  2762 raft_consensus.cc:3060] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.222388  2762 raft_consensus.cc:2468] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
I20260709 12:22:19.223744  2881 leader_election.cc:304] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:19.224731  4473 raft_consensus.cc:2804] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:19.226094  4473 raft_consensus.cc:697] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:19.227100  4473 consensus_queue.cc:237] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:19.227494  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf19d36b4d1b4825a45f6749aaa19896" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:19.227964  2917 leader_election.cc:290] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:19.228436  2843 raft_consensus.cc:3060] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.251411  4478 raft_consensus.cc:993] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:19.252111  4478 raft_consensus.cc:1081] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:19.252110  2843 raft_consensus.cc:2468] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
I20260709 12:22:19.251641  2664 catalog_manager.cc:5697] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:19.254968  2766 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38956
I20260709 12:22:19.255558  2766 raft_consensus.cc:493] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:19.255913  2766 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:19.266449  2766 raft_consensus.cc:515] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:19.269171  2766 leader_election.cc:290] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 4 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:19.270813  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:19.271492  2917 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:19.289719  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:19.290499  2843 raft_consensus.cc:3055] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:19.290823  2843 raft_consensus.cc:740] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:19.293339  2843 consensus_queue.cc:260] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:19.296790  2917 raft_consensus.cc:2468] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 4.
I20260709 12:22:19.298080  2843 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:19.300696  2910 raft_consensus.cc:1240] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Rejecting Update request from peer 9a10672cd53140d59ff945f75345cc06 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:19.319466  2732 leader_election.cc:304] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:19.320802  4456 raft_consensus.cc:2804] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:19.313019  2843 raft_consensus.cc:2468] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 4.
I20260709 12:22:19.328162  4456 raft_consensus.cc:697] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 4 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:19.330695  4456 consensus_queue.cc:237] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:19.356041  2766 consensus_queue.cc:237] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:19.357950  2662 catalog_manager.cc:5697] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 3 to 4, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 4 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:19.364634  2910 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 12:22:19.366015  2734 consensus_peers.cc:597] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 62274aac902146f7b127a2afb55b8a12. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:19.366358  2843 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:19.367151  4383 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:19.368999  4484 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:19.378827  4383 raft_consensus.cc:2955] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.383849  2843 raft_consensus.cc:2955] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.382391  2910 raft_consensus.cc:2955] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.404752  2662 catalog_manager.cc:5697] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 3 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:19.398880  2836 consensus_queue.cc:237] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
W20260709 12:22:19.425993  2809 consensus_peers.cc:597] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f26ca468f5384641ad5bd05af362aef9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:19.428310  2910 raft_consensus.cc:1275] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:19.429952  2766 raft_consensus.cc:1275] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:19.430055  4477 consensus_queue.cc:1048] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:19.435034  4477 consensus_queue.cc:1048] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:19.439909  4399 raft_consensus.cc:2955] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.445073  2910 raft_consensus.cc:2955] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.460395  2917 consensus_queue.cc:237] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:19.462633  2766 raft_consensus.cc:2955] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.464144  2662 catalog_manager.cc:5697] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 3, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 2 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:19.473253  2883 consensus_peers.cc:597] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12034fdcbbed4121a134d5d26c1dc5ac. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:19.477020  2836 raft_consensus.cc:1275] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:19.477579  2762 raft_consensus.cc:1275] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:19.480947  4443 consensus_queue.cc:1048] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:19.490979  4491 consensus_queue.cc:1048] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:19.499931  4462 raft_consensus.cc:2955] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.515141  2766 raft_consensus.cc:2955] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.523965  2910 consensus_queue.cc:237] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:19.534448  2666 catalog_manager.cc:5697] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:19.536633  2836 raft_consensus.cc:2955] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.559098  2843 raft_consensus.cc:1275] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:19.561056  4473 consensus_queue.cc:1048] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:19.570462  2766 raft_consensus.cc:1275] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:19.572772  4443 consensus_queue.cc:1048] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:22:19.584918  2883 consensus_peers.cc:597] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6291f8f287034586b4e9c0c9220bc6e7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:19.586493  4473 raft_consensus.cc:2955] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.590900  2836 raft_consensus.cc:2955] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.609714  2762 raft_consensus.cc:2955] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.609912  2666 catalog_manager.cc:5697] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:19.710106  2762 raft_consensus.cc:1275] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.711560  4473 consensus_queue.cc:1048] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:19.731628  2836 raft_consensus.cc:1275] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.737519  4443 consensus_queue.cc:1048] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:19.807935  2837 raft_consensus.cc:1275] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:19.809302  4484 consensus_queue.cc:1048] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:19.825965  2910 raft_consensus.cc:1275] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:19.827597  4484 consensus_queue.cc:1048] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:19.984382  4512 ts_tablet_manager.cc:933] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:22:19.986392  4512 tablet_copy_client.cc:323] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.65:35625
I20260709 12:22:19.989759  2778 tablet_copy_service.cc:140] P 442c92e5267046cd92f46e6f469570a5: Received BeginTabletCopySession request for tablet 62274aac902146f7b127a2afb55b8a12 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:33574)
I20260709 12:22:19.990377  2778 tablet_copy_service.cc:161] P 442c92e5267046cd92f46e6f469570a5: Beginning new tablet copy session on tablet 62274aac902146f7b127a2afb55b8a12 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:33574: session id = f440bbb3ddd545819242017140636ad4-62274aac902146f7b127a2afb55b8a12
I20260709 12:22:20.001802  2778 tablet_copy_source_session.cc:215] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:20.005976  4512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62274aac902146f7b127a2afb55b8a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:20.025671  4512 tablet_copy_client.cc:806] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:20.026436  4512 tablet_copy_client.cc:670] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:20.031848  4512 tablet_copy_client.cc:538] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:20.047227  4512 tablet_bootstrap.cc:492] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:20.061280  4514 ts_tablet_manager.cc:933] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:20.083753  4512 tablet_bootstrap.cc:492] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:20.084872  4512 tablet_bootstrap.cc:492] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:20.086254  4512 ts_tablet_manager.cc:1403] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.039s	user 0.023s	sys 0.014s
I20260709 12:22:20.089689  4512 raft_consensus.cc:359] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:20.090571  4512 raft_consensus.cc:740] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:20.091272  4512 consensus_queue.cc:260] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:20.095922  4512 ts_tablet_manager.cc:1434] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 12:22:20.096714  4517 ts_tablet_manager.cc:933] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:20.083909  4514 tablet_copy_client.cc:323] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:22:20.101756  2778 tablet_copy_service.cc:342] P 442c92e5267046cd92f46e6f469570a5: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-62274aac902146f7b127a2afb55b8a12 received from {username='slave'} at 127.0.0.1:33574
I20260709 12:22:20.101802  2852 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet f26ca468f5384641ad5bd05af362aef9 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:22:20.102977  2852 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet f26ca468f5384641ad5bd05af362aef9 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-f26ca468f5384641ad5bd05af362aef9
I20260709 12:22:20.103549  4517 tablet_copy_client.cc:323] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:22:20.102386  2778 tablet_copy_service.cc:434] P 442c92e5267046cd92f46e6f469570a5: ending tablet copy session f440bbb3ddd545819242017140636ad4-62274aac902146f7b127a2afb55b8a12 on tablet 62274aac902146f7b127a2afb55b8a12 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:20.105839  2926 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 12034fdcbbed4121a134d5d26c1dc5ac from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:22:20.106567  2926 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 12034fdcbbed4121a134d5d26c1dc5ac from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-12034fdcbbed4121a134d5d26c1dc5ac
I20260709 12:22:20.115085  2852 tablet_copy_source_session.cc:215] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:20.119185  4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26ca468f5384641ad5bd05af362aef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:20.132503  2926 tablet_copy_source_session.cc:215] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:20.136610  4512 ts_tablet_manager.cc:933] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:20.150924  4514 tablet_copy_client.cc:806] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:20.151666  4514 tablet_copy_client.cc:670] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:20.151770  4517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12034fdcbbed4121a134d5d26c1dc5ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:20.157125  4512 tablet_copy_client.cc:323] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:22:20.159365  2926 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 6291f8f287034586b4e9c0c9220bc6e7 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:22:20.159941  2926 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 6291f8f287034586b4e9c0c9220bc6e7 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-6291f8f287034586b4e9c0c9220bc6e7
I20260709 12:22:20.161945  4514 tablet_copy_client.cc:538] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:20.204741  2926 tablet_copy_source_session.cc:215] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:20.212009  4512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6291f8f287034586b4e9c0c9220bc6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:20.220619  4514 tablet_bootstrap.cc:492] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:20.223055  4517 tablet_copy_client.cc:806] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:20.223729  4517 tablet_copy_client.cc:670] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:20.228948  4517 tablet_copy_client.cc:538] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:20.232585  4512 tablet_copy_client.cc:806] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:20.233596  4512 tablet_copy_client.cc:670] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:20.241586  4512 tablet_copy_client.cc:538] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:20.242513  4517 tablet_bootstrap.cc:492] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:20.251762  4514 tablet_bootstrap.cc:492] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:20.252810  4514 tablet_bootstrap.cc:492] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:20.253618  4514 ts_tablet_manager.cc:1403] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.033s	user 0.024s	sys 0.004s
I20260709 12:22:20.255235  4512 tablet_bootstrap.cc:492] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:20.256981  4514 raft_consensus.cc:359] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:20.257984  4514 raft_consensus.cc:740] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:20.293352  4517 tablet_bootstrap.cc:492] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:20.294461  4517 tablet_bootstrap.cc:492] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:20.295272  4517 ts_tablet_manager.cc:1403] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.054s	user 0.022s	sys 0.008s
I20260709 12:22:20.290375  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:20.258731  4514 consensus_queue.cc:260] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:20.298743  4517 raft_consensus.cc:359] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:20.299739  4517 raft_consensus.cc:740] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:20.300451  4517 consensus_queue.cc:260] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:20.302142  4514 ts_tablet_manager.cc:1434] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 12:22:20.303544  4517 ts_tablet_manager.cc:1434] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:20.305044  2852 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-f26ca468f5384641ad5bd05af362aef9 received from {username='slave'} at 127.0.0.1:38132
I20260709 12:22:20.305748  2852 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-f26ca468f5384641ad5bd05af362aef9 on tablet f26ca468f5384641ad5bd05af362aef9 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:20.306787  2926 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-12034fdcbbed4121a134d5d26c1dc5ac received from {username='slave'} at 127.0.0.1:59582
I20260709 12:22:20.307317  2926 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-12034fdcbbed4121a134d5d26c1dc5ac on tablet 12034fdcbbed4121a134d5d26c1dc5ac with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:20.335835  4512 tablet_bootstrap.cc:492] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:20.336875  4512 tablet_bootstrap.cc:492] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:20.337669  4512 ts_tablet_manager.cc:1403] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.083s	user 0.030s	sys 0.016s
I20260709 12:22:20.343014  4512 raft_consensus.cc:359] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:20.344956  4512 raft_consensus.cc:740] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:20.355049  4512 consensus_queue.cc:260] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:20.358484  4512 ts_tablet_manager.cc:1434] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 12:22:20.363144  2926 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-6291f8f287034586b4e9c0c9220bc6e7 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:22:20.363677  2926 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-6291f8f287034586b4e9c0c9220bc6e7 on tablet 6291f8f287034586b4e9c0c9220bc6e7 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:20.392699  2917 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:20.393399  2917 raft_consensus.cc:606] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:20.396776  2766 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:20.397452  2766 raft_consensus.cc:606] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Received request to transfer leadership to TS 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:20.402387  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:20.404282  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:20.405045  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:20.490869  3474 raft_consensus.cc:1217] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:20.515985  3474 raft_consensus.cc:1217] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 12:22:20.572072  3474 raft_consensus.cc:1217] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:22:20.738940  3474 raft_consensus.cc:1217] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:20.813735  4484 raft_consensus.cc:993] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:20.814333  4484 raft_consensus.cc:1081] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Signalling peer 9a10672cd53140d59ff945f75345cc06 to start an election
I20260709 12:22:20.817700  2837 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
 from {username='slave'} at 127.0.0.1:42536
I20260709 12:22:20.818272  2837 raft_consensus.cc:493] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:20.818698  2837 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.827105  2837 raft_consensus.cc:515] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:20.828881  2837 leader_election.cc:290] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 4 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:20.829880  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:20.830034  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:20.830621  2910 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.830797  2762 raft_consensus.cc:3055] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:20.831172  2762 raft_consensus.cc:740] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:20.831962  2762 consensus_queue.cc:260] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:20.833315  2762 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.838908  2910 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 4.
I20260709 12:22:20.839756  2762 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 4.
I20260709 12:22:20.841390  2807 leader_election.cc:304] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:20.842250  4499 raft_consensus.cc:2804] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:20.842734  4499 raft_consensus.cc:697] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:20.843803  4499 consensus_queue.cc:237] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:20.861331  2664 catalog_manager.cc:5697] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 3 to 4, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 4 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:20.889410  4462 raft_consensus.cc:993] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:20.890475  4462 raft_consensus.cc:1081] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:20.892210  2766 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:20.892802  2766 raft_consensus.cc:493] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:20.893198  2766 raft_consensus.cc:3060] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:20.901974  2766 raft_consensus.cc:515] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:20.904232  2766 leader_election.cc:290] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:20.904989  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:20.905699  2917 raft_consensus.cc:3055] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:20.906011  2917 raft_consensus.cc:740] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:20.906260  3474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:20.906864  3474 raft_consensus.cc:3060] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:20.906821  2917 consensus_queue.cc:260] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:20.908324  2917 raft_consensus.cc:3060] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:20.926796  2917 raft_consensus.cc:2468] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:20.930487  2732 leader_election.cc:304] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:20.931339  4486 raft_consensus.cc:2804] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:20.931845  4486 raft_consensus.cc:697] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:20.932785  4486 consensus_queue.cc:237] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:20.939478  3474 raft_consensus.cc:2468] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:20.957713  2664 catalog_manager.cc:5697] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:21.069638  4478 raft_consensus.cc:1064] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:21.072449  4478 consensus_queue.cc:237] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:21.081096  3474 raft_consensus.cc:1275] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:22:21.084450  4499 consensus_queue.cc:1048] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:21.083261  2917 raft_consensus.cc:1275] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:22:21.092448  4530 consensus_queue.cc:1048] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:21.093218  4462 raft_consensus.cc:1064] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:21.098531  2768 raft_consensus.cc:1275] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:22:21.096581  4462 consensus_queue.cc:237] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:21.102346  4399 consensus_queue.cc:1048] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:21.131383  4443 raft_consensus.cc:1064] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:21.134684  4443 consensus_queue.cc:237] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:21.139597  2837 raft_consensus.cc:1275] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:21.140779  3474 raft_consensus.cc:1275] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:21.141471  4462 consensus_queue.cc:1048] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:21.143198  4473 consensus_queue.cc:1048] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:21.147898  2766 raft_consensus.cc:1275] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:21.149622  4473 consensus_queue.cc:1048] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:21.201320  3473 raft_consensus.cc:1275] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:21.211719  4399 raft_consensus.cc:2955] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.216333  4473 consensus_queue.cc:1048] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:22:21.232197  3474 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Advancing to term 4
I20260709 12:22:21.236305  2762 raft_consensus.cc:1275] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:21.235591  3473 raft_consensus.cc:2955] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.239252  4443 consensus_queue.cc:1048] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:21.241578  2768 raft_consensus.cc:2955] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.243337  2762 raft_consensus.cc:2955] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.240659  3474 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 4 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:22:21.247103  4499 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:21.247529  2765 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:22:21.252995  4499 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:21.258039  2837 raft_consensus.cc:1275] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:21.252526  4540 raft_consensus.cc:2955] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.264333  4473 consensus_queue.cc:1048] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:22:21.280174  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 67107.2us
I20260709 12:22:21.286144  3472 raft_consensus.cc:2955] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.314126  4478 raft_consensus.cc:1064] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
W20260709 12:22:21.314625  4478 raft_consensus.cc:1068] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06: Unable to promote non-voter f440bbb3ddd545819242017140636ad4: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:22:21.321700  2664 catalog_manager.cc:5697] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.329339  2666 catalog_manager.cc:5697] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:21.338850  2837 raft_consensus.cc:2955] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.393273  2837 consensus_queue.cc:237] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:21.399423  2917 consensus_queue.cc:237] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:21.405055  2910 raft_consensus.cc:2955] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.412729  3476 raft_consensus.cc:1275] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:21.414243  2843 raft_consensus.cc:1275] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:21.415876  4443 consensus_queue.cc:1048] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:21.417379  4542 log.cc:927] Time spent T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:22:21.419828  4443 consensus_queue.cc:1048] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:21.433501  4541 raft_consensus.cc:2955] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.444213  3474 raft_consensus.cc:2955] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.445833  4540 raft_consensus.cc:2955] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.447641  3475 raft_consensus.cc:2955] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.459746  2765 raft_consensus.cc:1275] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:21.463320  4399 consensus_queue.cc:1048] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:21.467104  3471 raft_consensus.cc:1275] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:21.474941  4486 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:21.486078  4399 raft_consensus.cc:1064] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:21.488298  4478 raft_consensus.cc:2955] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.505949  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 12034fdcbbed4121a134d5d26c1dc5ac with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:21.513636  2837 raft_consensus.cc:2955] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.458698  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:21.518446  3469 raft_consensus.cc:1275] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:21.511468  4399 consensus_queue.cc:237] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:21.517133  2765 raft_consensus.cc:2955] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.542321  2910 raft_consensus.cc:1275] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:21.545426  4530 consensus_queue.cc:1048] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:21.551873  3473 raft_consensus.cc:2955] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.555225  2663 catalog_manager.cc:5697] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.558894  2766 raft_consensus.cc:2955] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.574177  2910 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:22:21.629302  2843 raft_consensus.cc:2955] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.649080  3473 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 4 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:21.651659  4557 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:21.659068  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f26ca468f5384641ad5bd05af362aef9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:22:21.675549  2732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 96685.5us
W20260709 12:22:21.676313  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 116511us
I20260709 12:22:21.706384  2666 catalog_manager.cc:5697] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 4 to 5, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 2 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:21.715890  2913 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:21.727957  4530 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:21.729506  2837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12"
dest_uuid: "9a10672cd53140d59ff945f75345cc06"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:36554
I20260709 12:22:21.730206  2837 raft_consensus.cc:606] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:21.731130  4531 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:21.734752  2664 catalog_manager.cc:5697] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:21.737416  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet f26ca468f5384641ad5bd05af362aef9 with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47626
I20260709 12:22:21.766165  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet 12034fdcbbed4121a134d5d26c1dc5ac with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:22:21.771173  2910 consensus_queue.cc:237] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:21.778239  3469 raft_consensus.cc:1275] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:21.782546  4543 consensus_queue.cc:1048] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:21.792313  4462 raft_consensus.cc:2955] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.794219  3469 raft_consensus.cc:2955] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.799391  2765 raft_consensus.cc:1275] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:21.801628  2765 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:21.802287  2765 raft_consensus.cc:606] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Received request to transfer leadership to TS 7cc87046f1414b27928722690a5e388f
I20260709 12:22:21.801162  4543 consensus_queue.cc:1048] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:22:21.808383  2765 raft_consensus.cc:2955] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.813261  4567 tablet_replica.cc:333] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:21.815851  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6291f8f287034586b4e9c0c9220bc6e7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:21.818398  2762 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:21.828902  4567 raft_consensus.cc:2243] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.831920  4567 raft_consensus.cc:2272] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.837610  4567 ts_tablet_manager.cc:1916] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:21.840555  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:21.842787  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:21.841956  4561 raft_consensus.cc:2955] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 4 to 6, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.843745  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:21.852159  2917 raft_consensus.cc:2955] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 4 to 6, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
W20260709 12:22:21.854883  2651 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 6291f8f287034586b4e9c0c9220bc6e7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 12:22:21.868369  4478 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:21.869623  4565 tablet_replica.cc:333] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:21.871052  4565 raft_consensus.cc:2243] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.872876  2666 catalog_manager.cc:5697] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.872968  4565 raft_consensus.cc:2272] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.881804  4567 ts_tablet_manager.cc:1929] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:21.882376  4567 log.cc:1199] T 12034fdcbbed4121a134d5d26c1dc5ac P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/12034fdcbbed4121a134d5d26c1dc5ac
I20260709 12:22:21.884684  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet 12034fdcbbed4121a134d5d26c1dc5ac (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:21.897707  2766 raft_consensus.cc:2955] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 4 to 6, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.898849  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet 6291f8f287034586b4e9c0c9220bc6e7 with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:22:21.908090  3473 raft_consensus.cc:2955] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 4 to 6, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:21.917915  2662 catalog_manager.cc:5697] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 4 to 6, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:21.944159  4565 ts_tablet_manager.cc:1916] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:21.964217  4570 tablet_replica.cc:333] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:21.983150  4530 raft_consensus.cc:993] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:21.984958  4570 raft_consensus.cc:2243] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.984959  4530 raft_consensus.cc:1081] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:21.985832  4570 raft_consensus.cc:2272] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.991776  4570 ts_tablet_manager.cc:1916] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:21.998669  2762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38956
I20260709 12:22:21.999351  2762 raft_consensus.cc:493] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:21.999739  2762 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:22.029223  4570 ts_tablet_manager.cc:1929] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:22.029982  4570 log.cc:1199] T 6291f8f287034586b4e9c0c9220bc6e7 P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/6291f8f287034586b4e9c0c9220bc6e7
I20260709 12:22:22.031946  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet 6291f8f287034586b4e9c0c9220bc6e7 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:22.034664  4565 ts_tablet_manager.cc:1929] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:22:22.035271  4565 log.cc:1199] T f26ca468f5384641ad5bd05af362aef9 P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/f26ca468f5384641ad5bd05af362aef9
I20260709 12:22:22.034785  2762 raft_consensus.cc:515] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:22.037424  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet f26ca468f5384641ad5bd05af362aef9 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:22.041039  3473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:22.041626  2762 leader_election.cc:290] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 5 election: Requested vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:22.041728  3473 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:22.042536  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:22.043134  2917 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:22.049675  3473 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 5.
I20260709 12:22:22.051043  2917 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 5.
I20260709 12:22:22.062273  2732 leader_election.cc:304] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:22.062251  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:22.063140  2843 raft_consensus.cc:3055] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:22.063472  2843 raft_consensus.cc:740] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:22.064529  2843 consensus_queue.cc:260] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:22.063408  4550 raft_consensus.cc:2804] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:22.066083  4550 raft_consensus.cc:697] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 5 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:22.066507  2843 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:22.067061  4550 consensus_queue.cc:237] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:22.075090  2843 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 5.
I20260709 12:22:22.080374  2662 catalog_manager.cc:5697] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 4 to 5, leader changed from 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 5 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:22.165509  4559 raft_consensus.cc:993] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:22.166180  4559 raft_consensus.cc:1081] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Signalling peer 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:22.167769  2917 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:47638
I20260709 12:22:22.168356  2917 raft_consensus.cc:493] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:22.168638  2917 raft_consensus.cc:3060] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.176079  2917 raft_consensus.cc:515] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:22.178917  2917 leader_election.cc:290] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:22.179404  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:22.180089  2766 raft_consensus.cc:3055] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:22.180401  2766 raft_consensus.cc:740] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:22.180732  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:22.181350  2843 raft_consensus.cc:3060] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.181340  2766 consensus_queue.cc:260] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:22.182925  2766 raft_consensus.cc:3060] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.190383  2843 raft_consensus.cc:2468] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
I20260709 12:22:22.191574  2880 leader_election.cc:304] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:22.192500  4541 raft_consensus.cc:2804] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:22.193002  4541 raft_consensus.cc:697] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:22.193192  2766 raft_consensus.cc:2468] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
I20260709 12:22:22.193967  4541 consensus_queue.cc:237] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:22.208386  2662 catalog_manager.cc:5697] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:22.534446  2917 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:22.535741  4559 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:22.543391  3472 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:22.544752  4559 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:22.594208  2762 raft_consensus.cc:1275] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:22.595922  4541 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:22:22.603085  2809 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 47840.8us
I20260709 12:22:22.624908  2837 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:22.630722  4573 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:22.651549  2841 raft_consensus.cc:1275] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:22.662974  4541 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:23.043404  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:23.066656  2762 consensus_queue.cc:237] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:23.077301  3473 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:22:23.077891  2843 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:22:23.079427  4573 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:23.082369  4559 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:23.090062  4563 raft_consensus.cc:2955] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 5 LEADER]: Committing config change with OpId 5.8: config changed from index 6 to 8, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:23.095552  2841 raft_consensus.cc:2955] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Committing config change with OpId 5.8: config changed from index 6 to 8, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:23.110373  3472 raft_consensus.cc:2955] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Committing config change with OpId 5.8: config changed from index 6 to 8, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:23.124401  2766 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:23.125154  2766 raft_consensus.cc:606] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 5 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:23.130892  2910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:23.131587  2910 raft_consensus.cc:606] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:23.133549  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:23.135584  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:23.136412  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:23.139755  2666 catalog_manager.cc:5697] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 6 to 8, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 5 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:23.145929  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 62274aac902146f7b127a2afb55b8a12 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:23.166340  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet 62274aac902146f7b127a2afb55b8a12 with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:47626
I20260709 12:22:23.182560  4584 tablet_replica.cc:333] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:23.183995  4584 raft_consensus.cc:2243] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:23.185006  4584 raft_consensus.cc:2272] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:23.191184  4584 ts_tablet_manager.cc:1916] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:23.211076  4584 ts_tablet_manager.cc:1929] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.7
I20260709 12:22:23.211704  4584 log.cc:1199] T 62274aac902146f7b127a2afb55b8a12 P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/62274aac902146f7b127a2afb55b8a12
I20260709 12:22:23.218997  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet 62274aac902146f7b127a2afb55b8a12 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:23.363901  4586 raft_consensus.cc:993] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:23.364565  4586 raft_consensus.cc:1081] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:23.366386  2766 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:23.367038  2766 raft_consensus.cc:493] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:23.367425  2766 raft_consensus.cc:3060] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:23.373333  2766 raft_consensus.cc:515] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:23.375984  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:23.376411  3473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:23.376629  2910 raft_consensus.cc:3055] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:23.376989  3473 raft_consensus.cc:3060] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:23.376963  2910 raft_consensus.cc:740] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:23.378083  2766 leader_election.cc:290] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:23.378100  2910 consensus_queue.cc:260] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:23.380651  2910 raft_consensus.cc:3060] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:23.384512  3473 raft_consensus.cc:2468] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:23.385773  2734 leader_election.cc:304] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:23.386667  4559 raft_consensus.cc:2804] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:23.387146  4559 raft_consensus.cc:697] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:23.387729  2910 raft_consensus.cc:2468] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:23.388010  4559 consensus_queue.cc:237] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:23.399730  2662 catalog_manager.cc:5697] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:23.516386  4559 raft_consensus.cc:993] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:23.516948  4573 raft_consensus.cc:1081] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 5 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:23.518525  3473 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38416
I20260709 12:22:23.519023  3473 raft_consensus.cc:493] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:23.519428  3473 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.526343  3473 raft_consensus.cc:515] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:23.528226  3473 leader_election.cc:290] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:23.529034  2766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:23.529314  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22aad5981ec94ff2a74fdf45111c0692" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:23.529814  2766 raft_consensus.cc:3055] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:23.529870  2910 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.530146  2766 raft_consensus.cc:740] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:23.530974  2766 consensus_queue.cc:260] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:23.532226  2766 raft_consensus.cc:3060] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.537230  2910 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 6.
I20260709 12:22:23.538694  3440 leader_election.cc:304] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:23.538806  2766 raft_consensus.cc:2468] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 6.
I20260709 12:22:23.539511  4410 raft_consensus.cc:2804] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:23.540052  4410 raft_consensus.cc:697] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 6 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:23.540920  4410 consensus_queue.cc:237] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:23.552935  2662 catalog_manager.cc:5697] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 5 to 6, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 6 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:23.859855  3472 raft_consensus.cc:1275] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:23.861423  4573 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:23.872407  2910 raft_consensus.cc:1275] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:23.877173  4559 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:23.936694  2766 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 6 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:22:23.938390  4410 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:23.951648  2910 raft_consensus.cc:1275] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 6 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:22:23.955902  4595 consensus_queue.cc:1048] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:24.223870  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:24.266686  2841 consensus_queue.cc:237] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
W20260709 12:22:24.275466  2809 consensus_peers.cc:597] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 01e8aac6ebd14bf4b952c62c8075f13f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:24.275681  2917 raft_consensus.cc:1275] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:24.277465  4560 consensus_queue.cc:1048] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:24.277870  2762 raft_consensus.cc:1275] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:24.279458  2762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:24.280002  2762 raft_consensus.cc:606] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Received request to transfer leadership to TS 7cc87046f1414b27928722690a5e388f
I20260709 12:22:24.280040  4560 consensus_queue.cc:1048] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:24.281411  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:22:24.283252  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:24.284188  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:24.287451  4585 raft_consensus.cc:2955] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.290282  2917 raft_consensus.cc:2955] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.300848  2762 raft_consensus.cc:2955] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.312980  2666 catalog_manager.cc:5697] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.327711  4573 raft_consensus.cc:993] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:24.321192  2917 consensus_queue.cc:237] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:24.328327  4573 raft_consensus.cc:1081] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Signalling peer 7cc87046f1414b27928722690a5e388f to start an election
I20260709 12:22:24.330735  2910 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
 from {username='slave'} at 127.0.0.1:47638
I20260709 12:22:24.331291  2910 raft_consensus.cc:493] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:24.331632  2910 raft_consensus.cc:3060] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:24.353487  2841 raft_consensus.cc:1275] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:22:24.353487  2883 consensus_peers.cc:597] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34453a67223842ed98d4220533186ade. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:24.355636  2910 raft_consensus.cc:515] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:24.356395  4541 consensus_queue.cc:1048] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:24.358675  2762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:24.359362  2762 raft_consensus.cc:3055] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:24.361738  2762 raft_consensus.cc:740] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:24.362584  2762 consensus_queue.cc:260] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:24.364621  2762 raft_consensus.cc:3060] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:24.367683  2766 raft_consensus.cc:1275] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:24.371654  4543 consensus_queue.cc:1048] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:24.373432  2762 raft_consensus.cc:2468] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
I20260709 12:22:24.374747  2881 leader_election.cc:304] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:24.375842  4541 raft_consensus.cc:2804] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:24.378585  4541 raft_consensus.cc:697] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:24.380520  2910 leader_election.cc:290] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:24.381978  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:24.382694  2837 raft_consensus.cc:3060] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:24.379947  4541 consensus_queue.cc:237] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:24.418022  2837 raft_consensus.cc:2468] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
I20260709 12:22:24.426717  4543 raft_consensus.cc:2955] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.430653  2841 raft_consensus.cc:2955] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.435678  2664 catalog_manager.cc:5697] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:24.442237  2766 raft_consensus.cc:2955] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.458716  2841 consensus_queue.cc:237] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:24.470079  2662 catalog_manager.cc:5697] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
W20260709 12:22:24.480049  2809 consensus_peers.cc:597] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e4b953250214a6f8f3b4794fab90ce0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:24.480969  2766 raft_consensus.cc:1275] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:24.482934  4560 consensus_queue.cc:1048] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:24.484534  2917 raft_consensus.cc:1275] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:22:24.492897  4560 consensus_queue.cc:1048] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:24.501505  4560 raft_consensus.cc:2955] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.505581  2766 raft_consensus.cc:2955] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.510700  2917 raft_consensus.cc:2955] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.525568  2662 catalog_manager.cc:5697] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 2, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:24.530853  2762 consensus_queue.cc:237] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:24.548631  2917 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 12:22:24.549072  2734 consensus_peers.cc:597] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a66a027320bb4d04bb8735cd6fbffabf. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:24.549341  2841 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:24.550494  4573 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:24.551596  4559 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:24.565977  4559 raft_consensus.cc:2955] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.568037  2917 raft_consensus.cc:2955] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.568593  2843 raft_consensus.cc:2955] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.588860  2666 catalog_manager.cc:5697] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 3 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:24.808199  2766 raft_consensus.cc:1275] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:24.810528  4586 consensus_queue.cc:1048] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:24.811939  2843 raft_consensus.cc:1275] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:24.813553  4586 consensus_queue.cc:1048] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:24.899201  4621 ts_tablet_manager.cc:933] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:24.900547  4623 ts_tablet_manager.cc:933] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:24.914466  4621 tablet_copy_client.cc:323] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:22:24.914991  4623 tablet_copy_client.cc:323] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:22:24.916739  2926 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet 34453a67223842ed98d4220533186ade from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:22:24.916733  2852 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet 01e8aac6ebd14bf4b952c62c8075f13f from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:22:24.917526  2926 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet 34453a67223842ed98d4220533186ade from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-34453a67223842ed98d4220533186ade
I20260709 12:22:24.917726  2852 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet 01e8aac6ebd14bf4b952c62c8075f13f from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-01e8aac6ebd14bf4b952c62c8075f13f
I20260709 12:22:24.928476  2852 tablet_copy_source_session.cc:215] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:24.929473  2926 tablet_copy_source_session.cc:215] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:24.932030  4623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e8aac6ebd14bf4b952c62c8075f13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:24.934515  4621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34453a67223842ed98d4220533186ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:24.957497  4623 tablet_copy_client.cc:806] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:24.958295  4623 tablet_copy_client.cc:670] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:24.969928  4623 tablet_copy_client.cc:538] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:24.991197  4626 ts_tablet_manager.cc:933] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:24.993186  4623 tablet_bootstrap.cc:492] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:25.003181  4626 tablet_copy_client.cc:323] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.66:44215
I20260709 12:22:25.011196  2852 tablet_copy_service.cc:140] P 9a10672cd53140d59ff945f75345cc06: Received BeginTabletCopySession request for tablet 0e4b953250214a6f8f3b4794fab90ce0 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:38132)
I20260709 12:22:25.011832  2852 tablet_copy_service.cc:161] P 9a10672cd53140d59ff945f75345cc06: Beginning new tablet copy session on tablet 0e4b953250214a6f8f3b4794fab90ce0 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:38132: session id = f440bbb3ddd545819242017140636ad4-0e4b953250214a6f8f3b4794fab90ce0
I20260709 12:22:25.018069  4628 ts_tablet_manager.cc:933] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625)
I20260709 12:22:25.019276  4621 tablet_copy_client.cc:806] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:25.022316  4621 tablet_copy_client.cc:670] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:25.023674  2852 tablet_copy_source_session.cc:215] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:25.029196  4626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e4b953250214a6f8f3b4794fab90ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:25.032281  4621 tablet_copy_client.cc:538] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:25.045722  4628 tablet_copy_client.cc:323] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.65:35625
I20260709 12:22:25.056051  2778 tablet_copy_service.cc:140] P 442c92e5267046cd92f46e6f469570a5: Received BeginTabletCopySession request for tablet a66a027320bb4d04bb8735cd6fbffabf from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:33574)
I20260709 12:22:25.056700  2778 tablet_copy_service.cc:161] P 442c92e5267046cd92f46e6f469570a5: Beginning new tablet copy session on tablet a66a027320bb4d04bb8735cd6fbffabf from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:33574: session id = f440bbb3ddd545819242017140636ad4-a66a027320bb4d04bb8735cd6fbffabf
I20260709 12:22:25.057793  4626 tablet_copy_client.cc:806] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:25.058576  4626 tablet_copy_client.cc:670] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:25.068269  4626 tablet_copy_client.cc:538] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:25.075057  2778 tablet_copy_source_session.cc:215] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:22:25.080365  4628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a66a027320bb4d04bb8735cd6fbffabf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:25.084908  4623 tablet_bootstrap.cc:492] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:25.086023  4623 tablet_bootstrap.cc:492] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:25.086887  4623 ts_tablet_manager.cc:1403] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.094s	user 0.024s	sys 0.021s
I20260709 12:22:25.091740  4623 raft_consensus.cc:359] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:25.092778  4623 raft_consensus.cc:740] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:25.094823  4623 consensus_queue.cc:260] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:25.098984  4626 tablet_bootstrap.cc:492] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:25.103938  4623 ts_tablet_manager.cc:1434] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 12:22:25.104791  4621 tablet_bootstrap.cc:492] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:25.107930  2852 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-01e8aac6ebd14bf4b952c62c8075f13f received from {username='slave'} at 127.0.0.1:38132
I20260709 12:22:25.108521  2852 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-01e8aac6ebd14bf4b952c62c8075f13f on tablet 01e8aac6ebd14bf4b952c62c8075f13f with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:25.128908  4628 tablet_copy_client.cc:806] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:25.129768  4628 tablet_copy_client.cc:670] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:25.136722  4628 tablet_copy_client.cc:538] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:25.142414  4621 tablet_bootstrap.cc:492] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:25.143683  4621 tablet_bootstrap.cc:492] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:25.144498  4621 ts_tablet_manager.cc:1403] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.040s	user 0.021s	sys 0.004s
I20260709 12:22:25.147939  4621 raft_consensus.cc:359] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:25.148886  4621 raft_consensus.cc:740] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:25.149627  4621 consensus_queue.cc:260] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:25.152383  4621 ts_tablet_manager.cc:1434] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:22:25.155735  4628 tablet_bootstrap.cc:492] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:25.167812  2926 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-34453a67223842ed98d4220533186ade received from {username='slave'} at 127.0.0.1:59582
I20260709 12:22:25.168370  2926 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-34453a67223842ed98d4220533186ade on tablet 34453a67223842ed98d4220533186ade with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:25.171169  4626 tablet_bootstrap.cc:492] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:25.178344  4626 tablet_bootstrap.cc:492] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:25.179165  4626 ts_tablet_manager.cc:1403] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.020s
I20260709 12:22:25.182561  4626 raft_consensus.cc:359] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:25.183416  4626 raft_consensus.cc:740] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:25.184188  4626 consensus_queue.cc:260] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:25.187264  4626 ts_tablet_manager.cc:1434] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:22:25.189108  2852 tablet_copy_service.cc:342] P 9a10672cd53140d59ff945f75345cc06: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-0e4b953250214a6f8f3b4794fab90ce0 received from {username='slave'} at 127.0.0.1:38132
I20260709 12:22:25.189524  2852 tablet_copy_service.cc:434] P 9a10672cd53140d59ff945f75345cc06: ending tablet copy session f440bbb3ddd545819242017140636ad4-0e4b953250214a6f8f3b4794fab90ce0 on tablet 0e4b953250214a6f8f3b4794fab90ce0 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:25.206833  4628 tablet_bootstrap.cc:492] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:25.208009  4628 tablet_bootstrap.cc:492] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:25.208849  4628 ts_tablet_manager.cc:1403] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.053s	user 0.021s	sys 0.009s
I20260709 12:22:25.212903  4628 raft_consensus.cc:359] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:25.213866  4628 raft_consensus.cc:740] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:25.214793  4628 consensus_queue.cc:260] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:25.218629  4628 ts_tablet_manager.cc:1434] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:22:25.232419  2778 tablet_copy_service.cc:342] P 442c92e5267046cd92f46e6f469570a5: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-a66a027320bb4d04bb8735cd6fbffabf received from {username='slave'} at 127.0.0.1:33574
I20260709 12:22:25.233032  2778 tablet_copy_service.cc:434] P 442c92e5267046cd92f46e6f469570a5: ending tablet copy session f440bbb3ddd545819242017140636ad4-a66a027320bb4d04bb8735cd6fbffabf on tablet a66a027320bb4d04bb8735cd6fbffabf with peer f440bbb3ddd545819242017140636ad4
W20260709 12:22:25.244774  2731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 46355.3us
I20260709 12:22:25.330356  3472 raft_consensus.cc:1217] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:25.417232  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:25.470376  3472 raft_consensus.cc:1217] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:25.485420  3469 raft_consensus.cc:1217] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:22:25.503516  3473 raft_consensus.cc:1217] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 12:22:25.541999  2913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037"
dest_uuid: "7cc87046f1414b27928722690a5e388f"
mode: GRACEFUL
new_leader_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:60088
I20260709 12:22:25.542640  2913 raft_consensus.cc:606] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Received request to transfer leadership to TS 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:25.544144  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:22:25.546165  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:25.546983  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:25.744647  4586 raft_consensus.cc:1064] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:25.747769  4586 consensus_queue.cc:237] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:25.755263  3473 raft_consensus.cc:1275] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:25.755522  2843 raft_consensus.cc:1275] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:25.757012  4541 consensus_queue.cc:1048] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:25.758108  4620 consensus_queue.cc:1048] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:25.762303  2765 raft_consensus.cc:1275] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:25.763852  4541 consensus_queue.cc:1048] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:25.789520  4586 raft_consensus.cc:2955] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:25.795408  2765 raft_consensus.cc:2955] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:25.796762  2843 raft_consensus.cc:2955] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:25.806922  3473 raft_consensus.cc:2955] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:25.831385  2664 catalog_manager.cc:5697] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:25.887087  2910 consensus_queue.cc:237] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:25.900672  2765 raft_consensus.cc:1275] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:25.902556  4586 consensus_queue.cc:1048] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:25.908313  3473 raft_consensus.cc:1275] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:25.912544  4541 raft_consensus.cc:2955] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:25.915520  2762 raft_consensus.cc:2955] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:25.914839  4620 consensus_queue.cc:1048] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:25.926173  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 34453a67223842ed98d4220533186ade with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:25.938892  3469 raft_consensus.cc:2955] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:25.949662  2666 catalog_manager.cc:5697] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 reported cstate change: config changed from index 3 to 4, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 1 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:25.952672  4586 raft_consensus.cc:993] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f: : Instructing follower 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:25.953351  4618 raft_consensus.cc:1081] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Signalling peer 442c92e5267046cd92f46e6f469570a5 to start an election
I20260709 12:22:25.955936  2762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 12:22:25.956573  2762 raft_consensus.cc:493] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:25.957007  2762 raft_consensus.cc:3060] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:25.965802  4585 raft_consensus.cc:1064] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:25.968775  4585 consensus_queue.cc:237] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:25.980522  2762 raft_consensus.cc:515] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:25.982888  2762 leader_election.cc:290] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Requested vote from peers 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:25.988358  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet 34453a67223842ed98d4220533186ade with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42516
I20260709 12:22:25.991621  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:25.996636  4647 tablet_replica.cc:333] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:25.998544  3473 raft_consensus.cc:1275] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:25.999966  4647 raft_consensus.cc:2243] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.000169  4560 consensus_queue.cc:1048] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:26.004765  4647 raft_consensus.cc:2272] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.009516  2762 raft_consensus.cc:1275] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:26.010349  2917 raft_consensus.cc:1275] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:26.011991  4647 ts_tablet_manager.cc:1916] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:25.996922  2910 raft_consensus.cc:3055] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:26.014251  2910 raft_consensus.cc:740] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:26.014639  4560 consensus_queue.cc:1048] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:26.015336  2910 consensus_queue.cc:260] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:26.016611  2910 raft_consensus.cc:3060] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:26.016767  4560 consensus_queue.cc:1048] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:26.047070  3472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:26.047777  3472 raft_consensus.cc:3060] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:26.047812  4647 ts_tablet_manager.cc:1929] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:26.048794  4647 log.cc:1199] T 34453a67223842ed98d4220533186ade P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/34453a67223842ed98d4220533186ade
I20260709 12:22:26.059041  4560 raft_consensus.cc:1064] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:26.062559  4560 consensus_queue.cc:237] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:26.066987  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet 34453a67223842ed98d4220533186ade (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:26.073222  4634 raft_consensus.cc:1064] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:26.110184  3472 raft_consensus.cc:2468] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
I20260709 12:22:26.361064  3472 raft_consensus.cc:1275] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 LEARNER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:26.366350  4655 consensus_queue.cc:1048] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:26.370879  2910 raft_consensus.cc:2468] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442c92e5267046cd92f46e6f469570a5 in term 2.
W20260709 12:22:26.380806  2734 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 314771us
I20260709 12:22:26.385556  4634 consensus_queue.cc:237] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:26.394675  2732 leader_election.cc:304] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f; no voters: 
I20260709 12:22:26.408929  2913 raft_consensus.cc:1275] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:26.416163  2759 raft_consensus.cc:1275] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:22:26.417168  4658 raft_consensus.cc:2804] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:26.425115  4585 raft_consensus.cc:2955] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.443743  2768 raft_consensus.cc:2955] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.455343  4658 raft_consensus.cc:697] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Leader. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
W20260709 12:22:26.456830  4657 log.cc:927] Time spent T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:22:26.461977  4658 consensus_queue.cc:237] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:26.487468  4655 consensus_queue.cc:1048] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:26.498649  3472 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:26.501396  3473 raft_consensus.cc:2955] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.501724  2910 raft_consensus.cc:2955] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.503720  2913 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:26.504261  2841 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:26.517477  4652 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:26.519593  4559 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:26.523900  4637 consensus_queue.cc:1048] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:26.526125  2666 catalog_manager.cc:5697] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:26.573948  4563 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:26.589969  4637 raft_consensus.cc:2955] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.595527  2910 raft_consensus.cc:2955] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.613349  4563 raft_consensus.cc:2955] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.616504  2911 raft_consensus.cc:2955] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.626942  2843 consensus_queue.cc:237] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:26.629428  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:26.650754  3473 raft_consensus.cc:2955] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.661156  3472 raft_consensus.cc:2955] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.648789  2663 catalog_manager.cc:5697] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.669049  3469 raft_consensus.cc:1275] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:26.671097  4666 consensus_queue.cc:1048] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:26.671552  2917 raft_consensus.cc:1275] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:26.674217  2762 raft_consensus.cc:2955] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.685189  4585 raft_consensus.cc:2955] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.687409  2662 catalog_manager.cc:5697] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 reported cstate change: term changed from 1 to 2, leader changed from 7cc87046f1414b27928722690a5e388f (127.1.123.67) to 442c92e5267046cd92f46e6f469570a5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:26.697420  3472 raft_consensus.cc:2955] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.700496  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 01e8aac6ebd14bf4b952c62c8075f13f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:26.702595  4666 consensus_queue.cc:1048] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:26.703204  2843 raft_consensus.cc:2955] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } attrs { replace: true } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.711817  2911 raft_consensus.cc:2955] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.720476  2664 catalog_manager.cc:5697] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 2 to 3, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.772490  2666 catalog_manager.cc:5697] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 3 to 4, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:26.812256  2754 tablet_service.cc:1558] Processing DeleteTablet for tablet 01e8aac6ebd14bf4b952c62c8075f13f with delete_type TABLET_DATA_TOMBSTONED (TS 442c92e5267046cd92f46e6f469570a5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38926
I20260709 12:22:26.814666  4678 tablet_replica.cc:333] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:26.816097  4678 raft_consensus.cc:2243] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.817269  4678 raft_consensus.cc:2272] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.818996  2843 consensus_queue.cc:237] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:26.843428  3472 raft_consensus.cc:1275] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:26.845503  2762 raft_consensus.cc:1275] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:22:26.845825  4637 consensus_queue.cc:1048] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:26.847523  4666 consensus_queue.cc:1048] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:26.850914  2768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699"
dest_uuid: "442c92e5267046cd92f46e6f469570a5"
mode: GRACEFUL
new_leader_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:22:26.851531  2768 raft_consensus.cc:606] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Received request to transfer leadership to TS f440bbb3ddd545819242017140636ad4
I20260709 12:22:26.856105  4585 raft_consensus.cc:2955] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.859246  3472 raft_consensus.cc:2955] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.878482  2765 raft_consensus.cc:2955] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.862272  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:22:26.885717  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:26.890368  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:26.869154  2759 consensus_queue.cc:237] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:26.916108  3474 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:26.917979  4658 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:26.929704  2911 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:26.932819  4678 ts_tablet_manager.cc:1916] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:26.932608  4658 raft_consensus.cc:2955] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.934317  4563 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:26.965356  2650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0e4b953250214a6f8f3b4794fab90ce0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:26.966473  2911 raft_consensus.cc:2955] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.975340  2663 catalog_manager.cc:5697] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 reported cstate change: config changed from index 3 to 4, VOTER 7cc87046f1414b27928722690a5e388f (127.1.123.67) evicted. New cstate: current_term: 1 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:26.987365  3472 raft_consensus.cc:2955] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:27.007651  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a66a027320bb4d04bb8735cd6fbffabf with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:27.039297  2664 catalog_manager.cc:5697] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f reported cstate change: config changed from index 5 to 6, VOTER 9a10672cd53140d59ff945f75345cc06 (127.1.123.66) evicted. New cstate: current_term: 3 leader_uuid: "442c92e5267046cd92f46e6f469570a5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:27.048714  4678 ts_tablet_manager.cc:1929] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:27.049436  4678 log.cc:1199] T 01e8aac6ebd14bf4b952c62c8075f13f P 442c92e5267046cd92f46e6f469570a5: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/wals/01e8aac6ebd14bf4b952c62c8075f13f
I20260709 12:22:27.051469  2651 catalog_manager.cc:5028] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): tablet 01e8aac6ebd14bf4b952c62c8075f13f (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:27.054246  2899 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e4b953250214a6f8f3b4794fab90ce0 with delete_type TABLET_DATA_TOMBSTONED (TS 7cc87046f1414b27928722690a5e388f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47626
I20260709 12:22:27.077436  4680 tablet_replica.cc:333] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:27.078084  2911 raft_consensus.cc:1275] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:27.080174  4663 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:27.080690  4680 raft_consensus.cc:2243] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:27.081637  4680 raft_consensus.cc:2272] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:27.085896  3472 raft_consensus.cc:1275] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Refusing update from remote peer 442c92e5267046cd92f46e6f469570a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:22:27.087864  4680 ts_tablet_manager.cc:1916] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:27.087756  4663 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:27.107710  4680 ts_tablet_manager.cc:1929] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:22:27.108413  4680 log.cc:1199] T 0e4b953250214a6f8f3b4794fab90ce0 P 7cc87046f1414b27928722690a5e388f: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/wals/0e4b953250214a6f8f3b4794fab90ce0
I20260709 12:22:27.110737  2651 catalog_manager.cc:5028] TS 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): tablet 0e4b953250214a6f8f3b4794fab90ce0 (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:27.113760  2826 tablet_service.cc:1558] Processing DeleteTablet for tablet a66a027320bb4d04bb8735cd6fbffabf with delete_type TABLET_DATA_TOMBSTONED (TS 9a10672cd53140d59ff945f75345cc06 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42516
W20260709 12:22:27.156224  2697 debug-util.cc:398] Leaking SignalData structure 0x7b0800234fc0 after lost signal to thread 1520
W20260709 12:22:27.157791  2697 debug-util.cc:398] Leaking SignalData structure 0x7b0800332c00 after lost signal to thread 2700
W20260709 12:22:27.159277  2697 debug-util.cc:398] Leaking SignalData structure 0x7b08000390c0 after lost signal to thread 2792
W20260709 12:22:27.160645  2697 debug-util.cc:398] Leaking SignalData structure 0x7b0800038860 after lost signal to thread 2867
W20260709 12:22:27.162241  2697 debug-util.cc:398] Leaking SignalData structure 0x7b0800106600 after lost signal to thread 2941
W20260709 12:22:27.163784  2697 debug-util.cc:398] Leaking SignalData structure 0x7b080031e400 after lost signal to thread 3503
I20260709 12:22:27.287897  4683 tablet_replica.cc:333] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:27.287931  4658 raft_consensus.cc:993] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5: : Instructing follower f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:27.290723  4563 raft_consensus.cc:1081] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Signalling peer f440bbb3ddd545819242017140636ad4 to start an election
I20260709 12:22:27.292431  4683 raft_consensus.cc:2243] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:27.292632  3474 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699"
dest_uuid: "f440bbb3ddd545819242017140636ad4"
 from {username='slave'} at 127.0.0.1:38416
I20260709 12:22:27.293309  4683 raft_consensus.cc:2272] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:27.293334  3474 raft_consensus.cc:493] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:27.293922  3474 raft_consensus.cc:3060] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:27.297879  4683 ts_tablet_manager.cc:1916] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:22:27.300027  3474 raft_consensus.cc:515] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:27.302007  3474 leader_election.cc:290] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:27.303225  2765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "442c92e5267046cd92f46e6f469570a5"
I20260709 12:22:27.303923  2765 raft_consensus.cc:3055] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:27.304280  2765 raft_consensus.cc:740] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 442c92e5267046cd92f46e6f469570a5, State: Running, Role: LEADER
I20260709 12:22:27.305265  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42af6798b62746449070b7b2bc4e0699" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:27.305047  2765 consensus_queue.cc:260] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:27.306468  2765 raft_consensus.cc:3060] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:27.305895  2911 raft_consensus.cc:3060] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:27.315541  2765 raft_consensus.cc:2468] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 3.
I20260709 12:22:27.316973  3440 leader_election.cc:304] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:27.317925  4679 raft_consensus.cc:2804] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:27.318404  4679 raft_consensus.cc:697] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 3 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:27.319296  4683 ts_tablet_manager.cc:1929] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:22:27.319803  4683 log.cc:1199] T a66a027320bb4d04bb8735cd6fbffabf P 9a10672cd53140d59ff945f75345cc06: Deleting WAL directory at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/wals/a66a027320bb4d04bb8735cd6fbffabf
I20260709 12:22:27.319332  4679 consensus_queue.cc:237] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:27.321993  2650 catalog_manager.cc:5028] TS 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): tablet a66a027320bb4d04bb8735cd6fbffabf (table test-workload [id=6b661bc00226474da0908bf962632d5b]) successfully deleted
I20260709 12:22:27.325614  2911 raft_consensus.cc:2468] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 3.
I20260709 12:22:27.335806  2664 catalog_manager.cc:5697] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 3 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:27.748430  2765 raft_consensus.cc:1275] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:27.751492  4679 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:27.881655  2911 raft_consensus.cc:1275] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:27.894168  4410 consensus_queue.cc:1048] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:28.164718  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:28.234117  2917 consensus_queue.cc:237] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:28.244021  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:28.249950  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:28.257302  2765 raft_consensus.cc:1275] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:28.257431  2843 raft_consensus.cc:1275] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:28.259542  4687 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:28.264287  4698 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:28.266399  2883 consensus_peers.cc:597] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f -> Peer f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397): Couldn't send request to peer f440bbb3ddd545819242017140636ad4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9f24d490dc840f1b2d87604abe412d8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:28.257685  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:28.275513  4687 raft_consensus.cc:2955] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:28.286244  2842 raft_consensus.cc:2955] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:28.313302  2664 catalog_manager.cc:5697] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New cstate: current_term: 3 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:28.351198  2762 raft_consensus.cc:2955] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER f440bbb3ddd545819242017140636ad4 (127.1.123.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } } }
I20260709 12:22:28.492684  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:28.715814  1517 tablet_server.cc:179] TabletServer@127.1.123.65:0 shutting down...
W20260709 12:22:28.780551  3440 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
W20260709 12:22:28.858844  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 78941.8us
W20260709 12:22:28.887321  3440 consensus_peers.cc:597] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:28.888103  3440 consensus_peers.cc:597] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:28.918526  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 115716us
W20260709 12:22:28.928414  2807 consensus_peers.cc:597] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:28.934110  3440 consensus_peers.cc:597] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:28.950941  3440 consensus_peers.cc:597] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:28.987911  2807 consensus_peers.cc:597] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:28.988853  2807 consensus_peers.cc:597] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:28.989584  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:28.991983  1517 tablet_replica.cc:333] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:28.993520  1517 raft_consensus.cc:2243] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:28.990535  4709 ts_tablet_manager.cc:933] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4: Initiating tablet copy from peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:28.994961  1517 raft_consensus.cc:2272] T e07072d40c8b441e8c50e662b2daf24c P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:29.001992  2881 consensus_peers.cc:597] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:29.004877  2807 consensus_peers.cc:597] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:29.010456  2807 consensus_peers.cc:597] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.011197  1517 tablet_replica.cc:333] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
W20260709 12:22:29.008185  2881 consensus_peers.cc:597] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:29.014003  2807 consensus_peers.cc:597] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:29.014532  2881 consensus_peers.cc:597] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.014683  1517 raft_consensus.cc:2243] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:29.015218  2881 consensus_peers.cc:597] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.015522  1517 raft_consensus.cc:2272] T 0e4b953250214a6f8f3b4794fab90ce0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:29.015964  2881 consensus_peers.cc:597] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.019104  1517 tablet_replica.cc:333] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.011972  4709 tablet_copy_client.cc:323] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.123.67:37395
I20260709 12:22:29.032620  2926 tablet_copy_service.cc:140] P 7cc87046f1414b27928722690a5e388f: Received BeginTabletCopySession request for tablet c9f24d490dc840f1b2d87604abe412d8 from peer f440bbb3ddd545819242017140636ad4 ({username='slave'} at 127.0.0.1:59582)
I20260709 12:22:29.032752  1517 raft_consensus.cc:2243] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.043412  1517 raft_consensus.cc:2272] T a7c3a03a1304452ebc48e986f5629037 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.047632  1517 tablet_replica.cc:333] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
W20260709 12:22:29.050064  2807 consensus_peers.cc:597] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.033473  2926 tablet_copy_service.cc:161] P 7cc87046f1414b27928722690a5e388f: Beginning new tablet copy session on tablet c9f24d490dc840f1b2d87604abe412d8 from peer f440bbb3ddd545819242017140636ad4 at {username='slave'} at 127.0.0.1:59582: session id = f440bbb3ddd545819242017140636ad4-c9f24d490dc840f1b2d87604abe412d8
I20260709 12:22:29.066437  1517 raft_consensus.cc:2243] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.077495  1517 raft_consensus.cc:2272] T 9164313f06e045c3bdef4a46d365873c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.086571  1517 tablet_replica.cc:333] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
W20260709 12:22:29.062485  3440 consensus_peers.cc:597] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:29.098112  2807 consensus_peers.cc:597] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.105683  1517 raft_consensus.cc:2243] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.109843  1517 raft_consensus.cc:2272] T 56d2861f96e247a78ce40f9dfe9fe850 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:29.114908  3440 consensus_peers.cc:597] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:29.129200  2881 consensus_peers.cc:597] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:29.185086  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 55960.3us
I20260709 12:22:29.122216  1517 tablet_replica.cc:333] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.187956  1517 raft_consensus.cc:2243] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.188627  1517 raft_consensus.cc:2272] T 6291f8f287034586b4e9c0c9220bc6e7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:29.195231  2881 consensus_peers.cc:597] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.196408  1517 tablet_replica.cc:333] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.197446  1517 raft_consensus.cc:2243] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.198489  1517 raft_consensus.cc:2272] T 53752656656243c3abe78c7a9cb488c7 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.201695  1517 tablet_replica.cc:333] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.119992  2926 tablet_copy_source_session.cc:215] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:22:29.206709  2807 consensus_peers.cc:597] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:29.211455  2881 consensus_peers.cc:597] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:29.213863  3440 consensus_peers.cc:597] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.217344  1517 raft_consensus.cc:2243] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:29.218291  2807 consensus_peers.cc:597] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.219141  4709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9f24d490dc840f1b2d87604abe412d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:29.231463  1517 raft_consensus.cc:2272] T a9961074150e45dbb36db8e49d840c5d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.240334  1517 tablet_replica.cc:333] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
W20260709 12:22:29.246624  3440 consensus_peers.cc:597] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:29.249920  3440 consensus_peers.cc:597] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:29.267217  2807 consensus_peers.cc:597] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.251413  1517 raft_consensus.cc:2243] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.275935  1517 raft_consensus.cc:2272] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.291754  1517 tablet_replica.cc:333] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.302349  1517 raft_consensus.cc:2243] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.307829  1517 raft_consensus.cc:2272] T 34453a67223842ed98d4220533186ade P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.317879  1517 tablet_replica.cc:333] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
W20260709 12:22:29.324894  3440 consensus_peers.cc:597] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.335970  4709 tablet_copy_client.cc:806] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:29.336808  4709 tablet_copy_client.cc:670] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:29.348816  1517 raft_consensus.cc:2243] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.350757  4709 tablet_copy_client.cc:538] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:29.353487  1517 raft_consensus.cc:2272] T 4ff32c54020945d99bd67b7df9574bae P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.369417  1517 tablet_replica.cc:333] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
W20260709 12:22:29.370918  3440 consensus_peers.cc:597] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.389942  1517 raft_consensus.cc:2243] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.391273  1517 raft_consensus.cc:2272] T 1a061e32042842a9a967750b68aa8abf P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.402514  1517 tablet_replica.cc:333] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.408349  1517 raft_consensus.cc:2243] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.409790  1517 raft_consensus.cc:2272] T fe7ee185c0114c079f53469226dee8bf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.410601  4709 tablet_bootstrap.cc:492] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4: Bootstrap starting.
I20260709 12:22:29.422245  1517 tablet_replica.cc:333] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.423272  1517 raft_consensus.cc:2243] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.423907  1517 raft_consensus.cc:2272] T 6979a63415d34b6d91bb669ecec01bdf P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.450912  1517 tablet_replica.cc:333] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.454157  1517 raft_consensus.cc:2243] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.460168  1517 raft_consensus.cc:2272] T cf19d36b4d1b4825a45f6749aaa19896 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.471792  1517 tablet_replica.cc:333] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.475756  1517 raft_consensus.cc:2243] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.478539  1517 raft_consensus.cc:2272] T 22aad5981ec94ff2a74fdf45111c0692 P 442c92e5267046cd92f46e6f469570a5 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.489880  1517 tablet_replica.cc:333] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.492332  1517 raft_consensus.cc:2243] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.501874  1517 raft_consensus.cc:2272] T 7e4cde589c024472b0a4cec236ea69a3 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.505046  1517 tablet_replica.cc:333] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.506287  1517 raft_consensus.cc:2243] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.515268  1517 raft_consensus.cc:2272] T 42af6798b62746449070b7b2bc4e0699 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.519019  1517 tablet_replica.cc:333] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.539255  1517 raft_consensus.cc:2243] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.540117  1517 raft_consensus.cc:2272] T e3545f99312f483eb6dc8965c5bc418b P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.543653  1517 tablet_replica.cc:333] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.544631  1517 raft_consensus.cc:2243] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.545912  1517 raft_consensus.cc:2272] T 6855fa72e5f9486bb478855f721e77ce P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.552093  1517 tablet_replica.cc:333] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.564033  1517 raft_consensus.cc:2243] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.565064  1517 raft_consensus.cc:2272] T 659c2c3a61024d07a2296bc83d87a52d P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.579008  1517 tablet_replica.cc:333] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.585659  4709 tablet_bootstrap.cc:492] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:29.586337  1517 raft_consensus.cc:2243] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.599335  1517 raft_consensus.cc:2272] T d500b7bb3bd042f09ffa7310e921aa24 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.605414  1517 tablet_replica.cc:333] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.606997  1517 raft_consensus.cc:2243] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.628777  1517 raft_consensus.cc:2272] T 021d84b2e9364749b9ae4e7fde0b14bc P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.639034  4709 tablet_bootstrap.cc:492] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4: Bootstrap complete.
I20260709 12:22:29.640252  4709 ts_tablet_manager.cc:1403] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4: Time spent bootstrapping tablet: real 0.251s	user 0.031s	sys 0.009s
I20260709 12:22:29.654502  1517 tablet_replica.cc:333] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.655711  1517 raft_consensus.cc:2243] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.656394  4709 raft_consensus.cc:359] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:29.657099  1517 raft_consensus.cc:2272] T 2aa6b7efbb774150809e590d3dde361f P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:29.657491  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 132581us
I20260709 12:22:29.665805  4709 raft_consensus.cc:740] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: f440bbb3ddd545819242017140636ad4, State: Initialized, Role: LEARNER
I20260709 12:22:29.707692  4709 consensus_queue.cc:260] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
W20260709 12:22:29.718847  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 44092.5us
I20260709 12:22:29.735924  4709 ts_tablet_manager.cc:1434] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4: Time spent starting tablet: real 0.095s	user 0.007s	sys 0.001s
I20260709 12:22:29.772926  2926 tablet_copy_service.cc:342] P 7cc87046f1414b27928722690a5e388f: Request end of tablet copy session f440bbb3ddd545819242017140636ad4-c9f24d490dc840f1b2d87604abe412d8 received from {username='slave'} at 127.0.0.1:59582
I20260709 12:22:29.773607  2926 tablet_copy_service.cc:434] P 7cc87046f1414b27928722690a5e388f: ending tablet copy session f440bbb3ddd545819242017140636ad4-c9f24d490dc840f1b2d87604abe412d8 on tablet c9f24d490dc840f1b2d87604abe412d8 with peer f440bbb3ddd545819242017140636ad4
I20260709 12:22:29.773686  1517 tablet_replica.cc:333] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.774725  1517 raft_consensus.cc:2243] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.775385  1517 raft_consensus.cc:2272] T 41f5e1f4763f4d978619c2696e7aaa71 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.787492  4705 raft_consensus.cc:493] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:29.788347  4705 raft_consensus.cc:515] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:29.792079  4705 leader_election.cc:290] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:29.793352  3472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1bafa30972440499c69fb1221eae27" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:29.793730  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1bafa30972440499c69fb1221eae27" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:29.796586  2731 leader_election.cc:304] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4
I20260709 12:22:29.797930  4705 raft_consensus.cc:2749] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:29.792693  1517 tablet_replica.cc:333] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.806672  1517 raft_consensus.cc:2243] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.808015  1517 raft_consensus.cc:2272] T a66a027320bb4d04bb8735cd6fbffabf P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.823349  4705 raft_consensus.cc:493] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f440bbb3ddd545819242017140636ad4)
I20260709 12:22:29.824093  4705 raft_consensus.cc:515] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:29.827690  3472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:29.829010  4705 leader_election.cc:290] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:29.830834  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f8a42191e14f07a43032df18499a9c" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:29.832434  2731 leader_election.cc:304] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4
I20260709 12:22:29.833683  4705 raft_consensus.cc:2749] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:29.831705  1517 tablet_replica.cc:333] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.850382  1517 raft_consensus.cc:2243] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.851327  1517 raft_consensus.cc:2272] T a2f8a42191e14f07a43032df18499a9c P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.852674  4658 raft_consensus.cc:493] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:29.853291  4658 raft_consensus.cc:515] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: NON_VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: true } }
I20260709 12:22:29.855625  1517 tablet_replica.cc:333] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.856782  4658 leader_election.cc:290] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:29.851346  4705 raft_consensus.cc:493] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:29.858366  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:29.858558  4705 raft_consensus.cc:515] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:29.860873  4705 leader_election.cc:290] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:29.862511  1517 raft_consensus.cc:2243] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.862576  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:29.862857  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f24d490dc840f1b2d87604abe412d8" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:29.863389  1517 raft_consensus.cc:2272] T 547d5e0cde57459bbad31d06eb64d442 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.864404  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1605919970c4ce68b488b932d3e81b3" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:29.867805  1517 tablet_replica.cc:333] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.868752  1517 raft_consensus.cc:2243] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.869901  2732 leader_election.cc:304] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:29.871246  2732 leader_election.cc:304] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:29.872313  4658 raft_consensus.cc:2706] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Received pre-election callback for term 4 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:22:29.879403  4663 raft_consensus.cc:2749] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:29.879515  1517 raft_consensus.cc:2272] T c9f24d490dc840f1b2d87604abe412d8 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.884603  1517 tablet_replica.cc:333] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.885509  1517 raft_consensus.cc:2243] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.890066  1517 raft_consensus.cc:2272] T 627b3359300e44fc9d2b83ed88272a9f P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.893350  1517 tablet_replica.cc:333] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.895550  1517 raft_consensus.cc:2243] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.901657  1517 raft_consensus.cc:2272] T 2594ea75a8d44503a64ea4e06916c4b3 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.909083  1517 tablet_replica.cc:333] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.910141  1517 raft_consensus.cc:2243] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.913497  1517 raft_consensus.cc:2272] T 62274aac902146f7b127a2afb55b8a12 P 442c92e5267046cd92f46e6f469570a5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.925555  1517 tablet_replica.cc:333] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.926573  1517 raft_consensus.cc:2243] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.927310  1517 raft_consensus.cc:2272] T dc860efbfaf84207b7128fd001b59fe6 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.930110  1517 tablet_replica.cc:333] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.931496  1517 raft_consensus.cc:2243] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.939957  4705 raft_consensus.cc:493] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:29.940709  4705 raft_consensus.cc:515] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:29.943990  4705 leader_election.cc:290] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:29.944562  3472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0df81094e2493cbe804d2dba9aa9d4" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:29.945199  1517 raft_consensus.cc:2272] T 133424caefd34fa8b83015c9fe06d612 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.947219  2911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0df81094e2493cbe804d2dba9aa9d4" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:29.948916  2732 leader_election.cc:304] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4
I20260709 12:22:29.949285  1517 tablet_replica.cc:333] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.949797  4705 raft_consensus.cc:2749] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:29.959378  1517 raft_consensus.cc:2243] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.989039  1517 raft_consensus.cc:2272] T 6487a298e2bb425bbfe83bed12805837 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.992293  1517 tablet_replica.cc:333] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:29.993274  1517 raft_consensus.cc:2243] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.994396  1517 raft_consensus.cc:2272] T bb1bafa30972440499c69fb1221eae27 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:30.000510  1517 tablet_replica.cc:333] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:30.001480  1517 raft_consensus.cc:2243] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:30.002228  1517 raft_consensus.cc:2272] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:30.022010  4705 raft_consensus.cc:493] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:30.022655  4705 raft_consensus.cc:515] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:30.025009  4705 leader_election.cc:290] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:30.028236  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:30.031251  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa220c2195724c24b522d44636d98970" candidate_uuid: "442c92e5267046cd92f46e6f469570a5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:30.033362  2732 leader_election.cc:304] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442c92e5267046cd92f46e6f469570a5; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:30.035252  4658 raft_consensus.cc:2749] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:30.041831  1517 tablet_replica.cc:333] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:30.042932  1517 raft_consensus.cc:2243] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:30.044279  1517 raft_consensus.cc:2272] T fe9fc4cceb94485ca488ce52e6d94e25 P 442c92e5267046cd92f46e6f469570a5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:30.047139  1517 tablet_replica.cc:333] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:30.050205  1517 raft_consensus.cc:2243] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:30.050951  1517 raft_consensus.cc:2272] T fa220c2195724c24b522d44636d98970 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:30.053725  1517 tablet_replica.cc:333] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:30.054601  1517 raft_consensus.cc:2243] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:30.056329  1517 raft_consensus.cc:2272] T 99bcb73e9233433a85aae0da2439d851 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:30.060170  1517 tablet_replica.cc:333] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:30.061030  1517 raft_consensus.cc:2243] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:30.062000  1517 raft_consensus.cc:2272] T 1f0df81094e2493cbe804d2dba9aa9d4 P 442c92e5267046cd92f46e6f469570a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:30.064725  1517 tablet_replica.cc:333] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:30.065567  1517 raft_consensus.cc:2243] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:30.066823  1517 raft_consensus.cc:2272] T 15476716416b4e729148408ed7846297 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:30.071575  1517 tablet_replica.cc:333] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:30.072429  1517 raft_consensus.cc:2243] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:30.073658  1517 raft_consensus.cc:2272] T f26ca468f5384641ad5bd05af362aef9 P 442c92e5267046cd92f46e6f469570a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:30.076625  1517 tablet_replica.cc:333] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:30.077597  1517 raft_consensus.cc:2243] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:30.083675  3472 raft_consensus.cc:1217] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 12:22:30.086158  1517 raft_consensus.cc:2272] T f1605919970c4ce68b488b932d3e81b3 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:30.089439  1517 tablet_replica.cc:333] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5: stopping tablet replica
I20260709 12:22:30.090257  1517 raft_consensus.cc:2243] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:30.090937  1517 raft_consensus.cc:2272] T 532f0b78131648af8fd4930674a8bce0 P 442c92e5267046cd92f46e6f469570a5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:30.120821  4719 raft_consensus.cc:493] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.121481  4719 raft_consensus.cc:515] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:30.123881  4719 leader_election.cc:290] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:30.125361  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
W20260709 12:22:30.131007  2807 leader_election.cc:336] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:22:30.131672  2807 leader_election.cc:304] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:30.132572  4719 raft_consensus.cc:2749] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:30.137887  4721 raft_consensus.cc:493] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.138700  4721 raft_consensus.cc:515] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:30.144214  4721 leader_election.cc:290] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:30.145355  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:30.146140  2843 raft_consensus.cc:2468] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 4.
I20260709 12:22:30.148792  2880 leader_election.cc:304] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
W20260709 12:22:30.150996  2881 leader_election.cc:336] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:22:30.153193  4721 raft_consensus.cc:2804] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:30.153807  4721 raft_consensus.cc:493] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.154232  4721 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:30.182965  4721 raft_consensus.cc:515] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:30.186828  4712 raft_consensus.cc:493] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.187599  4712 raft_consensus.cc:515] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:30.192028  3474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
W20260709 12:22:30.194159  2881 leader_election.cc:336] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:22:30.199388  2881 leader_election.cc:336] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:22:30.200006  2881 leader_election.cc:304] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4
I20260709 12:22:30.203455  4712 leader_election.cc:290] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:30.205668  4702 raft_consensus.cc:2749] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:30.211107  4721 leader_election.cc:290] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:30.214447  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07072d40c8b441e8c50e662b2daf24c" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:30.215200  2843 raft_consensus.cc:3060] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:30.244374  2843 raft_consensus.cc:2468] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 5.
I20260709 12:22:30.248976  2880 leader_election.cc:304] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:30.256126  4702 raft_consensus.cc:2804] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:30.256799  4702 raft_consensus.cc:697] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 5 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:30.257947  4702 consensus_queue.cc:237] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:30.294446  4687 raft_consensus.cc:493] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.295104  4687 raft_consensus.cc:515] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:30.297467  4687 leader_election.cc:290] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:30.318694  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
W20260709 12:22:30.319618  2881 leader_election.cc:336] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:22:30.314180  2664 catalog_manager.cc:5697] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 4 to 5, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 5 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:30.331794  2880 leader_election.cc:304] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:30.333287  4687 raft_consensus.cc:2749] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:30.367422  4717 raft_consensus.cc:493] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.368086  4717 raft_consensus.cc:515] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:30.370451  4717 leader_election.cc:290] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:30.375530  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:30.376367  2917 raft_consensus.cc:2468] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:30.380923  3440 leader_election.cc:304] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:30.395071  4679 raft_consensus.cc:2804] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:30.395571  4679 raft_consensus.cc:493] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.395913  4679 raft_consensus.cc:3060] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:30.415259  3440 leader_election.cc:336] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:30.407094  4679 raft_consensus.cc:515] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:30.420866  4679 leader_election.cc:290] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:30.427152  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7c3a03a1304452ebc48e986f5629037" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:30.427893  2917 raft_consensus.cc:3060] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:30.430619  3440 leader_election.cc:336] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:30.475312  2917 raft_consensus.cc:2468] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 3.
I20260709 12:22:30.477159  3440 leader_election.cc:304] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:30.478066  4679 raft_consensus.cc:2804] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:30.478598  4679 raft_consensus.cc:697] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 3 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:30.479549  4679 consensus_queue.cc:237] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:30.501063  4718 raft_consensus.cc:493] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.501827  4718 raft_consensus.cc:515] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:30.504257  4718 leader_election.cc:290] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:30.506835  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:30.507690  2917 raft_consensus.cc:2468] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 2.
I20260709 12:22:30.511577  2807 leader_election.cc:304] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:30.513509  4719 raft_consensus.cc:2804] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:30.514098  4719 raft_consensus.cc:493] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.514569  4719 raft_consensus.cc:3060] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:30.517717  2807 leader_election.cc:336] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:30.539584  4717 raft_consensus.cc:493] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.540410  4717 raft_consensus.cc:515] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:30.543030  4717 leader_election.cc:290] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:30.544936  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe7ee185c0114c079f53469226dee8bf" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:30.546350  4719 raft_consensus.cc:515] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:22:30.548573  3440 leader_election.cc:336] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:30.551599  3440 leader_election.cc:304] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:30.552762  4719 leader_election.cc:290] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:30.552907  4679 raft_consensus.cc:2749] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:30.559190  2664 catalog_manager.cc:5697] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 3 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:30.567310  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56d2861f96e247a78ce40f9dfe9fe850" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:30.568220  2917 raft_consensus.cc:3060] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:30.584204  2807 leader_election.cc:336] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:30.619673  2917 raft_consensus.cc:2468] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 3.
I20260709 12:22:30.653383  4717 raft_consensus.cc:493] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.665879  4712 raft_consensus.cc:1064] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f: attempting to promote NON_VOTER f440bbb3ddd545819242017140636ad4 to VOTER
I20260709 12:22:30.710809  4717 raft_consensus.cc:515] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:30.714102  4712 consensus_queue.cc:237] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:30.722409  2841 raft_consensus.cc:1275] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:30.724062  4687 consensus_queue.cc:1048] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:30.731875  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4cde589c024472b0a4cec236ea69a3" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:30.713465  4717 leader_election.cc:290] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:30.735869  3440 leader_election.cc:336] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:30.737782  3440 leader_election.cc:304] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:30.737856  4687 raft_consensus.cc:493] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.742498  4687 raft_consensus.cc:515] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:30.744900  4687 leader_election.cc:290] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:30.748725  4679 raft_consensus.cc:2749] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:30.748965  2842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
W20260709 12:22:30.759342  2881 leader_election.cc:336] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:30.760048  2881 leader_election.cc:304] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:30.761051  4702 raft_consensus.cc:2749] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:30.769115  4719 raft_consensus.cc:493] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.769834  4719 raft_consensus.cc:515] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:30.772230  4719 leader_election.cc:290] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:30.775484  2807 leader_election.cc:304] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:30.782938  4718 raft_consensus.cc:2804] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:30.783746  4718 raft_consensus.cc:697] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:30.785981  4718 consensus_queue.cc:237] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:30.792151  4717 raft_consensus.cc:493] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.792862  4717 raft_consensus.cc:515] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:30.794546  3474 raft_consensus.cc:1275] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 LEARNER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:30.795209  2841 raft_consensus.cc:1275] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:30.795573  4717 leader_election.cc:290] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:30.797127  4721 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:30.802014  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:30.803593  4721 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:22:30.807385  3440 leader_election.cc:336] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:30.808523  3440 leader_election.cc:304] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:30.818351  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:30.819712  2917 raft_consensus.cc:2468] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 2.
I20260709 12:22:30.835047  4711 raft_consensus.cc:2749] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:30.835579  2807 leader_election.cc:336] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:30.842326  2881 consensus_peers.cc:597] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:30.856117  2807 leader_election.cc:304] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:30.863849  4719 raft_consensus.cc:2804] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:22:30.876344  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 67923.1us
I20260709 12:22:30.865630  4721 raft_consensus.cc:493] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.882879  4719 raft_consensus.cc:493] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
W20260709 12:22:30.911310  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 53577.4us
I20260709 12:22:30.919425  4719 raft_consensus.cc:3060] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.890226  4721 raft_consensus.cc:515] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:30.937090  4743 raft_consensus.cc:493] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:30.937817  4743 raft_consensus.cc:515] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:30.940361  4743 leader_election.cc:290] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:30.957206  4721 leader_election.cc:290] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:30.967878  2664 catalog_manager.cc:5697] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.004329  4719 raft_consensus.cc:515] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.007643  4719 leader_election.cc:290] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:31.010818  2881 consensus_peers.cc:597] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:31.019730  4721 raft_consensus.cc:493] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.020586  4721 raft_consensus.cc:515] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.023663  4721 leader_election.cc:290] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
W20260709 12:22:31.024679  2881 leader_election.cc:336] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.042544  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:31.043593  2910 raft_consensus.cc:2468] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 2.
W20260709 12:22:31.045776  2881 leader_election.cc:336] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.055918  3474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe7ee185c0114c079f53469226dee8bf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:31.056926  3474 raft_consensus.cc:2468] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:22:31.059360  4718 raft_consensus.cc:493] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.060045  4718 raft_consensus.cc:515] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.063308  4718 leader_election.cc:290] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:31.066273  3472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:31.068725  4721 raft_consensus.cc:493] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.068902  4751 raft_consensus.cc:493] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.069380  4721 raft_consensus.cc:515] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.069948  4751 raft_consensus.cc:515] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.072227  4721 leader_election.cc:290] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:31.072399  4751 leader_election.cc:290] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:31.075495  3473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
W20260709 12:22:31.076265  2807 leader_election.cc:336] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.083578  4679 raft_consensus.cc:493] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.084455  4679 raft_consensus.cc:515] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.086306  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a061e32042842a9a967750b68aa8abf" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:31.086964  2910 raft_consensus.cc:3060] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.087271  4679 leader_election.cc:290] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:31.088869  2807 leader_election.cc:336] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:31.083631  2881 leader_election.cc:336] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:31.089632  2881 leader_election.cc:336] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.076309  3473 raft_consensus.cc:2468] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:31.094604  4737 raft_consensus.cc:493] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.095284  4737 raft_consensus.cc:515] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.095873  2910 raft_consensus.cc:2468] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 3.
I20260709 12:22:31.102908  4737 leader_election.cc:290] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:31.089663  2807 consensus_peers.cc:597] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.066505  4746 raft_consensus.cc:493] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.115003  4746 raft_consensus.cc:515] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.117393  4746 leader_election.cc:290] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:31.117396  3440 consensus_peers.cc:597] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:31.122910  4718 raft_consensus.cc:493] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.123596  4718 raft_consensus.cc:515] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.125734  4744 raft_consensus.cc:493] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.126402  4744 raft_consensus.cc:515] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.127765  4718 leader_election.cc:290] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:31.128813  4744 leader_election.cc:290] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:31.129406  4744 raft_consensus.cc:493] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.131170  4744 raft_consensus.cc:515] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:22:31.132190  2881 leader_election.cc:336] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:31.132887  3440 consensus_peers.cc:597] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.133711  4744 leader_election.cc:290] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:31.136116  2917 raft_consensus.cc:1275] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:31.136298  2883 leader_election.cc:304] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5
W20260709 12:22:31.137945  3440 leader_election.cc:336] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.138432  4745 raft_consensus.cc:2804] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:22:31.138638  3440 leader_election.cc:336] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.138886  4745 raft_consensus.cc:493] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.139290  4745 raft_consensus.cc:3060] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:31.141503  2881 leader_election.cc:336] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.142472  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:31.143169  2910 raft_consensus.cc:2468] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 3.
I20260709 12:22:31.144520  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:31.145157  2910 raft_consensus.cc:2468] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 2.
I20260709 12:22:31.145470  2807 leader_election.cc:304] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:31.147171  4719 raft_consensus.cc:2804] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:31.147640  4719 raft_consensus.cc:493] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.148097  4719 raft_consensus.cc:3060] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.147966  4751 raft_consensus.cc:2955] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:31.152796  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:31.154362  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:31.157934  2841 raft_consensus.cc:2468] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:31.156364  4737 consensus_queue.cc:1048] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:31.159662  3440 leader_election.cc:304] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:31.160604  4737 raft_consensus.cc:2804] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:31.161010  4737 raft_consensus.cc:493] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.161361  4737 raft_consensus.cc:3060] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:31.162070  2881 leader_election.cc:336] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.162557  2841 raft_consensus.cc:2955] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:31.165645  2807 leader_election.cc:304] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:31.172674  4737 raft_consensus.cc:515] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.173867  4743 raft_consensus.cc:2804] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:31.175277  3440 leader_election.cc:304] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:31.177109  4743 raft_consensus.cc:697] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:31.177867  4737 leader_election.cc:290] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:31.178133  4743 consensus_queue.cc:237] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.181090  2880 leader_election.cc:304] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:31.184800  4744 raft_consensus.cc:2804] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:31.185247  4744 raft_consensus.cc:493] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.185657  4744 raft_consensus.cc:3060] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.195779  4744 raft_consensus.cc:515] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.156466  4745 raft_consensus.cc:515] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.205794  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:31.206744  2910 raft_consensus.cc:2393] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a10672cd53140d59ff945f75345cc06 in current term 3: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:22:31.209463  4679 raft_consensus.cc:2804] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:31.210060  4679 raft_consensus.cc:493] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.210448  4679 raft_consensus.cc:3060] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:31.219008  3440 leader_election.cc:336] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.221294  4744 leader_election.cc:290] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:31.222868  4679 raft_consensus.cc:515] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.233238  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:31.230501  4745 leader_election.cc:290] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:31.234985  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6487a298e2bb425bbfe83bed12805837" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:31.275867  2917 raft_consensus.cc:3060] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.276263  2841 raft_consensus.cc:3060] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:31.238513  3439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 66514.2us
I20260709 12:22:31.284406  2917 raft_consensus.cc:2468] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 4.
I20260709 12:22:31.285856  3440 leader_election.cc:304] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:31.288340  4711 raft_consensus.cc:2804] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:31.140975  2883 leader_election.cc:304] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:31.288971  4711 raft_consensus.cc:697] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 4 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:31.289476  4679 leader_election.cc:290] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:31.290153  4745 raft_consensus.cc:2804] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:31.290757  4745 raft_consensus.cc:493] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.290100  4711 consensus_queue.cc:237] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.290814  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:31.291265  4745 raft_consensus.cc:3060] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:31.295557  3440 leader_election.cc:336] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:31.303767  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 162863us
I20260709 12:22:31.306144  3472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4cde589c024472b0a4cec236ea69a3" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:31.306942  3472 raft_consensus.cc:2468] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:22:31.306242  2664 catalog_manager.cc:5697] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.310592  2841 raft_consensus.cc:2468] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
I20260709 12:22:31.312100  2880 leader_election.cc:304] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:31.313372  4746 raft_consensus.cc:2804] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:31.313920  4746 raft_consensus.cc:697] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:31.314927  4746 consensus_queue.cc:237] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.316901  2883 leader_election.cc:304] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:31.323871  4749 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.324604  4712 raft_consensus.cc:2804] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:31.325112  4712 raft_consensus.cc:493] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.325590  4712 raft_consensus.cc:3060] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.326272  3474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a66a027320bb4d04bb8735cd6fbffabf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
W20260709 12:22:31.265879  2807 leader_election.cc:336] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.329793  2807 leader_election.cc:304] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4
W20260709 12:22:31.330794  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 64953.1us
I20260709 12:22:31.330993  3472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4" is_pre_election: true
I20260709 12:22:31.331733  3472 raft_consensus.cc:2468] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:31.324630  4749 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:22:31.335873  1517 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 2789)
I20260709 12:22:31.334671  4743 raft_consensus.cc:2749] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:31.339298  3473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe7ee185c0114c079f53469226dee8bf" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:31.340067  3473 raft_consensus.cc:3060] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:31.273576  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 53385.9us
I20260709 12:22:31.346807  4721 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.347424  4721 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.350003  4721 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:31.350587  4743 raft_consensus.cc:493] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.350819  2836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:31.351558  2836 raft_consensus.cc:2468] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 4.
I20260709 12:22:31.351344  4743 raft_consensus.cc:515] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.352880  2880 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:31.353855  4747 raft_consensus.cc:493] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.353940  4746 raft_consensus.cc:2804] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:31.354576  4746 raft_consensus.cc:493] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.355057  4746 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:31.354622  4747 raft_consensus.cc:515] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.350963  4749 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:31.335096  3474 raft_consensus.cc:2955] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:31.360644  4743 leader_election.cc:290] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:31.357967  4747 leader_election.cc:290] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:31.361929  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 45138.2us
W20260709 12:22:31.365334  3440 leader_election.cc:336] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.365886  4746 raft_consensus.cc:515] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.366822  3473 raft_consensus.cc:2468] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:31.369668  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:31.370543  2841 raft_consensus.cc:3060] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:31.373922  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:31.374701  2837 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 5.
I20260709 12:22:31.312270  4719 raft_consensus.cc:515] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.376021  2843 raft_consensus.cc:2393] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cc87046f1414b27928722690a5e388f in current term 3: Already voted for candidate 9a10672cd53140d59ff945f75345cc06 in this term.
I20260709 12:22:31.378896  2880 leader_election.cc:304] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:31.369448  4712 raft_consensus.cc:515] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.379875  4719 leader_election.cc:290] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:31.380252  4702 raft_consensus.cc:493] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.381140  2883 leader_election.cc:304] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4
I20260709 12:22:31.381621  4751 raft_consensus.cc:3060] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.381879  4712 leader_election.cc:290] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:31.382680  4687 raft_consensus.cc:2749] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:31.382795  2883 leader_election.cc:304] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:31.384115  4687 raft_consensus.cc:2804] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:31.384533  4687 raft_consensus.cc:493] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.384956  4687 raft_consensus.cc:3060] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.339001  4745 raft_consensus.cc:515] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.386194  2917 raft_consensus.cc:2393] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f440bbb3ddd545819242017140636ad4 in current term 3: Already voted for candidate 7cc87046f1414b27928722690a5e388f in this term.
I20260709 12:22:31.387229  4745 leader_election.cc:290] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:31.391229  3440 leader_election.cc:304] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:31.392714  4679 raft_consensus.cc:2749] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:31.380852  4702 raft_consensus.cc:515] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.395218  4702 leader_election.cc:290] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:31.398984  2837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99bcb73e9233433a85aae0da2439d851" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:31.399609  4751 raft_consensus.cc:2749] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:31.399803  2837 raft_consensus.cc:2468] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 1.
I20260709 12:22:31.400049  2664 catalog_manager.cc:5697] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 4 to 5, f440bbb3ddd545819242017140636ad4 (127.1.123.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } attrs { replace: true } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:31.402102  2880 leader_election.cc:304] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:31.403565  4702 raft_consensus.cc:2804] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:31.404032  4702 raft_consensus.cc:493] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.404520  4702 raft_consensus.cc:3060] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.410486  4687 raft_consensus.cc:515] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.418726  2910 raft_consensus.cc:1275] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:31.421401  2841 raft_consensus.cc:2468] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 5.
I20260709 12:22:31.422854  2880 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
W20260709 12:22:31.424511  2807 leader_election.cc:336] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.425155  2807 leader_election.cc:304] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:31.435264  3439 leader_election.cc:304] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:31.442682  4736 raft_consensus.cc:2804] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:31.443234  4736 raft_consensus.cc:493] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.443607  4736 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:31.445112  3440 consensus_peers.cc:597] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.425612  4746 leader_election.cc:290] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:31.427150  4745 raft_consensus.cc:2804] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:31.447002  4745 raft_consensus.cc:697] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 5 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:31.448014  4745 consensus_queue.cc:237] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:22:31.449807  3440 consensus_peers.cc:597] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.455125  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe9fc4cceb94485ca488ce52e6d94e25" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:31.459553  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
W20260709 12:22:31.462942  3440 consensus_peers.cc:597] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.437098  4749 raft_consensus.cc:2749] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:31.466502  2917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa6b7efbb774150809e590d3dde361f" candidate_uuid: "9a10672cd53140d59ff945f75345cc06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
W20260709 12:22:31.467196  2807 consensus_peers.cc:597] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:31.454200  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 69896.9us
W20260709 12:22:31.467892  2807 consensus_peers.cc:597] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.475358  2917 raft_consensus.cc:2468] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a10672cd53140d59ff945f75345cc06 in term 3.
I20260709 12:22:31.468650  4687 leader_election.cc:290] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), f440bbb3ddd545819242017140636ad4 (127.1.123.68:35397)
I20260709 12:22:31.476569  2883 leader_election.cc:304] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:31.479677  4687 raft_consensus.cc:2804] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:31.480191  4687 raft_consensus.cc:697] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:31.481202  4687 consensus_queue.cc:237] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.486001  2662 catalog_manager.cc:5697] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 3 to 4, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 4 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:31.497776  4702 raft_consensus.cc:515] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.500258  2807 leader_election.cc:304] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:31.515355  4748 raft_consensus.cc:2804] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:22:31.515539  2807 leader_election.cc:336] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.515933  4748 raft_consensus.cc:697] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Becoming Leader. State: Replica: 9a10672cd53140d59ff945f75345cc06, State: Running, Role: LEADER
I20260709 12:22:31.516072  2807 leader_election.cc:304] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:31.516952  4749 raft_consensus.cc:2749] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:31.504236  4743 consensus_queue.cc:1048] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:22:31.517094  2807 leader_election.cc:336] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:31.513044  2881 leader_election.cc:336] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.519409  2807 leader_election.cc:304] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
W20260709 12:22:31.519840  2881 leader_election.cc:336] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:31.520709  2881 consensus_peers.cc:597] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:31.521140  2881 consensus_peers.cc:597] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:31.521575  2881 consensus_peers.cc:597] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:31.525779  2807 leader_election.cc:336] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:31.524010  2881 leader_election.cc:336] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.452854  4736 raft_consensus.cc:515] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.527931  3474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4cde589c024472b0a4cec236ea69a3" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:31.528556  3474 raft_consensus.cc:3060] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.529271  4736 leader_election.cc:290] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:31.523090  4718 raft_consensus.cc:2749] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:31.516901  4748 consensus_queue.cc:237] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.533092  3471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:31.533690  4745 raft_consensus.cc:493] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.534134  3471 raft_consensus.cc:2393] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7cc87046f1414b27928722690a5e388f in current term 3: Already voted for candidate f440bbb3ddd545819242017140636ad4 in this term.
I20260709 12:22:31.534403  4745 raft_consensus.cc:515] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:22:31.535305  3440 leader_election.cc:336] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.538327  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06" is_pre_election: true
I20260709 12:22:31.538987  2841 raft_consensus.cc:2468] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:31.540268  2880 leader_election.cc:304] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
W20260709 12:22:31.545006  2807 consensus_peers.cc:597] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:31.545794  2807 consensus_peers.cc:597] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.548890  4712 raft_consensus.cc:2804] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:31.549437  4712 raft_consensus.cc:493] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 442c92e5267046cd92f46e6f469570a5)
I20260709 12:22:31.549932  4712 raft_consensus.cc:3060] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.550325  3473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6855fa72e5f9486bb478855f721e77ce" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f440bbb3ddd545819242017140636ad4"
I20260709 12:22:31.551002  3473 raft_consensus.cc:3060] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.554286  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62274aac902146f7b127a2afb55b8a12" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:31.554960  2841 raft_consensus.cc:3060] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:31.558082  3440 consensus_peers.cc:597] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.561561  4745 leader_election.cc:290] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:31.581264  3440 consensus_peers.cc:597] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.585094  3474 raft_consensus.cc:2468] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:31.598181  4702 leader_election.cc:290] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:31.598766  4712 raft_consensus.cc:515] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.600253  2664 catalog_manager.cc:5697] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 9a10672cd53140d59ff945f75345cc06 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "9a10672cd53140d59ff945f75345cc06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.620684  2841 raft_consensus.cc:2468] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 6.
I20260709 12:22:31.623737  2843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99bcb73e9233433a85aae0da2439d851" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:31.627110  2843 raft_consensus.cc:3060] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.626122  3473 raft_consensus.cc:2468] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
W20260709 12:22:31.626291  2881 leader_election.cc:336] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:31.624414  3440 consensus_peers.cc:597] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.628181  3439 leader_election.cc:304] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9a10672cd53140d59ff945f75345cc06, f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5
I20260709 12:22:31.630110  2883 leader_election.cc:304] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
W20260709 12:22:31.632606  2881 leader_election.cc:336] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:31.633155  2881 leader_election.cc:336] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.638404  4712 leader_election.cc:290] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:31.638947  4736 raft_consensus.cc:2804] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:31.639503  4736 raft_consensus.cc:697] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 6 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:31.640447  4736 consensus_queue.cc:237] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:31.641762  4757 raft_consensus.cc:2804] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:31.642253  2841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15476716416b4e729148408ed7846297" candidate_uuid: "7cc87046f1414b27928722690a5e388f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9a10672cd53140d59ff945f75345cc06"
I20260709 12:22:31.642436  4757 raft_consensus.cc:697] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:31.642849  2841 raft_consensus.cc:3060] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.643368  4757 consensus_queue.cc:237] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
W20260709 12:22:31.655057  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 72435.7us
W20260709 12:22:31.655942  2807 consensus_peers.cc:597] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:31.633525  2881 leader_election.cc:304] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc87046f1414b27928722690a5e388f; no voters: 442c92e5267046cd92f46e6f469570a5, f440bbb3ddd545819242017140636ad4
I20260709 12:22:31.660907  2841 raft_consensus.cc:2468] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 3.
I20260709 12:22:31.662252  2843 raft_consensus.cc:2468] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc87046f1414b27928722690a5e388f in term 2.
I20260709 12:22:31.664224  2880 leader_election.cc:304] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:31.668769  2880 leader_election.cc:304] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06; no voters: 
I20260709 12:22:31.669771  4758 raft_consensus.cc:2804] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:31.670368  4758 raft_consensus.cc:697] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:31.670712  4702 raft_consensus.cc:2804] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:31.671299  4702 raft_consensus.cc:697] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:31.671343  4758 consensus_queue.cc:237] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
I20260709 12:22:31.672272  4702 consensus_queue.cc:237] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } }
W20260709 12:22:31.673353  2881 leader_election.cc:336] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.673866  4712 raft_consensus.cc:2749] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:22:31.674341  2881 leader_election.cc:336] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.686269  2883 leader_election.cc:304] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5
W20260709 12:22:31.691946  2807 consensus_peers.cc:597] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:31.692667  2807 consensus_peers.cc:597] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.712473  4712 raft_consensus.cc:2804] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:31.713197  4712 raft_consensus.cc:697] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Becoming Leader. State: Replica: 7cc87046f1414b27928722690a5e388f, State: Running, Role: LEADER
I20260709 12:22:31.717867  4712 consensus_queue.cc:237] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
W20260709 12:22:31.785420  2881 leader_election.cc:336] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:31.786520  2881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 60795.9us
W20260709 12:22:31.787209  2881 leader_election.cc:336] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.790681  2910 consensus_queue.cc:237] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
W20260709 12:22:31.803741  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 48223.7us
W20260709 12:22:31.823825  2807 consensus_peers.cc:597] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:31.824436  2807 consensus_peers.cc:597] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:31.834048  3440 consensus_peers.cc:597] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.845060  2837 raft_consensus.cc:1275] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:31.848106  3473 raft_consensus.cc:1275] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:31.847987  4758 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:22:31.869663  4687 consensus_queue.cc:1048] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:22:31.909268  2880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 53613.2us
W20260709 12:22:31.855261  4765 log.cc:927] Time spent T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 12:22:31.925570  2807 consensus_peers.cc:597] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:31.950358  2910 raft_consensus.cc:1275] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:22:31.926220  2807 consensus_peers.cc:597] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:31.970775  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 44621.1us
W20260709 12:22:31.973187  2881 leader_election.cc:336] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:31.961360  2662 catalog_manager.cc:5697] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 5 to 6, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 6 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:22:32.001011  3440 consensus_peers.cc:597] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:32.021749  2843 raft_consensus.cc:1275] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:22:32.022966  3440 consensus_peers.cc:597] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:32.026836  4743 consensus_queue.cc:1048] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:32.030125  2843 raft_consensus.cc:1275] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:22:32.033591  2881 consensus_peers.cc:597] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:32.039081  2664 catalog_manager.cc:5697] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 4 to 5, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 5 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:32.046720  2664 catalog_manager.cc:5697] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 1 to 2, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 2 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:32.048727  2664 catalog_manager.cc:5697] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:32.050823  2664 catalog_manager.cc:5697] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 1 to 2, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 2 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:32.053397  2881 consensus_peers.cc:597] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:32.057891  4777 consensus_queue.cc:1048] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:32.067991  4702 consensus_queue.cc:1048] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:32.075794  2881 consensus_peers.cc:597] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:32.083742  2881 consensus_peers.cc:597] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:32.084409  2881 consensus_peers.cc:597] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:32.092974  2881 consensus_peers.cc:597] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:32.095364  4775 raft_consensus.cc:2955] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:32.104688  2842 raft_consensus.cc:1275] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 6 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:22:32.107546  2917 raft_consensus.cc:1275] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:22:32.107506  2836 raft_consensus.cc:2955] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:32.121196  2664 catalog_manager.cc:5697] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:32.123523  2664 catalog_manager.cc:5697] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 1 to 2, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 2 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:32.125643  2664 catalog_manager.cc:5697] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f reported cstate change: term changed from 2 to 3, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to 7cc87046f1414b27928722690a5e388f (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:32.152068  3469 raft_consensus.cc:1275] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:32.158159  4778 consensus_queue.cc:1048] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:32.208823  2807 consensus_peers.cc:597] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:32.212262  3440 consensus_peers.cc:597] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:32.205855  3469 raft_consensus.cc:2955] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:32.237332  3472 raft_consensus.cc:1275] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:22:32.257670  2881 consensus_peers.cc:597] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:32.259605  2842 raft_consensus.cc:1275] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:22:32.261767  2881 consensus_peers.cc:597] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:32.274240  2843 raft_consensus.cc:1275] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:32.265552  4747 consensus_queue.cc:1048] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:22:32.273216  4746 consensus_queue.cc:1048] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
W20260709 12:22:32.279198  2881 consensus_peers.cc:597] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:32.283685  4745 consensus_queue.cc:1048] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:32.298951  2910 raft_consensus.cc:1275] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Refusing update from remote peer 9a10672cd53140d59ff945f75345cc06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:32.299913  3473 raft_consensus.cc:1275] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Refusing update from remote peer 7cc87046f1414b27928722690a5e388f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:22:32.300024  3440 consensus_peers.cc:597] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:32.300768  3440 consensus_peers.cc:597] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:32.305858  4758 consensus_queue.cc:1048] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:32.305213  2651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c9f24d490dc840f1b2d87604abe412d8 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:22:32.308102  2662 catalog_manager.cc:5697] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 reported cstate change: config changed from index 5 to 6, VOTER 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) evicted. New cstate: current_term: 3 leader_uuid: "7cc87046f1414b27928722690a5e388f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } } }
I20260709 12:22:32.315754  4746 consensus_queue.cc:1048] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:22:32.316670  2881 consensus_peers.cc:597] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:32.322896  2881 consensus_peers.cc:597] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:32.333083  4774 consensus_queue.cc:1048] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:22:32.328635  2807 consensus_peers.cc:597] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:32.336892  1517 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 2789)
I20260709 12:22:32.378952  4718 consensus_queue.cc:1048] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:22:32.386343  4779 log.cc:927] Time spent T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 12:22:32.471745  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 66220.3us
W20260709 12:22:32.519248  4785 log.cc:927] Time spent T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: Append to log took a long time: real 0.106s	user 0.003s	sys 0.000s
W20260709 12:22:32.558542  4791 log.cc:927] Time spent T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:22:32.871225  4794 log.cc:927] Time spent T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:22:32.877864  2651 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 339041us
W20260709 12:22:33.005826  2651 catalog_manager.cc:4755] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c9f24d490dc840f1b2d87604abe412d8: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:33.092582  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 131610us
W20260709 12:22:33.098465  4801 log.cc:927] Time spent T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f: Append to log took a long time: real 0.220s	user 0.004s	sys 0.000s
W20260709 12:22:33.113741  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:35625, user_credentials={real_user=slave}} blocked reactor thread for 57589.7us
W20260709 12:22:33.290693  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 59353.6us
W20260709 12:22:33.337441  1517 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 2789)
W20260709 12:22:33.368903  2807 consensus_peers.cc:597] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:33.410450  4803 raft_consensus.cc:493] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.411141  4803 raft_consensus.cc:515] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:33.413316  4803 leader_election.cc:290] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:33.425586  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f" is_pre_election: true
I20260709 12:22:33.426385  2910 raft_consensus.cc:2468] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 3.
I20260709 12:22:33.430908  3440 leader_election.cc:304] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:33.439172  4803 raft_consensus.cc:2804] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.439654  4803 raft_consensus.cc:493] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.439996  4803 raft_consensus.cc:3060] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:33.439206  3440 leader_election.cc:336] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:33.455060  4803 raft_consensus.cc:515] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:33.457645  4803 leader_election.cc:290] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 election: Requested vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:33.473471  2910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500b7bb3bd042f09ffa7310e921aa24" candidate_uuid: "f440bbb3ddd545819242017140636ad4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7cc87046f1414b27928722690a5e388f"
I20260709 12:22:33.474900  2910 raft_consensus.cc:3060] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.507211  2910 raft_consensus.cc:2468] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f440bbb3ddd545819242017140636ad4 in term 4.
I20260709 12:22:33.514529  3440 leader_election.cc:304] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc87046f1414b27928722690a5e388f, f440bbb3ddd545819242017140636ad4; no voters: 
I20260709 12:22:33.516866  4737 raft_consensus.cc:2804] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:22:33.537441  3440 leader_election.cc:336] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:33.546052  4737 raft_consensus.cc:697] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 4 LEADER]: Becoming Leader. State: Replica: f440bbb3ddd545819242017140636ad4, State: Running, Role: LEADER
I20260709 12:22:33.555207  4737 consensus_queue.cc:237] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:33.669509  2662 catalog_manager.cc:5697] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 reported cstate change: term changed from 2 to 4, leader changed from 442c92e5267046cd92f46e6f469570a5 (127.1.123.65) to f440bbb3ddd545819242017140636ad4 (127.1.123.68). New cstate: current_term: 4 leader_uuid: "f440bbb3ddd545819242017140636ad4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:22:33.721858  3440 consensus_peers.cc:597] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:33.788249  2881 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111) [suppressed 387 similar messages]
W20260709 12:22:33.872444  2807 consensus_peers.cc:597] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:33.882097  3440 consensus_peers.cc:597] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:33.901906  2881 consensus_peers.cc:597] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:33.929795  2807 consensus_peers.cc:597] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:33.959484  3440 consensus_peers.cc:597] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:33.962787  3440 consensus_peers.cc:597] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.025872  2807 consensus_peers.cc:597] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.033231  2881 consensus_peers.cc:597] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.036870  2881 consensus_peers.cc:597] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:34.074294  2881 consensus_peers.cc:597] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.075399  3440 consensus_peers.cc:597] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.116263  3440 consensus_peers.cc:597] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:34.121604  2911 raft_consensus.cc:1275] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Refusing update from remote peer f440bbb3ddd545819242017140636ad4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:22:34.134925  2807 consensus_peers.cc:597] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.143121  2807 consensus_peers.cc:597] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:34.174556  4737 consensus_queue.cc:1048] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:22:34.318125  2807 consensus_peers.cc:597] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.338204  1517 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 2789)
W20260709 12:22:34.422745  2651 catalog_manager.cc:4755] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c9f24d490dc840f1b2d87604abe412d8: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:34.545451  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 82029.4us
W20260709 12:22:34.553748  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 43514.9us
W20260709 12:22:34.567272  2883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 57640.2us
W20260709 12:22:34.608311  3440 consensus_peers.cc:597] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:34.609040  3440 consensus_peers.cc:597] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:34.634163  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 52395us
W20260709 12:22:34.636541  2807 consensus_peers.cc:597] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.637207  2807 consensus_peers.cc:597] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.650758  2807 consensus_peers.cc:597] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:34.661412  2881 consensus_peers.cc:597] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.668298  2807 consensus_peers.cc:597] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.682461  3440 consensus_peers.cc:597] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.686368  3440 consensus_peers.cc:597] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.687311  2881 consensus_peers.cc:597] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:34.693395  3440 consensus_peers.cc:597] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.706230  3440 consensus_peers.cc:597] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.708140  3440 consensus_peers.cc:597] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.716722  2807 consensus_peers.cc:597] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.721700  2881 consensus_peers.cc:597] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:34.825603  2807 consensus_peers.cc:597] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:34.826752  3440 consensus_peers.cc:597] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.829288  2881 consensus_peers.cc:597] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:34.851272  2881 consensus_peers.cc:597] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:34.979787  2881 consensus_peers.cc:597] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:35.033208  2881 consensus_peers.cc:597] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:35.042505  3440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 73748.8us
W20260709 12:22:35.068089  2881 consensus_peers.cc:597] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:35.106699  2881 consensus_peers.cc:597] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:35.171563  2807 consensus_peers.cc:597] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:35.183619  2881 consensus_peers.cc:597] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:35.184151  2881 consensus_peers.cc:597] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:35.197641  3440 consensus_peers.cc:597] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:35.565747  1517 tablet_server.cc:196] TabletServer@127.1.123.65:0 shutdown complete.
W20260709 12:22:35.681798  2651 catalog_manager.cc:4755] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c9f24d490dc840f1b2d87604abe412d8: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:35.717165  2809 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:35397, user_credentials={real_user=slave}} blocked reactor thread for 55180.2us
W20260709 12:22:35.774986  3439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:44215, user_credentials={real_user=slave}} blocked reactor thread for 115991us
W20260709 12:22:35.795034  2807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:37395, user_credentials={real_user=slave}} blocked reactor thread for 133894us
W20260709 12:22:35.823576  2807 consensus_peers.cc:597] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:22:35.877398  1517 tablet_server.cc:179] TabletServer@127.1.123.66:0 shutting down...
W20260709 12:22:35.914527  2880 consensus_peers.cc:597] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:35.923724  3439 consensus_peers.cc:597] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:35.946358  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:35.949393  1517 tablet_replica.cc:333] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:35.951251  1517 raft_consensus.cc:2243] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:35.952191  1517 raft_consensus.cc:2272] T eface8ee310347d4bb08e3c5864ce5d9 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:35.961858  3439 consensus_peers.cc:597] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:35.962747  1517 tablet_replica.cc:333] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
W20260709 12:22:35.963814  2880 consensus_peers.cc:597] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:35.963855  1517 raft_consensus.cc:2243] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:35.965963  1517 raft_consensus.cc:2272] T 0e4b953250214a6f8f3b4794fab90ce0 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.969102  1517 tablet_replica.cc:333] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:35.970718  1517 raft_consensus.cc:2243] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:35.971319  3439 consensus_peers.cc:597] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:35.972981  3439 consensus_peers.cc:597] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:35.971525  1517 raft_consensus.cc:2272] T 386a0166b36244f789cc8361a1cb0a46 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.976101  1517 tablet_replica.cc:333] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:35.976996  1517 raft_consensus.cc:2243] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:35.978153  1517 raft_consensus.cc:2272] T e80bb6a87c6e403fa76954737ffda91a P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.980777  1517 tablet_replica.cc:333] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:35.981420  1517 raft_consensus.cc:2243] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:35.981992  1517 raft_consensus.cc:2272] T d436a6cfd22640bea76c0256f1d01684 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.984010  1517 tablet_replica.cc:333] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:35.984629  1517 raft_consensus.cc:2243] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:35.985507  1517 raft_consensus.cc:2272] T a9961074150e45dbb36db8e49d840c5d P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.987751  1517 tablet_replica.cc:333] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:35.988581  1517 raft_consensus.cc:2243] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:35.989650  1517 raft_consensus.cc:2272] T 56d2861f96e247a78ce40f9dfe9fe850 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.992354  1517 tablet_replica.cc:333] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:35.993247  1517 raft_consensus.cc:2243] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:22:35.993914  2880 consensus_peers.cc:597] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:35.994462  1517 raft_consensus.cc:2272] T 53752656656243c3abe78c7a9cb488c7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.997956  1517 tablet_replica.cc:333] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:35.999002  1517 raft_consensus.cc:2243] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:35.999617  1517 raft_consensus.cc:2272] T b46ad496aaf14cbc8cf01697dceb4cf5 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.002985  1517 tablet_replica.cc:333] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.003923  1517 raft_consensus.cc:2243] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.004530  1517 raft_consensus.cc:2272] T 4ff32c54020945d99bd67b7df9574bae P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:36.006561  3439 consensus_peers.cc:597] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.007401  1517 tablet_replica.cc:333] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.008361  1517 raft_consensus.cc:2243] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.009512  1517 raft_consensus.cc:2272] T 1a061e32042842a9a967750b68aa8abf P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.012588  1517 tablet_replica.cc:333] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.013583  1517 raft_consensus.cc:2243] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.014732  1517 raft_consensus.cc:2272] T 6979a63415d34b6d91bb669ecec01bdf P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.017567  1517 tablet_replica.cc:333] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.018543  1517 raft_consensus.cc:2243] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.019217  1517 raft_consensus.cc:2272] T cf19d36b4d1b4825a45f6749aaa19896 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.022074  1517 tablet_replica.cc:333] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.023072  1517 raft_consensus.cc:2243] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:22:36.023435  2880 consensus_peers.cc:597] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:36.024390  2880 consensus_peers.cc:597] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.024503  1517 raft_consensus.cc:2272] T 2aa6b7efbb774150809e590d3dde361f P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.027496  1517 tablet_replica.cc:333] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.028501  1517 raft_consensus.cc:2243] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.029273  1517 raft_consensus.cc:2272] T c737810e34ba42deaa41e6478e84fe87 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.032366  1517 tablet_replica.cc:333] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.033234  1517 raft_consensus.cc:2243] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.033869  1517 raft_consensus.cc:2272] T 41f5e1f4763f4d978619c2696e7aaa71 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.036556  1517 tablet_replica.cc:333] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.037433  1517 raft_consensus.cc:2243] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.038655  1517 raft_consensus.cc:2272] T 659c2c3a61024d07a2296bc83d87a52d P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.041482  1517 tablet_replica.cc:333] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.042488  1517 raft_consensus.cc:2243] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.043190  1517 raft_consensus.cc:2272] T e07072d40c8b441e8c50e662b2daf24c P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.045976  1517 tablet_replica.cc:333] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.046957  1517 raft_consensus.cc:2243] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.047684  1517 raft_consensus.cc:2272] T 2594ea75a8d44503a64ea4e06916c4b3 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:36.048940  2880 consensus_peers.cc:597] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.050496  1517 tablet_replica.cc:333] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.051493  1517 raft_consensus.cc:2243] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.052376  1517 raft_consensus.cc:2272] T 62274aac902146f7b127a2afb55b8a12 P 9a10672cd53140d59ff945f75345cc06 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.055186  1517 tablet_replica.cc:333] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.056116  1517 raft_consensus.cc:2243] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:22:36.056227  2880 consensus_peers.cc:597] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.057381  1517 raft_consensus.cc:2272] T 627b3359300e44fc9d2b83ed88272a9f P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.060182  1517 tablet_replica.cc:333] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.061272  1517 raft_consensus.cc:2243] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.062191  1517 raft_consensus.cc:2272] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.065075  1517 tablet_replica.cc:333] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.065966  1517 raft_consensus.cc:2243] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.066599  1517 raft_consensus.cc:2272] T 547d5e0cde57459bbad31d06eb64d442 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.069353  1517 tablet_replica.cc:333] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.070307  1517 raft_consensus.cc:2243] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.070958  1517 raft_consensus.cc:2272] T f1605919970c4ce68b488b932d3e81b3 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:36.073330  3439 consensus_peers.cc:597] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.073916  1517 tablet_replica.cc:333] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.074923  1517 raft_consensus.cc:2243] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.075899  1517 raft_consensus.cc:2272] T 532f0b78131648af8fd4930674a8bce0 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.078773  1517 tablet_replica.cc:333] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.082459  1517 raft_consensus.cc:2243] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.083211  1517 raft_consensus.cc:2272] T a2f8a42191e14f07a43032df18499a9c P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.086392  1517 tablet_replica.cc:333] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.087306  1517 raft_consensus.cc:2243] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.088501  1517 raft_consensus.cc:2272] T 021d84b2e9364749b9ae4e7fde0b14bc P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.091934  1517 tablet_replica.cc:333] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.092856  1517 raft_consensus.cc:2243] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.094072  1517 raft_consensus.cc:2272] T 01e8aac6ebd14bf4b952c62c8075f13f P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:36.096441  3439 consensus_peers.cc:597] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.096879  1517 tablet_replica.cc:333] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.098449  1517 raft_consensus.cc:2243] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.099411  1517 raft_consensus.cc:2272] T dc860efbfaf84207b7128fd001b59fe6 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.102532  1517 tablet_replica.cc:333] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.104327  1517 raft_consensus.cc:2243] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.104957  1517 raft_consensus.cc:2272] T fe9fc4cceb94485ca488ce52e6d94e25 P 9a10672cd53140d59ff945f75345cc06 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:36.105979  2880 consensus_peers.cc:597] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.107801  1517 tablet_replica.cc:333] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.108843  1517 raft_consensus.cc:2243] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.109997  1517 raft_consensus.cc:2272] T fa220c2195724c24b522d44636d98970 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.112876  1517 tablet_replica.cc:333] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.113869  1517 raft_consensus.cc:2243] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.114565  1517 raft_consensus.cc:2272] T 99bcb73e9233433a85aae0da2439d851 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.117501  1517 tablet_replica.cc:333] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.118391  1517 raft_consensus.cc:2243] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.119611  1517 raft_consensus.cc:2272] T 710edfdfdd7e4720b77f4ec27f5f41c0 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:36.120730  2880 consensus_peers.cc:597] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.122394  1517 tablet_replica.cc:333] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.123385  1517 raft_consensus.cc:2243] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.124388  1517 raft_consensus.cc:2272] T 2a6389f65afb4adabe1d70b6f09e99e7 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.127336  1517 tablet_replica.cc:333] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.128286  1517 raft_consensus.cc:2243] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.129292  1517 raft_consensus.cc:2272] T c9f24d490dc840f1b2d87604abe412d8 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.132285  1517 tablet_replica.cc:333] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.133141  1517 raft_consensus.cc:2243] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.133828  1517 raft_consensus.cc:2272] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.136541  1517 tablet_replica.cc:333] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.137353  1517 raft_consensus.cc:2243] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.138013  1517 raft_consensus.cc:2272] T 15476716416b4e729148408ed7846297 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.140762  1517 tablet_replica.cc:333] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.141585  1517 raft_consensus.cc:2243] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.142756  1517 raft_consensus.cc:2272] T f26ca468f5384641ad5bd05af362aef9 P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.145717  1517 tablet_replica.cc:333] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.146529  1517 raft_consensus.cc:2243] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.147681  1517 raft_consensus.cc:2272] T 51910d09518b4757be25a91c35ee2ec2 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.150426  1517 tablet_replica.cc:333] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.151314  1517 raft_consensus.cc:2243] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.152037  1517 raft_consensus.cc:2272] T 12034fdcbbed4121a134d5d26c1dc5ac P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.154835  1517 tablet_replica.cc:333] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.155818  1517 raft_consensus.cc:2243] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.156486  1517 raft_consensus.cc:2272] T 6487a298e2bb425bbfe83bed12805837 P 9a10672cd53140d59ff945f75345cc06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.159152  1517 tablet_replica.cc:333] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.159713  1517 raft_consensus.cc:2243] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.160329  1517 raft_consensus.cc:2272] T 63fa205fa4864ef4b19bda073c06a5d4 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.162487  1517 tablet_replica.cc:333] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
I20260709 12:22:36.163151  1517 raft_consensus.cc:2243] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.164093  1517 raft_consensus.cc:2272] T bb1bafa30972440499c69fb1221eae27 P 9a10672cd53140d59ff945f75345cc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.166419  1517 tablet_replica.cc:333] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06: stopping tablet replica
W20260709 12:22:36.166481  2880 consensus_peers.cc:597] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.167061  1517 raft_consensus.cc:2243] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.167559  1517 raft_consensus.cc:2272] T 3dbc0cbfcb67483f9f554976e140ba7f P 9a10672cd53140d59ff945f75345cc06 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:36.180241  3439 consensus_peers.cc:597] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:36.189062  3439 consensus_peers.cc:597] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:36.224954  2880 consensus_peers.cc:597] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:36.240628  2880 consensus_peers.cc:597] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:36.249104  2880 consensus_peers.cc:597] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:36.278254  2880 consensus_peers.cc:597] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:36.286724  3439 consensus_peers.cc:597] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:36.300503  2880 consensus_peers.cc:597] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:36.367256  2880 consensus_peers.cc:597] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:36.419193  3440 consensus_peers.cc:597] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:22:36.488627  3440 consensus_peers.cc:597] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:22:36.538306  2881 consensus_peers.cc:597] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:22:36.557142  2881 consensus_peers.cc:597] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:22:36.557277  1517 tablet_server.cc:196] TabletServer@127.1.123.66:0 shutdown complete.
W20260709 12:22:36.579522  3440 consensus_peers.cc:597] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:22:36.591730  3440 consensus_peers.cc:597] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:36.607975  2938 debug-util.cc:398] Leaking SignalData structure 0x7b08002fd820 after lost signal to thread 1520
W20260709 12:22:36.609257  2938 debug-util.cc:398] Leaking SignalData structure 0x7b08001b88a0 after lost signal to thread 2700
W20260709 12:22:36.610703  2938 debug-util.cc:398] Leaking SignalData structure 0x7b08001d4fe0 after lost signal to thread 2867
W20260709 12:22:36.611795  2938 debug-util.cc:398] Leaking SignalData structure 0x7b080009ce80 after lost signal to thread 2941
W20260709 12:22:36.612982  2938 debug-util.cc:398] Leaking SignalData structure 0x7b0800361400 after lost signal to thread 3503
W20260709 12:22:36.670560  3440 consensus_peers.cc:597] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:22:36.685873  2881 consensus_peers.cc:597] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:22:36.705255  1517 tablet_server.cc:179] TabletServer@127.1.123.67:0 shutting down...
W20260709 12:22:36.711732  3440 consensus_peers.cc:597] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:36.738518  3440 consensus_peers.cc:597] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:36.778483  2881 consensus_peers.cc:597] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:36.787556  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:36.793701  1517 tablet_replica.cc:333] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.794816  1517 raft_consensus.cc:2243] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.795549  1517 raft_consensus.cc:2272] T a7c3a03a1304452ebc48e986f5629037 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.798518  1517 tablet_replica.cc:333] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.799443  1517 raft_consensus.cc:2243] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.800079  1517 raft_consensus.cc:2272] T 9164313f06e045c3bdef4a46d365873c P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.802788  1517 tablet_replica.cc:333] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.803730  1517 raft_consensus.cc:2243] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.804940  1517 raft_consensus.cc:2272] T eface8ee310347d4bb08e3c5864ce5d9 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.807926  1517 tablet_replica.cc:333] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.808835  1517 raft_consensus.cc:2243] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.810073  1517 raft_consensus.cc:2272] T e07072d40c8b441e8c50e662b2daf24c P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.812815  1517 tablet_replica.cc:333] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.813748  1517 raft_consensus.cc:2243] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.814433  1517 raft_consensus.cc:2272] T e80bb6a87c6e403fa76954737ffda91a P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:36.816177  3440 consensus_peers.cc:597] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:36.817191  1517 tablet_replica.cc:333] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
W20260709 12:22:36.817997  3440 consensus_peers.cc:597] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.818222  1517 raft_consensus.cc:2243] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.819454  1517 raft_consensus.cc:2272] T 12034fdcbbed4121a134d5d26c1dc5ac P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:36.821748  3440 consensus_peers.cc:597] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.822208  1517 tablet_replica.cc:333] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.823173  1517 raft_consensus.cc:2243] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.824617  1517 raft_consensus.cc:2272] T 7e4cde589c024472b0a4cec236ea69a3 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.827430  1517 tablet_replica.cc:333] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.828356  1517 raft_consensus.cc:2243] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.828995  1517 raft_consensus.cc:2272] T 22aad5981ec94ff2a74fdf45111c0692 P 7cc87046f1414b27928722690a5e388f [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.831704  1517 tablet_replica.cc:333] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.832659  1517 raft_consensus.cc:2243] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:22:36.832894  3440 consensus_peers.cc:597] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:36.833889  1517 raft_consensus.cc:2272] T cf19d36b4d1b4825a45f6749aaa19896 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.836763  1517 tablet_replica.cc:333] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.837873  1517 raft_consensus.cc:2243] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.838512  1517 raft_consensus.cc:2272] T 56d2861f96e247a78ce40f9dfe9fe850 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.841228  1517 tablet_replica.cc:333] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.842196  1517 raft_consensus.cc:2243] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.843369  1517 raft_consensus.cc:2272] T 386a0166b36244f789cc8361a1cb0a46 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.846036  1517 tablet_replica.cc:333] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.846920  1517 raft_consensus.cc:2243] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.848040  1517 raft_consensus.cc:2272] T 6291f8f287034586b4e9c0c9220bc6e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.850716  1517 tablet_replica.cc:333] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.851604  1517 raft_consensus.cc:2243] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.852157  1517 raft_consensus.cc:2272] T 6979a63415d34b6d91bb669ecec01bdf P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.854853  1517 tablet_replica.cc:333] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.855791  1517 raft_consensus.cc:2243] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.856945  1517 raft_consensus.cc:2272] T d436a6cfd22640bea76c0256f1d01684 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.859665  1517 tablet_replica.cc:333] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.860570  1517 raft_consensus.cc:2243] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.861128  1517 raft_consensus.cc:2272] T a9961074150e45dbb36db8e49d840c5d P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.863781  1517 tablet_replica.cc:333] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.864636  1517 raft_consensus.cc:2243] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.865929  1517 raft_consensus.cc:2272] T fe7ee185c0114c079f53469226dee8bf P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.868566  1517 tablet_replica.cc:333] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.869445  1517 raft_consensus.cc:2243] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:36.869846  3440 consensus_peers.cc:597] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.870026  1517 raft_consensus.cc:2272] T 1a061e32042842a9a967750b68aa8abf P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.872668  1517 tablet_replica.cc:333] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.873587  1517 raft_consensus.cc:2243] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.874667  1517 raft_consensus.cc:2272] T 34453a67223842ed98d4220533186ade P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.877354  1517 tablet_replica.cc:333] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.878230  1517 raft_consensus.cc:2243] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.879135  1517 raft_consensus.cc:2272] T a66a027320bb4d04bb8735cd6fbffabf P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.881781  1517 tablet_replica.cc:333] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.882690  1517 raft_consensus.cc:2243] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.883267  1517 raft_consensus.cc:2272] T 2aa6b7efbb774150809e590d3dde361f P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.885815  1517 tablet_replica.cc:333] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.886680  1517 raft_consensus.cc:2243] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.887818  1517 raft_consensus.cc:2272] T c737810e34ba42deaa41e6478e84fe87 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.890718  1517 tablet_replica.cc:333] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.891736  1517 raft_consensus.cc:2243] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.892933  1517 raft_consensus.cc:2272] T 41f5e1f4763f4d978619c2696e7aaa71 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.895771  1517 tablet_replica.cc:333] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.896622  1517 raft_consensus.cc:2243] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.897258  1517 raft_consensus.cc:2272] T 42af6798b62746449070b7b2bc4e0699 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.900025  1517 tablet_replica.cc:333] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.901021  1517 raft_consensus.cc:2243] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.901880  1517 raft_consensus.cc:2272] T e3545f99312f483eb6dc8965c5bc418b P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.904950  1517 tablet_replica.cc:333] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.905990  1517 raft_consensus.cc:2243] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.907440  1517 raft_consensus.cc:2272] T 6855fa72e5f9486bb478855f721e77ce P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.910281  1517 tablet_replica.cc:333] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.911190  1517 raft_consensus.cc:2243] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.911782  1517 raft_consensus.cc:2272] T b46ad496aaf14cbc8cf01697dceb4cf5 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.914420  1517 tablet_replica.cc:333] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.915232  1517 raft_consensus.cc:2243] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.915828  1517 raft_consensus.cc:2272] T d500b7bb3bd042f09ffa7310e921aa24 P 7cc87046f1414b27928722690a5e388f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.918455  1517 tablet_replica.cc:333] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.919281  1517 raft_consensus.cc:2243] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.919798  1517 raft_consensus.cc:2272] T 021d84b2e9364749b9ae4e7fde0b14bc P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.922627  1517 tablet_replica.cc:333] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.923451  1517 raft_consensus.cc:2243] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.924021  1517 raft_consensus.cc:2272] T 7bb611b2490b4682a0c5a45f2bbe7c94 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:36.925398  3440 consensus_peers.cc:597] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.926764  1517 tablet_replica.cc:333] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.927707  1517 raft_consensus.cc:2243] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.928939  1517 raft_consensus.cc:2272] T f1605919970c4ce68b488b932d3e81b3 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.932019  1517 tablet_replica.cc:333] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.932897  1517 raft_consensus.cc:2243] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.933625  1517 raft_consensus.cc:2272] T 2a6389f65afb4adabe1d70b6f09e99e7 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.936666  1517 tablet_replica.cc:333] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.937520  1517 raft_consensus.cc:2243] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.938670  1517 raft_consensus.cc:2272] T 63fa205fa4864ef4b19bda073c06a5d4 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.941483  1517 tablet_replica.cc:333] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.942349  1517 raft_consensus.cc:2243] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.943486  1517 raft_consensus.cc:2272] T 2cba50a5280d4b00b9d2617ce7a22ec1 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.946193  1517 tablet_replica.cc:333] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.947005  1517 raft_consensus.cc:2243] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.948109  1517 raft_consensus.cc:2272] T 6487a298e2bb425bbfe83bed12805837 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.950827  1517 tablet_replica.cc:333] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.951653  1517 raft_consensus.cc:2243] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.952208  1517 raft_consensus.cc:2272] T 133424caefd34fa8b83015c9fe06d612 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.954952  1517 tablet_replica.cc:333] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.955794  1517 raft_consensus.cc:2243] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.956990  1517 raft_consensus.cc:2272] T c9f24d490dc840f1b2d87604abe412d8 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.959621  1517 tablet_replica.cc:333] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.960482  1517 raft_consensus.cc:2243] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.961668  1517 raft_consensus.cc:2272] T fe9fc4cceb94485ca488ce52e6d94e25 P 7cc87046f1414b27928722690a5e388f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.964313  1517 tablet_replica.cc:333] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.965128  1517 raft_consensus.cc:2243] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.965723  1517 raft_consensus.cc:2272] T fa220c2195724c24b522d44636d98970 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.968307  1517 tablet_replica.cc:333] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.969092  1517 raft_consensus.cc:2243] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.970288  1517 raft_consensus.cc:2272] T 99bcb73e9233433a85aae0da2439d851 P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.972893  1517 tablet_replica.cc:333] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.973698  1517 raft_consensus.cc:2243] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.974299  1517 raft_consensus.cc:2272] T 01e8aac6ebd14bf4b952c62c8075f13f P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.976863  1517 tablet_replica.cc:333] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.977705  1517 raft_consensus.cc:2243] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:36.977730  3440 consensus_peers.cc:597] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.978377  1517 raft_consensus.cc:2272] T 3dbc0cbfcb67483f9f554976e140ba7f P 7cc87046f1414b27928722690a5e388f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:36.980391  3440 consensus_peers.cc:597] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:36.981117  1517 tablet_replica.cc:333] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.981895  1517 raft_consensus.cc:2243] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.982941  1517 raft_consensus.cc:2272] T 1f0df81094e2493cbe804d2dba9aa9d4 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.985515  1517 tablet_replica.cc:333] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.986240  1517 raft_consensus.cc:2243] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.987205  1517 raft_consensus.cc:2272] T 0a1a771f5b2742d8bd641ec0afec2cf4 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.989807  1517 tablet_replica.cc:333] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.990483  1517 raft_consensus.cc:2243] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:36.991613  1517 raft_consensus.cc:2272] T 15476716416b4e729148408ed7846297 P 7cc87046f1414b27928722690a5e388f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:36.994503  1517 tablet_replica.cc:333] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f: stopping tablet replica
I20260709 12:22:36.995177  1517 raft_consensus.cc:2243] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:36.996064  1517 raft_consensus.cc:2272] T 51910d09518b4757be25a91c35ee2ec2 P 7cc87046f1414b27928722690a5e388f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:37.059950  3440 consensus_peers.cc:597] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:37.114311  3440 consensus_peers.cc:597] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:22:37.197335  3440 consensus_peers.cc:597] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:37.205160  4841 raft_consensus.cc:493] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:37.206930  4841 raft_consensus.cc:515] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:37.209316  4841 leader_election.cc:290] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:37.213083  3440 consensus_peers.cc:597] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:37.216509  3440 leader_election.cc:336] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
W20260709 12:22:37.217674  3439 leader_election.cc:336] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:37.218180  3439 leader_election.cc:304] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:37.219517  4841 raft_consensus.cc:2749] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:37.220149  4850 raft_consensus.cc:493] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:37.220736  4850 raft_consensus.cc:515] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:37.223246  4850 leader_election.cc:290] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:37.226464  3440 leader_election.cc:336] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:37.227576  3439 leader_election.cc:336] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:37.227953  3439 leader_election.cc:304] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:37.228557  4850 raft_consensus.cc:2749] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:37.245826  4850 raft_consensus.cc:493] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:37.246439  4850 raft_consensus.cc:515] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
W20260709 12:22:37.247762  3440 consensus_peers.cc:597] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:22:37.248597  4850 leader_election.cc:290] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:37.252640  3439 leader_election.cc:336] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:37.253331  3440 leader_election.cc:336] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:37.253734  3440 leader_election.cc:304] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:37.254410  4850 raft_consensus.cc:2749] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:37.279024  3440 consensus_peers.cc:597] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:22:37.296075  4850 raft_consensus.cc:493] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:37.296669  4850 raft_consensus.cc:515] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:37.298825  4841 raft_consensus.cc:493] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:37.299448  4841 raft_consensus.cc:515] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:37.301506  4841 leader_election.cc:290] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:37.298794  4850 leader_election.cc:290] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:37.302156  4859 raft_consensus.cc:493] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:37.302410  4850 raft_consensus.cc:493] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:37.302788  4859 raft_consensus.cc:515] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:37.303104  4850 raft_consensus.cc:515] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:37.305115  4859 leader_election.cc:290] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:37.305738  4850 leader_election.cc:290] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:37.306380  4850 raft_consensus.cc:493] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:37.306831  4850 raft_consensus.cc:515] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
W20260709 12:22:37.307873  3439 leader_election.cc:336] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:37.309041  4850 leader_election.cc:290] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:37.311352  3439 leader_election.cc:336] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:37.311910  3439 leader_election.cc:336] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:37.312381  3439 leader_election.cc:336] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:37.315425  3440 leader_election.cc:336] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:37.315864  3440 leader_election.cc:304] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
W20260709 12:22:37.316927  3439 leader_election.cc:336] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:37.317785  4860 raft_consensus.cc:2749] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:37.317842  3440 leader_election.cc:336] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:37.318625  3440 leader_election.cc:304] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:37.319325  4860 raft_consensus.cc:2749] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:37.322935  3440 consensus_peers.cc:597] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:37.323436  3440 leader_election.cc:336] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:37.323794  3440 leader_election.cc:304] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:37.324457  4860 raft_consensus.cc:2749] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:37.325976  3440 leader_election.cc:336] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:37.326499  3440 leader_election.cc:304] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:37.327253  4850 raft_consensus.cc:2749] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:37.327454  3440 leader_election.cc:336] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:37.328073  3440 leader_election.cc:304] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:37.328799  4850 raft_consensus.cc:2749] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:37.494577  3440 consensus_peers.cc:597] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:22:37.511157  3440 consensus_peers.cc:597] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:22:37.528721  4850 raft_consensus.cc:493] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:37.529464  4850 raft_consensus.cc:515] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:37.532025  4850 leader_election.cc:290] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:37.543649  3439 leader_election.cc:336] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:37.547739  3440 consensus_peers.cc:597] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:22:37.548358  3440 leader_election.cc:336] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:37.548787  3440 leader_election.cc:304] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:37.549634  4850 raft_consensus.cc:2749] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:37.572741  4850 raft_consensus.cc:493] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:37.573377  4850 raft_consensus.cc:515] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:37.577643  4850 leader_election.cc:290] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:37.585155  3440 leader_election.cc:336] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
W20260709 12:22:37.587967  3439 leader_election.cc:336] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:37.590014  3439 leader_election.cc:304] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:37.591722  4850 raft_consensus.cc:2749] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:37.620491  4850 raft_consensus.cc:493] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:37.621168  4850 raft_consensus.cc:515] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:37.625373  4850 leader_election.cc:290] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:37.630714  3439 leader_election.cc:336] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:37.634342  3440 leader_election.cc:336] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:37.634857  3440 leader_election.cc:304] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:37.635648  4850 raft_consensus.cc:2749] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:37.690168  3440 consensus_peers.cc:597] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:22:37.756850  2651 catalog_manager.cc:4755] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c9f24d490dc840f1b2d87604abe412d8: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:37.918512  4850 raft_consensus.cc:493] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:37.919796  4850 raft_consensus.cc:515] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:37.923200  4850 leader_election.cc:290] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:37.939826  3440 leader_election.cc:336] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:37.941251  3440 leader_election.cc:336] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:37.941727  3440 leader_election.cc:304] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:37.942454  4850 raft_consensus.cc:2749] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:37.988656  4850 raft_consensus.cc:493] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:37.989679  4850 raft_consensus.cc:515] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:37.992365  4850 leader_election.cc:290] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:37.996376  3440 leader_election.cc:336] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:37.998647  3440 leader_election.cc:336] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:37.999120  3440 leader_election.cc:304] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:37.999993  4860 raft_consensus.cc:2749] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:38.042002  4860 raft_consensus.cc:493] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.042863  4860 raft_consensus.cc:515] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.044979  4860 leader_election.cc:290] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.049147  3439 leader_election.cc:336] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:38.049715  3440 leader_election.cc:336] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.050124  3440 leader_election.cc:304] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:38.050743  4860 raft_consensus.cc:2749] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:38.100596  4860 raft_consensus.cc:493] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.101498  4860 raft_consensus.cc:515] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.104176  4860 leader_election.cc:290] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:38.105067  4850 raft_consensus.cc:493] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.107777  4850 raft_consensus.cc:515] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.110277  4850 leader_election.cc:290] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.112398  3439 leader_election.cc:336] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:38.116504  1517 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 2938)
W20260709 12:22:38.119306  3440 leader_election.cc:336] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.119845  3440 leader_election.cc:304] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:38.120677  4850 raft_consensus.cc:2749] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:38.120853  3440 leader_election.cc:336] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.124336  4860 raft_consensus.cc:493] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.125116  4860 raft_consensus.cc:515] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.127758  4850 raft_consensus.cc:493] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.128669  4850 raft_consensus.cc:515] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.129702  4860 leader_election.cc:290] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.130903  3440 leader_election.cc:336] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:38.131386  3440 leader_election.cc:304] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:38.132161  4860 raft_consensus.cc:2749] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:38.135152  3439 leader_election.cc:336] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:38.136016  4850 leader_election.cc:290] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.143209  3440 leader_election.cc:336] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:38.144447  3440 leader_election.cc:336] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.144835  3440 leader_election.cc:304] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:38.145673  4850 raft_consensus.cc:2749] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:38.146471  3440 leader_election.cc:336] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.147001  3440 leader_election.cc:304] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:38.147737  4850 raft_consensus.cc:2749] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:38.152956  4850 raft_consensus.cc:493] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.153635  4850 raft_consensus.cc:515] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.155473  4850 leader_election.cc:290] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.159737  3440 leader_election.cc:336] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:38.160507  3440 leader_election.cc:336] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.160861  3440 leader_election.cc:304] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:38.161486  4850 raft_consensus.cc:2749] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:38.197899  4850 raft_consensus.cc:493] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.198876  4866 raft_consensus.cc:493] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.198652  4850 raft_consensus.cc:515] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.199693  4866 raft_consensus.cc:515] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.201272  4850 leader_election.cc:290] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:38.202028  4866 leader_election.cc:290] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.205271  3439 leader_election.cc:336] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:38.205957  3439 leader_election.cc:336] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:38.207415  3440 leader_election.cc:336] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.207758  3440 leader_election.cc:304] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:38.208365  4866 raft_consensus.cc:2749] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:38.210814  3440 leader_election.cc:336] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.211171  3440 leader_election.cc:304] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:38.211771  4866 raft_consensus.cc:2749] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:38.223973  4866 raft_consensus.cc:493] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.224751  4866 raft_consensus.cc:515] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.227494  4866 leader_election.cc:290] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.232146  3440 leader_election.cc:336] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:38.232970  3440 leader_election.cc:336] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.233349  3440 leader_election.cc:304] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:38.234097  4866 raft_consensus.cc:2749] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:38.237005  4866 raft_consensus.cc:493] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.237736  4866 raft_consensus.cc:515] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.240983  4866 leader_election.cc:290] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.245079  3439 leader_election.cc:336] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:38.246843  3440 leader_election.cc:336] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.247246  3440 leader_election.cc:304] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:38.248026  4866 raft_consensus.cc:2749] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:38.255076  3439 consensus_peers.cc:597] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:38.278666  4866 raft_consensus.cc:493] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.279454  4866 raft_consensus.cc:515] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
W20260709 12:22:38.280171  3439 consensus_peers.cc:597] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:38.281847  4866 leader_election.cc:290] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.285529  3440 leader_election.cc:336] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
W20260709 12:22:38.286904  3439 leader_election.cc:336] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:38.287871  3439 leader_election.cc:304] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:38.288679  4850 raft_consensus.cc:2749] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:38.292346  4850 raft_consensus.cc:493] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.293150  4850 raft_consensus.cc:515] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.295686  4850 leader_election.cc:290] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.299364  3440 leader_election.cc:336] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:38.300343  3440 leader_election.cc:336] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.300666  3440 leader_election.cc:304] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:38.301246  4850 raft_consensus.cc:2749] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:38.307605  4850 raft_consensus.cc:493] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:38.308211  4850 raft_consensus.cc:515] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.310189  4850 leader_election.cc:290] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.314332  3440 leader_election.cc:336] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
W20260709 12:22:38.314802  3439 leader_election.cc:336] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:38.315280  3439 leader_election.cc:304] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:38.316046  4850 raft_consensus.cc:2749] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:38.345232  3439 consensus_peers.cc:597] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:38.374471  3439 consensus_peers.cc:597] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:38.428598  3439 consensus_peers.cc:597] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:38.514659  3439 consensus_peers.cc:597] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:38.598944  3439 consensus_peers.cc:597] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:38.607029  3439 consensus_peers.cc:597] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:38.705766  3439 consensus_peers.cc:597] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:38.791937  3440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111) [suppressed 490 similar messages]
W20260709 12:22:38.820039  3440 consensus_peers.cc:597] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:22:38.862258  3439 consensus_peers.cc:597] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:38.961802  4867 raft_consensus.cc:493] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:38.962673  4867 raft_consensus.cc:515] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:38.965327  4867 leader_election.cc:290] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:38.969173  3439 leader_election.cc:336] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:38.970010  3440 leader_election.cc:336] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:38.970456  3440 leader_election.cc:304] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:38.971246  4867 raft_consensus.cc:2749] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:39.050402  3440 consensus_peers.cc:597] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:39.097172  4850 raft_consensus.cc:493] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:39.097874  4850 raft_consensus.cc:515] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.100095  4850 leader_election.cc:290] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:39.102828  4850 raft_consensus.cc:493] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:39.103729  4850 raft_consensus.cc:515] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
W20260709 12:22:39.104588  3440 leader_election.cc:336] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:39.105346  3439 leader_election.cc:336] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:39.105934  4850 leader_election.cc:290] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
I20260709 12:22:39.105865  3439 leader_election.cc:304] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:39.106863  4850 raft_consensus.cc:2749] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:39.109807  3440 leader_election.cc:336] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:39.111593  3439 leader_election.cc:336] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:39.111961  3439 leader_election.cc:304] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:39.112691  4850 raft_consensus.cc:2749] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:39.117127  1517 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 2938)
W20260709 12:22:39.138355  3440 consensus_peers.cc:597] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:39.139163  3440 consensus_peers.cc:597] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:22:39.193001  3440 consensus_peers.cc:597] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:22:39.193629  3440 consensus_peers.cc:597] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:22:39.232705  3440 consensus_peers.cc:597] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:39.325752  3440 consensus_peers.cc:597] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:39.363919  3440 consensus_peers.cc:597] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:39.366451  3440 consensus_peers.cc:597] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:22:39.398079  3440 consensus_peers.cc:597] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:22:39.448419  4867 raft_consensus.cc:493] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:39.449335  4867 raft_consensus.cc:515] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.451921  4867 leader_election.cc:290] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:39.455804  3440 leader_election.cc:336] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:39.456413  3439 leader_election.cc:336] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:39.456868  3439 leader_election.cc:304] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:39.457711  4867 raft_consensus.cc:2749] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:39.463699  4867 raft_consensus.cc:493] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:39.464252  4867 raft_consensus.cc:515] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.466378  4867 leader_election.cc:290] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:39.469946  3440 leader_election.cc:336] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:39.470944  3439 leader_election.cc:336] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:39.471325  3439 leader_election.cc:304] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:39.471935  4867 raft_consensus.cc:2749] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:39.488058  4867 raft_consensus.cc:493] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:39.488708  4867 raft_consensus.cc:515] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.490868  4867 leader_election.cc:290] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:39.495227  3440 leader_election.cc:336] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:39.496006  3440 consensus_peers.cc:597] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:39.496819  3439 leader_election.cc:336] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:39.497397  3439 leader_election.cc:304] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:39.498239  4867 raft_consensus.cc:2749] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:39.523888  4867 raft_consensus.cc:493] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:39.524597  4867 raft_consensus.cc:515] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.526841  4867 leader_election.cc:290] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:39.530525  3439 leader_election.cc:336] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:39.532038  3440 leader_election.cc:336] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:39.532527  3440 leader_election.cc:304] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:39.533417  4867 raft_consensus.cc:2749] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:39.538211  3440 consensus_peers.cc:597] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:22:39.539139  4867 raft_consensus.cc:493] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:39.539727  4867 raft_consensus.cc:515] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.541777  4867 leader_election.cc:290] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:39.546496  3439 leader_election.cc:336] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:39.547161  3440 leader_election.cc:336] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:39.547518  3440 leader_election.cc:304] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:39.548131  4867 raft_consensus.cc:2749] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:39.549576  3440 consensus_peers.cc:597] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:22:39.564195  3440 consensus_peers.cc:597] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:22:39.565799  4850 raft_consensus.cc:493] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:39.566463  4850 raft_consensus.cc:515] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.568516  4850 leader_election.cc:290] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:39.572763  3440 leader_election.cc:336] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:39.574509  3439 leader_election.cc:336] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:39.575088  3439 leader_election.cc:304] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
W20260709 12:22:39.575598  3440 consensus_peers.cc:597] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:22:39.575906  4850 raft_consensus.cc:2749] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:39.610860  4850 raft_consensus.cc:493] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:39.611641  4850 raft_consensus.cc:515] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.613727  4850 leader_election.cc:290] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:39.618840  3440 leader_election.cc:336] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:39.619336  3439 leader_election.cc:336] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:39.619849  3439 leader_election.cc:304] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:39.621268  4850 raft_consensus.cc:2749] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:39.660856  3440 consensus_peers.cc:597] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:39.662874  4867 raft_consensus.cc:493] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:39.663519  4867 raft_consensus.cc:515] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.665660  4867 leader_election.cc:290] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:39.670077  3440 leader_election.cc:336] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:39.670712  3440 leader_election.cc:336] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:39.671059  3440 leader_election.cc:304] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:39.671700  4867 raft_consensus.cc:2749] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:39.674562  3440 consensus_peers.cc:597] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:39.681723  3440 consensus_peers.cc:597] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:39.691474  3440 consensus_peers.cc:597] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:39.738279  3440 consensus_peers.cc:597] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:39.839802  4850 raft_consensus.cc:493] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:39.840685  4850 raft_consensus.cc:515] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.843019  4850 leader_election.cc:290] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:39.846887  3439 leader_election.cc:336] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:39.847540  3440 leader_election.cc:336] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:39.847899  3440 leader_election.cc:304] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:39.848528  4850 raft_consensus.cc:2749] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:39.894518  3440 consensus_peers.cc:597] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:22:39.926452  4850 raft_consensus.cc:493] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:39.927150  4850 raft_consensus.cc:515] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.928547  4867 raft_consensus.cc:493] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:39.929046  4850 leader_election.cc:290] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:39.929186  4867 raft_consensus.cc:515] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.931687  4867 leader_election.cc:290] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:39.933753  3440 leader_election.cc:336] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
W20260709 12:22:39.934823  3439 leader_election.cc:336] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:39.935272  3439 leader_election.cc:304] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:39.936021  4867 raft_consensus.cc:2749] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:39.937472  3440 leader_election.cc:336] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
W20260709 12:22:39.938041  3440 leader_election.cc:336] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:39.938347  3440 leader_election.cc:304] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:39.938946  4867 raft_consensus.cc:2749] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:39.948477  3440 consensus_peers.cc:597] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 -> Peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Couldn't send request to peer 7cc87046f1414b27928722690a5e388f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:39.965613  4850 raft_consensus.cc:493] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:39.966326  4850 raft_consensus.cc:515] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:39.968575  4850 leader_election.cc:290] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:39.972822  3440 leader_election.cc:336] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:39.974123  3440 leader_election.cc:336] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:39.974579  3440 leader_election.cc:304] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:39.975258  4850 raft_consensus.cc:2749] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:40.001039  4850 raft_consensus.cc:493] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:40.001642  4850 raft_consensus.cc:515] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:40.003746  4850 leader_election.cc:290] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:40.007642  3440 leader_election.cc:336] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
W20260709 12:22:40.008263  3439 leader_election.cc:336] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
I20260709 12:22:40.008741  3439 leader_election.cc:304] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:40.009590  4850 raft_consensus.cc:2749] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:40.023808  4867 raft_consensus.cc:493] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:40.024493  4867 raft_consensus.cc:515] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:40.026723  4867 leader_election.cc:290] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:40.031394  3440 leader_election.cc:336] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:40.032145  3440 leader_election.cc:336] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:40.032553  3440 leader_election.cc:304] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:40.033349  4867 raft_consensus.cc:2749] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:40.041399  4867 raft_consensus.cc:493] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:40.042040  4867 raft_consensus.cc:515] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
W20260709 12:22:40.043728  3440 consensus_peers.cc:597] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:22:40.044252  4867 leader_election.cc:290] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:40.052729  3440 leader_election.cc:336] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:40.053473  3440 leader_election.cc:336] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:40.053942  3440 leader_election.cc:304] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:40.054641  4867 raft_consensus.cc:2749] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:40.059224  4867 raft_consensus.cc:493] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:40.059899  4867 raft_consensus.cc:515] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:40.062522  4867 leader_election.cc:290] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:40.064347  4850 raft_consensus.cc:493] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:40.064868  4850 raft_consensus.cc:515] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:40.066949  4850 leader_election.cc:290] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:40.067147  3440 consensus_peers.cc:597] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:22:40.067728  3440 leader_election.cc:336] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:40.070500  3440 leader_election.cc:336] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:40.070936  3440 leader_election.cc:304] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:40.071686  4850 raft_consensus.cc:2749] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:40.074537  3440 leader_election.cc:336] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
W20260709 12:22:40.075428  3440 leader_election.cc:336] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:40.075815  3440 leader_election.cc:304] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 7cc87046f1414b27928722690a5e388f
I20260709 12:22:40.076537  4850 raft_consensus.cc:2749] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:40.118017  1517 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 2938)
I20260709 12:22:40.132295  4850 raft_consensus.cc:493] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:40.133047  4850 raft_consensus.cc:515] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:40.135085  4850 leader_election.cc:290] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:40.139957  3439 leader_election.cc:336] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:40.141441  3440 leader_election.cc:336] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:40.142004  3440 leader_election.cc:304] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:40.142840  4867 raft_consensus.cc:2749] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:40.215735  3440 consensus_peers.cc:597] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 -> Peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Couldn't send request to peer 442c92e5267046cd92f46e6f469570a5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:22:40.235663  4867 raft_consensus.cc:493] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:40.236415  4867 raft_consensus.cc:515] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:40.240437  4867 leader_election.cc:290] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:40.242738  3439 leader_election.cc:336] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:40.245572  3440 leader_election.cc:336] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:40.245977  3440 leader_election.cc:304] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:40.246613  4867 raft_consensus.cc:2749] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:40.274086  4867 raft_consensus.cc:493] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:40.274694  4867 raft_consensus.cc:515] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:40.276955  4867 leader_election.cc:290] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:40.280984  3439 leader_election.cc:336] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:40.283407  3440 leader_election.cc:336] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:40.283835  3440 leader_election.cc:304] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:40.284538  4867 raft_consensus.cc:2749] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:40.331776  4867 raft_consensus.cc:493] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:40.332377  4867 raft_consensus.cc:515] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:40.334425  4867 leader_election.cc:290] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:40.336611  4867 raft_consensus.cc:493] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:40.337230  4867 raft_consensus.cc:515] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
W20260709 12:22:40.339092  3439 leader_election.cc:336] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:40.339641  3440 leader_election.cc:336] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:40.339790  4867 leader_election.cc:290] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:40.340157  3440 leader_election.cc:304] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:40.341027  4867 raft_consensus.cc:2749] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:40.343895  3439 leader_election.cc:336] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:40.345733  3440 leader_election.cc:336] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:40.346088  3440 leader_election.cc:304] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:40.346750  4867 raft_consensus.cc:2749] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:40.528811  4850 raft_consensus.cc:493] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7cc87046f1414b27928722690a5e388f)
I20260709 12:22:40.529476  4850 raft_consensus.cc:515] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:40.531625  4850 leader_election.cc:290] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
W20260709 12:22:40.535173  3439 leader_election.cc:336] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:40.537753  3440 leader_election.cc:336] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:40.538164  3440 leader_election.cc:304] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:40.538801  4850 raft_consensus.cc:2749] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:40.609982  3439 consensus_peers.cc:597] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:40.648962  1517 tablet_server.cc:196] TabletServer@127.1.123.67:0 shutdown complete.
W20260709 12:22:40.672809  3439 consensus_peers.cc:597] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:40.705523  3439 consensus_peers.cc:597] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:40.725649  3439 consensus_peers.cc:597] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:40.746387  4867 raft_consensus.cc:493] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:40.747011  4867 raft_consensus.cc:515] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "442c92e5267046cd92f46e6f469570a5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 35625 } } peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:40.749275  4867 leader_election.cc:290] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625), 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215)
W20260709 12:22:40.754179  3439 leader_election.cc:336] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:40.755293  3440 leader_election.cc:336] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:40.755803  3440 leader_election.cc:304] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 442c92e5267046cd92f46e6f469570a5, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:40.756693  4850 raft_consensus.cc:2749] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:40.799075  3439 consensus_peers.cc:597] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 -> Peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Couldn't send request to peer 9a10672cd53140d59ff945f75345cc06. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:40.802733  1517 tablet_server.cc:179] TabletServer@127.1.123.68:0 shutting down...
I20260709 12:22:40.846946  4850 raft_consensus.cc:493] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a10672cd53140d59ff945f75345cc06)
I20260709 12:22:40.847620  4850 raft_consensus.cc:515] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9a10672cd53140d59ff945f75345cc06" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 44215 } } peers { permanent_uuid: "7cc87046f1414b27928722690a5e388f" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 37395 } } peers { permanent_uuid: "f440bbb3ddd545819242017140636ad4" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 35397 } attrs { promote: false } }
I20260709 12:22:40.848506  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:40.849642  4850 leader_election.cc:290] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215), 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395)
I20260709 12:22:40.849710  1517 tablet_replica.cc:333] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.851559  1517 raft_consensus.cc:2243] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.852959  1517 raft_consensus.cc:2272] T 0e4b953250214a6f8f3b4794fab90ce0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:40.854547  3439 leader_election.cc:336] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9a10672cd53140d59ff945f75345cc06 (127.1.123.66:44215): Network error: Client connection negotiation failed: client connection to 127.1.123.66:44215: connect: Connection refused (error 111)
W20260709 12:22:40.855127  3440 leader_election.cc:336] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cc87046f1414b27928722690a5e388f (127.1.123.67:37395): Network error: Client connection negotiation failed: client connection to 127.1.123.67:37395: connect: Connection refused (error 111)
I20260709 12:22:40.855604  3440 leader_election.cc:304] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f440bbb3ddd545819242017140636ad4; no voters: 7cc87046f1414b27928722690a5e388f, 9a10672cd53140d59ff945f75345cc06
I20260709 12:22:40.856310  4850 raft_consensus.cc:2749] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:40.856751  1517 tablet_replica.cc:333] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.857739  1517 raft_consensus.cc:2243] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.858373  1517 raft_consensus.cc:2272] T 34453a67223842ed98d4220533186ade P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.860854  1517 tablet_replica.cc:333] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.861527  1517 raft_consensus.cc:2243] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.862607  1517 raft_consensus.cc:2272] T 42af6798b62746449070b7b2bc4e0699 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.864872  1517 tablet_replica.cc:333] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.865736  1517 raft_consensus.cc:2243] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.866541  1517 raft_consensus.cc:2272] T 6291f8f287034586b4e9c0c9220bc6e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.868700  1517 tablet_replica.cc:333] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.869293  1517 raft_consensus.cc:2243] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.869925  1517 raft_consensus.cc:2272] T eface8ee310347d4bb08e3c5864ce5d9 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.872459  1517 tablet_replica.cc:333] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.872985  1517 raft_consensus.cc:2243] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.873787  1517 raft_consensus.cc:2272] T 386a0166b36244f789cc8361a1cb0a46 P f440bbb3ddd545819242017140636ad4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.876103  1517 tablet_replica.cc:333] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.876861  1517 raft_consensus.cc:2243] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.877558  1517 raft_consensus.cc:2272] T e80bb6a87c6e403fa76954737ffda91a P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.879765  1517 tablet_replica.cc:333] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.880430  1517 raft_consensus.cc:2243] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.882022  1517 raft_consensus.cc:2272] T a66a027320bb4d04bb8735cd6fbffabf P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.885354  1517 tablet_replica.cc:333] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.886128  1517 raft_consensus.cc:2243] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 6 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.887043  1517 raft_consensus.cc:2272] T 22aad5981ec94ff2a74fdf45111c0692 P f440bbb3ddd545819242017140636ad4 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.889072  1517 tablet_replica.cc:333] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.890225  1517 raft_consensus.cc:2243] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.891150  1517 raft_consensus.cc:2272] T 659c2c3a61024d07a2296bc83d87a52d P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.893851  1517 tablet_replica.cc:333] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.894526  1517 raft_consensus.cc:2243] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.895139  1517 raft_consensus.cc:2272] T fe7ee185c0114c079f53469226dee8bf P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.897374  1517 tablet_replica.cc:333] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.898108  1517 raft_consensus.cc:2243] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.898820  1517 raft_consensus.cc:2272] T 51910d09518b4757be25a91c35ee2ec2 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.900940  1517 tablet_replica.cc:333] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.901700  1517 raft_consensus.cc:2243] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 6 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.902936  1517 raft_consensus.cc:2272] T 62274aac902146f7b127a2afb55b8a12 P f440bbb3ddd545819242017140636ad4 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.905416  1517 tablet_replica.cc:333] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.906100  1517 raft_consensus.cc:2243] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.906791  1517 raft_consensus.cc:2272] T 627b3359300e44fc9d2b83ed88272a9f P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.909621  1517 tablet_replica.cc:333] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.910318  1517 raft_consensus.cc:2243] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.911322  1517 raft_consensus.cc:2272] T 2594ea75a8d44503a64ea4e06916c4b3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.914079  1517 tablet_replica.cc:333] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.914644  1517 raft_consensus.cc:2243] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.915210  1517 raft_consensus.cc:2272] T 2a6389f65afb4adabe1d70b6f09e99e7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.917292  1517 tablet_replica.cc:333] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.917948  1517 raft_consensus.cc:2243] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.918470  1517 raft_consensus.cc:2272] T f26ca468f5384641ad5bd05af362aef9 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.920537  1517 tablet_replica.cc:333] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.921115  1517 raft_consensus.cc:2243] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.921684  1517 raft_consensus.cc:2272] T 1f0df81094e2493cbe804d2dba9aa9d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.924304  1517 tablet_replica.cc:333] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.925120  1517 raft_consensus.cc:2243] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.925910  1517 raft_consensus.cc:2272] T 0a1a771f5b2742d8bd641ec0afec2cf4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.928172  1517 tablet_replica.cc:333] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.928795  1517 raft_consensus.cc:2243] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.929476  1517 raft_consensus.cc:2272] T 2cba50a5280d4b00b9d2617ce7a22ec1 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.931475  1517 tablet_replica.cc:333] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.932080  1517 raft_consensus.cc:2243] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.932686  1517 raft_consensus.cc:2272] T 63fa205fa4864ef4b19bda073c06a5d4 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.934758  1517 tablet_replica.cc:333] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.935382  1517 raft_consensus.cc:2243] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.936273  1517 raft_consensus.cc:2272] T 133424caefd34fa8b83015c9fe06d612 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.938815  1517 tablet_replica.cc:333] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.939420  1517 raft_consensus.cc:2243] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.939978  1517 raft_consensus.cc:2272] T 710edfdfdd7e4720b77f4ec27f5f41c0 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.942451  1517 tablet_replica.cc:333] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.943086  1517 raft_consensus.cc:2243] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.943658  1517 raft_consensus.cc:2272] T 01e8aac6ebd14bf4b952c62c8075f13f P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.946457  1517 tablet_replica.cc:333] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.947141  1517 raft_consensus.cc:2243] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.948071  1517 raft_consensus.cc:2272] T dc860efbfaf84207b7128fd001b59fe6 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.950608  1517 tablet_replica.cc:333] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.951270  1517 raft_consensus.cc:2243] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.952129  1517 raft_consensus.cc:2272] T 7e4cde589c024472b0a4cec236ea69a3 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.954576  1517 tablet_replica.cc:333] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.955173  1517 raft_consensus.cc:2243] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.955828  1517 raft_consensus.cc:2272] T 6855fa72e5f9486bb478855f721e77ce P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.958081  1517 tablet_replica.cc:333] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.958683  1517 raft_consensus.cc:2243] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.959219  1517 raft_consensus.cc:2272] T c737810e34ba42deaa41e6478e84fe87 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.961330  1517 tablet_replica.cc:333] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.961957  1517 raft_consensus.cc:2243] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.962581  1517 raft_consensus.cc:2272] T d436a6cfd22640bea76c0256f1d01684 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.964689  1517 tablet_replica.cc:333] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.965286  1517 raft_consensus.cc:2243] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.966256  1517 raft_consensus.cc:2272] T a2f8a42191e14f07a43032df18499a9c P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.968300  1517 tablet_replica.cc:333] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.968859  1517 raft_consensus.cc:2243] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.969769  1517 raft_consensus.cc:2272] T e3545f99312f483eb6dc8965c5bc418b P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.971581  1517 tablet_replica.cc:333] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.972311  1517 raft_consensus.cc:2243] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.973071  1517 raft_consensus.cc:2272] T 53752656656243c3abe78c7a9cb488c7 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.975492  1517 tablet_replica.cc:333] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.976147  1517 raft_consensus.cc:2243] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.977089  1517 raft_consensus.cc:2272] T 3dbc0cbfcb67483f9f554976e140ba7f P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.979308  1517 tablet_replica.cc:333] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.979897  1517 raft_consensus.cc:2243] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.980832  1517 raft_consensus.cc:2272] T 9164313f06e045c3bdef4a46d365873c P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.983079  1517 tablet_replica.cc:333] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.983700  1517 raft_consensus.cc:2243] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.984743  1517 raft_consensus.cc:2272] T d500b7bb3bd042f09ffa7310e921aa24 P f440bbb3ddd545819242017140636ad4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.986852  1517 tablet_replica.cc:333] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.987461  1517 raft_consensus.cc:2243] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.988601  1517 raft_consensus.cc:2272] T a7c3a03a1304452ebc48e986f5629037 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.990796  1517 tablet_replica.cc:333] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.991379  1517 raft_consensus.cc:2243] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:40.992298  1517 raft_consensus.cc:2272] T b46ad496aaf14cbc8cf01697dceb4cf5 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.994419  1517 tablet_replica.cc:333] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.995010  1517 raft_consensus.cc:2243] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.995581  1517 raft_consensus.cc:2272] T 12034fdcbbed4121a134d5d26c1dc5ac P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:40.998009  1517 tablet_replica.cc:333] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:40.998567  1517 raft_consensus.cc:2243] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:40.999179  1517 raft_consensus.cc:2272] T bb1bafa30972440499c69fb1221eae27 P f440bbb3ddd545819242017140636ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:41.001268  1517 tablet_replica.cc:333] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:41.001956  1517 raft_consensus.cc:2243] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:41.002672  1517 raft_consensus.cc:2272] T c9f24d490dc840f1b2d87604abe412d8 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:41.004949  1517 tablet_replica.cc:333] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:41.005507  1517 raft_consensus.cc:2243] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:41.006449  1517 raft_consensus.cc:2272] T 7bb611b2490b4682a0c5a45f2bbe7c94 P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:41.008519  1517 tablet_replica.cc:333] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:41.009115  1517 raft_consensus.cc:2243] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:41.010106  1517 raft_consensus.cc:2272] T 532f0b78131648af8fd4930674a8bce0 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:41.012254  1517 tablet_replica.cc:333] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:41.012761  1517 raft_consensus.cc:2243] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:41.013664  1517 raft_consensus.cc:2272] T 547d5e0cde57459bbad31d06eb64d442 P f440bbb3ddd545819242017140636ad4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:41.015758  1517 tablet_replica.cc:333] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4: stopping tablet replica
I20260709 12:22:41.016238  1517 raft_consensus.cc:2243] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:41.017097  1517 raft_consensus.cc:2272] T 4ff32c54020945d99bd67b7df9574bae P f440bbb3ddd545819242017140636ad4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:41.803699  1517 tablet_server.cc:196] TabletServer@127.1.123.68:0 shutdown complete.
I20260709 12:22:41.878041  1517 master.cc:562] Master@127.1.123.126:36731 shutting down...
W20260709 12:22:41.899621  2651 catalog_manager.cc:4755] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c9f24d490dc840f1b2d87604abe412d8: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
W20260709 12:22:45.594002  2697 debug-util.cc:398] Leaking SignalData structure 0x7b080021fa00 after lost signal to thread 1520
W20260709 12:22:45.594702  2697 debug-util.cc:398] Leaking SignalData structure 0x7b08002ed3a0 after lost signal to thread 2700
W20260709 12:22:50.100392  2651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111) [suppressed 213 similar messages]
W20260709 12:22:50.105881  2651 catalog_manager.cc:4755] TS 442c92e5267046cd92f46e6f469570a5 (127.1.123.65:35625): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c9f24d490dc840f1b2d87604abe412d8: Network error: Client connection negotiation failed: client connection to 127.1.123.65:35625: connect: Connection refused (error 111)
I20260709 12:22:50.659969  1517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:50.661602  1517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:50.662135  1517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 54239aeb1f214e00bccab640927a1e8f: stopping tablet replica
I20260709 12:22:50.793668  1517 master.cc:584] Master@127.1.123.126:36731 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (83053 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:22:50.918056  1517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.123.126:38285
I20260709 12:22:50.919454  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:50.926321  4874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:50.926317  4875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:50.927206  4877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:50.929674  1517 server_base.cc:1061] running on GCE node
I20260709 12:22:50.931934  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:50.932133  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:50.932345  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599770932331 us; error 0 us; skew 500 ppm
I20260709 12:22:50.932927  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:50.935817  1517 webserver.cc:533] Webserver started at http://127.1.123.126:44503/ using document root <none> and password file <none>
I20260709 12:22:50.936362  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:50.936565  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:50.936872  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:50.938273  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/master-0-root/instance:
uuid: "ce6fc7638c8549faa2535fb5774c32c1"
format_stamp: "Formatted at 2026-07-09 12:22:50 on dist-test-slave-08l4"
I20260709 12:22:50.943177  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:50.946986  4882 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:50.947896  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:22:50.948309  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/master-0-root
uuid: "ce6fc7638c8549faa2535fb5774c32c1"
format_stamp: "Formatted at 2026-07-09 12:22:50 on dist-test-slave-08l4"
I20260709 12:22:50.948580  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:50.967996  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:50.969345  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:51.013871  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.126:38285
I20260709 12:22:51.014070  4933 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.126:38285 every 8 connection(s)
I20260709 12:22:51.018471  4934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.032346  4934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: Bootstrap starting.
I20260709 12:22:51.037990  4934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:51.043231  4934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: No bootstrap required, opened a new log
I20260709 12:22:51.045878  4934 raft_consensus.cc:359] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6fc7638c8549faa2535fb5774c32c1" member_type: VOTER }
I20260709 12:22:51.046372  4934 raft_consensus.cc:385] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:51.046569  4934 raft_consensus.cc:740] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce6fc7638c8549faa2535fb5774c32c1, State: Initialized, Role: FOLLOWER
I20260709 12:22:51.047098  4934 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6fc7638c8549faa2535fb5774c32c1" member_type: VOTER }
I20260709 12:22:51.047574  4934 raft_consensus.cc:399] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:51.047786  4934 raft_consensus.cc:493] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:51.048062  4934 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:51.053351  4934 raft_consensus.cc:515] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6fc7638c8549faa2535fb5774c32c1" member_type: VOTER }
I20260709 12:22:51.054029  4934 leader_election.cc:304] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce6fc7638c8549faa2535fb5774c32c1; no voters: 
I20260709 12:22:51.055296  4934 leader_election.cc:290] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:51.055840  4937 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:51.057461  4937 raft_consensus.cc:697] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [term 1 LEADER]: Becoming Leader. State: Replica: ce6fc7638c8549faa2535fb5774c32c1, State: Running, Role: LEADER
I20260709 12:22:51.058313  4937 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6fc7638c8549faa2535fb5774c32c1" member_type: VOTER }
I20260709 12:22:51.058769  4934 sys_catalog.cc:565] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:51.070396  4938 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce6fc7638c8549faa2535fb5774c32c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6fc7638c8549faa2535fb5774c32c1" member_type: VOTER } }
I20260709 12:22:51.070396  4939 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce6fc7638c8549faa2535fb5774c32c1. Latest consensus state: current_term: 1 leader_uuid: "ce6fc7638c8549faa2535fb5774c32c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6fc7638c8549faa2535fb5774c32c1" member_type: VOTER } }
I20260709 12:22:51.071333  4938 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:51.071352  4939 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:51.074095  4948 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:51.082623  4948 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:51.083928  1517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:51.093806  4948 catalog_manager.cc:1383] Generated new cluster ID: 04e1bc70a2514ede965bae7a9aeea38c
I20260709 12:22:51.094076  4948 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:51.111492  4948 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:51.112955  4948 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:51.135442  4948 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: Generated new TSK 0
I20260709 12:22:51.136154  4948 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:51.151243  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:51.159427  4955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:51.159792  4956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:51.161185  4958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:51.162782  1517 server_base.cc:1061] running on GCE node
I20260709 12:22:51.164624  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:51.164889  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:51.165087  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599771165069 us; error 0 us; skew 500 ppm
I20260709 12:22:51.165781  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:51.168526  1517 webserver.cc:533] Webserver started at http://127.1.123.65:36141/ using document root <none> and password file <none>
I20260709 12:22:51.169152  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:51.169384  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:51.169720  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:51.171206  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/instance:
uuid: "9847a7797ddc4df689a1ab0911af8785"
format_stamp: "Formatted at 2026-07-09 12:22:51 on dist-test-slave-08l4"
I20260709 12:22:51.177676  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.004s
I20260709 12:22:51.181998  4963 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.183142  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:22:51.183583  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-0-root
uuid: "9847a7797ddc4df689a1ab0911af8785"
format_stamp: "Formatted at 2026-07-09 12:22:51 on dist-test-slave-08l4"
I20260709 12:22:51.183887  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:51.203724  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:51.205494  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:51.208143  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:51.211854  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:51.212065  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.212387  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:51.212581  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.260936  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.65:37253
I20260709 12:22:51.261103  5025 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.65:37253 every 8 connection(s)
I20260709 12:22:51.266258  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:51.275002  5030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:51.275285  5031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:51.279434  1517 server_base.cc:1061] running on GCE node
W20260709 12:22:51.279821  5033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:51.281759  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:51.282130  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:51.282348  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599771282327 us; error 0 us; skew 500 ppm
I20260709 12:22:51.282939  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:51.283742  5026 heartbeater.cc:344] Connected to a master server at 127.1.123.126:38285
I20260709 12:22:51.284165  5026 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:51.285112  5026 heartbeater.cc:507] Master 127.1.123.126:38285 requested a full tablet report, sending...
I20260709 12:22:51.285907  1517 webserver.cc:533] Webserver started at http://127.1.123.66:45199/ using document root <none> and password file <none>
I20260709 12:22:51.286480  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:51.286691  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:51.286971  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:51.287632  4899 ts_manager.cc:194] Registered new tserver with Master: 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:51.288363  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/instance:
uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
format_stamp: "Formatted at 2026-07-09 12:22:51 on dist-test-slave-08l4"
I20260709 12:22:51.289865  4899 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43218
I20260709 12:22:51.293602  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 12:22:51.297446  5038 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.298418  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:51.298743  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-1-root
uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
format_stamp: "Formatted at 2026-07-09 12:22:51 on dist-test-slave-08l4"
I20260709 12:22:51.299057  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:51.321483  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:51.322824  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:51.324393  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:51.326946  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:51.327235  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.327564  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:51.327800  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.376189  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.66:33955
I20260709 12:22:51.376380  5100 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.66:33955 every 8 connection(s)
I20260709 12:22:51.381914  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:51.390806  5104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:51.392911  5105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:51.394603  1517 server_base.cc:1061] running on GCE node
W20260709 12:22:51.396009  5107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:51.397146  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:51.397441  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:51.397642  5101 heartbeater.cc:344] Connected to a master server at 127.1.123.126:38285
I20260709 12:22:51.397722  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599771397701 us; error 0 us; skew 500 ppm
I20260709 12:22:51.398204  5101 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:51.398753  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:51.399263  5101 heartbeater.cc:507] Master 127.1.123.126:38285 requested a full tablet report, sending...
I20260709 12:22:51.402180  4899 ts_manager.cc:194] Registered new tserver with Master: 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:51.402182  1517 webserver.cc:533] Webserver started at http://127.1.123.67:40557/ using document root <none> and password file <none>
I20260709 12:22:51.403223  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:51.403592  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:51.404070  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:51.404563  4899 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43222
I20260709 12:22:51.405660  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/instance:
uuid: "ceec344dd00f40c18ef58414ce147016"
format_stamp: "Formatted at 2026-07-09 12:22:51 on dist-test-slave-08l4"
I20260709 12:22:51.411379  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.003s
I20260709 12:22:51.415412  5112 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.416254  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:22:51.416600  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-2-root
uuid: "ceec344dd00f40c18ef58414ce147016"
format_stamp: "Formatted at 2026-07-09 12:22:51 on dist-test-slave-08l4"
I20260709 12:22:51.416924  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:51.427960  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:51.429232  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:51.430886  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:51.433241  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:51.433502  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.433813  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:51.434003  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.482131  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.67:46855
I20260709 12:22:51.482273  5174 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.67:46855 every 8 connection(s)
I20260709 12:22:51.487363  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:51.496166  5178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:51.497067  5179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:51.501077  1517 server_base.cc:1061] running on GCE node
I20260709 12:22:51.502256  5175 heartbeater.cc:344] Connected to a master server at 127.1.123.126:38285
I20260709 12:22:51.502743  5175 heartbeater.cc:461] Registering TS with master...
W20260709 12:22:51.502660  5181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:51.504231  5175 heartbeater.cc:507] Master 127.1.123.126:38285 requested a full tablet report, sending...
I20260709 12:22:51.504345  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:51.504648  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:51.504886  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599771504864 us; error 0 us; skew 500 ppm
I20260709 12:22:51.505896  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:51.507300  4899 ts_manager.cc:194] Registered new tserver with Master: ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:51.509199  1517 webserver.cc:533] Webserver started at http://127.1.123.68:34045/ using document root <none> and password file <none>
I20260709 12:22:51.509223  4899 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43226
I20260709 12:22:51.509995  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:51.510304  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:51.510684  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:51.512647  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-3-root/instance:
uuid: "90e5220a62ed42fe83c82d0302d18fac"
format_stamp: "Formatted at 2026-07-09 12:22:51 on dist-test-slave-08l4"
I20260709 12:22:51.517777  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.005s
I20260709 12:22:51.521891  5186 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.522850  1517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:22:51.523240  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-3-root
uuid: "90e5220a62ed42fe83c82d0302d18fac"
format_stamp: "Formatted at 2026-07-09 12:22:51 on dist-test-slave-08l4"
I20260709 12:22:51.523566  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:51.552239  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:51.553843  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:51.555392  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:51.558075  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:51.558342  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.558612  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:51.558815  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:51.608695  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.68:43449
I20260709 12:22:51.608806  5248 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.68:43449 every 8 connection(s)
I20260709 12:22:51.626053  5249 heartbeater.cc:344] Connected to a master server at 127.1.123.126:38285
I20260709 12:22:51.626808  5249 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:51.627810  5249 heartbeater.cc:507] Master 127.1.123.126:38285 requested a full tablet report, sending...
I20260709 12:22:51.630246  4899 ts_manager.cc:194] Registered new tserver with Master: 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:51.631171  1517 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.018991432s
I20260709 12:22:51.632475  4899 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43234
I20260709 12:22:51.663213  4899 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43236:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:22:51.928205  5066 tablet_service.cc:1511] Processing CreateTablet for tablet bfdb4c20c33743d79d57552db35c484b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:22:51.930015  5066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfdb4c20c33743d79d57552db35c484b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.931521  5065 tablet_service.cc:1511] Processing CreateTablet for tablet 874aa8edc94946d69e4a25563a5a0f0f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:22:51.933594  5065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 874aa8edc94946d69e4a25563a5a0f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.934368  5214 tablet_service.cc:1511] Processing CreateTablet for tablet bfdb4c20c33743d79d57552db35c484b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:22:51.935855  5214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfdb4c20c33743d79d57552db35c484b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.937999  5064 tablet_service.cc:1511] Processing CreateTablet for tablet 5244acc0775748b7933286d588ec2b56 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:22:51.938803  5213 tablet_service.cc:1511] Processing CreateTablet for tablet 4eb4fbb07495443bac93cfbcfa13bbe2 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:22:51.939697  5064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5244acc0775748b7933286d588ec2b56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.940227  5213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eb4fbb07495443bac93cfbcfa13bbe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.942879  5212 tablet_service.cc:1511] Processing CreateTablet for tablet 5244acc0775748b7933286d588ec2b56 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:22:51.943609  5063 tablet_service.cc:1511] Processing CreateTablet for tablet 5de3ceedceac4e0ca06de7a806ec05e3 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:22:51.944372  5212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5244acc0775748b7933286d588ec2b56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.945027  5063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de3ceedceac4e0ca06de7a806ec05e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.951891  5062 tablet_service.cc:1511] Processing CreateTablet for tablet 159b5709d37c488d935b1fe85d29c5bc (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:22:51.954145  5062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 159b5709d37c488d935b1fe85d29c5bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.959096  5061 tablet_service.cc:1511] Processing CreateTablet for tablet 63912da780fe41a2887dde6d2051ac8b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:22:51.960718  5061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63912da780fe41a2887dde6d2051ac8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.972141  5211 tablet_service.cc:1511] Processing CreateTablet for tablet 159b5709d37c488d935b1fe85d29c5bc (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:22:51.973904  5211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 159b5709d37c488d935b1fe85d29c5bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.974460  5060 tablet_service.cc:1511] Processing CreateTablet for tablet 09749d11015f4695b5c630c6e5dcbcb0 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:22:51.976369  5060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09749d11015f4695b5c630c6e5dcbcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.984208  5059 tablet_service.cc:1511] Processing CreateTablet for tablet 2b6b7243efc34b21882fab443d3a3720 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:22:51.986927  5059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b6b7243efc34b21882fab443d3a3720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.992326  5210 tablet_service.cc:1511] Processing CreateTablet for tablet 5e15328865c9479dab64cbc3fa161e86 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:22:51.992254  5058 tablet_service.cc:1511] Processing CreateTablet for tablet acca9bc33b6d43519a79138d66901f9b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:22:51.994354  5210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e15328865c9479dab64cbc3fa161e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:51.994652  5058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acca9bc33b6d43519a79138d66901f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.000175  5057 tablet_service.cc:1511] Processing CreateTablet for tablet 15dbeb6b54ba4f0c80adc43d8f184cb5 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:22:52.011849  5057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15dbeb6b54ba4f0c80adc43d8f184cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.025434  5207 tablet_service.cc:1511] Processing CreateTablet for tablet c0a39e3f43214d8ea4ff72401ffdbdcb (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:22:52.026605  5209 tablet_service.cc:1511] Processing CreateTablet for tablet 63912da780fe41a2887dde6d2051ac8b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:22:52.027961  5207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a39e3f43214d8ea4ff72401ffdbdcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.029399  5208 tablet_service.cc:1511] Processing CreateTablet for tablet 2b6b7243efc34b21882fab443d3a3720 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:22:52.030997  5208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b6b7243efc34b21882fab443d3a3720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.034996  5206 tablet_service.cc:1511] Processing CreateTablet for tablet acca9bc33b6d43519a79138d66901f9b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:22:52.036661  5206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acca9bc33b6d43519a79138d66901f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.038908  5205 tablet_service.cc:1511] Processing CreateTablet for tablet 30a1d0ab099f4c37a8f1fbe48688ac24 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:22:52.040583  5205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30a1d0ab099f4c37a8f1fbe48688ac24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.040465  4990 tablet_service.cc:1511] Processing CreateTablet for tablet 874aa8edc94946d69e4a25563a5a0f0f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:22:52.043044  4989 tablet_service.cc:1511] Processing CreateTablet for tablet 4eb4fbb07495443bac93cfbcfa13bbe2 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:22:52.045244  4988 tablet_service.cc:1511] Processing CreateTablet for tablet 5de3ceedceac4e0ca06de7a806ec05e3 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:22:52.047394  4987 tablet_service.cc:1511] Processing CreateTablet for tablet 159b5709d37c488d935b1fe85d29c5bc (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:22:52.049635  4986 tablet_service.cc:1511] Processing CreateTablet for tablet 5e15328865c9479dab64cbc3fa161e86 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:22:52.038949  4991 tablet_service.cc:1511] Processing CreateTablet for tablet bfdb4c20c33743d79d57552db35c484b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:22:52.052279  4990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 874aa8edc94946d69e4a25563a5a0f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.058775  4987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 159b5709d37c488d935b1fe85d29c5bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.063308  4985 tablet_service.cc:1511] Processing CreateTablet for tablet 09749d11015f4695b5c630c6e5dcbcb0 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:22:52.066689  4984 tablet_service.cc:1511] Processing CreateTablet for tablet 2b6b7243efc34b21882fab443d3a3720 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:22:52.068014  5209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63912da780fe41a2887dde6d2051ac8b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:22:52.075316  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:52.077494  4988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de3ceedceac4e0ca06de7a806ec05e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.078925  4982 tablet_service.cc:1511] Processing CreateTablet for tablet 15dbeb6b54ba4f0c80adc43d8f184cb5 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:22:52.077258  4983 tablet_service.cc:1511] Processing CreateTablet for tablet c0a39e3f43214d8ea4ff72401ffdbdcb (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:22:52.082929  5140 tablet_service.cc:1511] Processing CreateTablet for tablet 874aa8edc94946d69e4a25563a5a0f0f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:22:52.084620  5140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 874aa8edc94946d69e4a25563a5a0f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.086800  5134 tablet_service.cc:1511] Processing CreateTablet for tablet 09749d11015f4695b5c630c6e5dcbcb0 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:22:52.088393  5134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09749d11015f4695b5c630c6e5dcbcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.088127  5138 tablet_service.cc:1511] Processing CreateTablet for tablet 5244acc0775748b7933286d588ec2b56 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:22:52.090862  5136 tablet_service.cc:1511] Processing CreateTablet for tablet 5e15328865c9479dab64cbc3fa161e86 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:22:52.093390  5137 tablet_service.cc:1511] Processing CreateTablet for tablet 5de3ceedceac4e0ca06de7a806ec05e3 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:22:52.086843  5139 tablet_service.cc:1511] Processing CreateTablet for tablet 4eb4fbb07495443bac93cfbcfa13bbe2 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:22:52.084205  5135 tablet_service.cc:1511] Processing CreateTablet for tablet 63912da780fe41a2887dde6d2051ac8b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:22:52.097095  5138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5244acc0775748b7933286d588ec2b56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.097913  5136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e15328865c9479dab64cbc3fa161e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.098596  5139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eb4fbb07495443bac93cfbcfa13bbe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.099370  5135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63912da780fe41a2887dde6d2051ac8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.099028  5133 tablet_service.cc:1511] Processing CreateTablet for tablet c0a39e3f43214d8ea4ff72401ffdbdcb (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:22:52.100131  5137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de3ceedceac4e0ca06de7a806ec05e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.099112  5066 tablet_service.cc:1511] Processing CreateTablet for tablet 30a1d0ab099f4c37a8f1fbe48688ac24 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:22:52.100955  5133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a39e3f43214d8ea4ff72401ffdbdcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.101054  4991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfdb4c20c33743d79d57552db35c484b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.101356  5066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30a1d0ab099f4c37a8f1fbe48688ac24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.104490  4982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15dbeb6b54ba4f0c80adc43d8f184cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.105997  4983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a39e3f43214d8ea4ff72401ffdbdcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.113386  4985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09749d11015f4695b5c630c6e5dcbcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.118364  4986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e15328865c9479dab64cbc3fa161e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.122097  4984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b6b7243efc34b21882fab443d3a3720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.144027  5270 tablet_bootstrap.cc:492] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:52.144191  5214 tablet_service.cc:1511] Processing CreateTablet for tablet 9d055165bf9e4bfdb042cd7bd63ec993 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:22:52.147819  5132 tablet_service.cc:1511] Processing CreateTablet for tablet acca9bc33b6d43519a79138d66901f9b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:22:52.150573  5131 tablet_service.cc:1511] Processing CreateTablet for tablet 15dbeb6b54ba4f0c80adc43d8f184cb5 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:22:52.155972  4989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eb4fbb07495443bac93cfbcfa13bbe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.156436  5214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d055165bf9e4bfdb042cd7bd63ec993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.156466  5132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acca9bc33b6d43519a79138d66901f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.167222  5213 tablet_service.cc:1511] Processing CreateTablet for tablet db646a2919184c1682011b96d25d0cc9 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:22:52.153748  5211 tablet_service.cc:1511] Processing CreateTablet for tablet 58bebed3b53f4f178dae221a0daae185 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:22:52.179188  5131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15dbeb6b54ba4f0c80adc43d8f184cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.190412  5211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58bebed3b53f4f178dae221a0daae185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.195855  5212 tablet_service.cc:1511] Processing CreateTablet for tablet a6f50ca0c2db44278637fcdffffc03a2 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
W20260709 12:22:52.200832  4885 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 50508.2us
I20260709 12:22:52.197422  5212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6f50ca0c2db44278637fcdffffc03a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.210064  5213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db646a2919184c1682011b96d25d0cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.218931  5062 tablet_service.cc:1511] Processing CreateTablet for tablet a6f50ca0c2db44278637fcdffffc03a2 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:22:52.219923  5269 tablet_bootstrap.cc:492] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:52.223735  5057 tablet_service.cc:1511] Processing CreateTablet for tablet db646a2919184c1682011b96d25d0cc9 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:22:52.248534  5210 tablet_service.cc:1511] Processing CreateTablet for tablet 35db327bbc4d4777969286cb1174a85d (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:22:52.266258  5063 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfc1cfb224a4fd989b97af5dcf7da23 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:22:52.278342  5062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6f50ca0c2db44278637fcdffffc03a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.337250  5137 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfc1cfb224a4fd989b97af5dcf7da23 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:22:52.338959  5137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfc1cfb224a4fd989b97af5dcf7da23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.342366  5063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfc1cfb224a4fd989b97af5dcf7da23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.286633  5206 tablet_service.cc:1511] Processing CreateTablet for tablet eb1b479abeca4781986d8821b704c31f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:22:52.270743  5207 tablet_service.cc:1511] Processing CreateTablet for tablet 9ee83a16af0a40998fc58f3e882c6080 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:22:52.293610  5026 heartbeater.cc:499] Master 127.1.123.126:38285 was elected leader, sending a full tablet report...
I20260709 12:22:52.343441  5057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db646a2919184c1682011b96d25d0cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.295161  5064 tablet_service.cc:1511] Processing CreateTablet for tablet 63938e2a81834411b8018a2eb6c6818f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:22:52.346654  5064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63938e2a81834411b8018a2eb6c6818f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:22:52.349924  4884 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 134162us
I20260709 12:22:52.333045  5139 tablet_service.cc:1511] Processing CreateTablet for tablet db646a2919184c1682011b96d25d0cc9 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:22:52.371538  5058 tablet_service.cc:1511] Processing CreateTablet for tablet 7be74bdf01fe4190992575e84da14996 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:22:52.322190  5136 tablet_service.cc:1511] Processing CreateTablet for tablet 9d055165bf9e4bfdb042cd7bd63ec993 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:22:52.377245  5136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d055165bf9e4bfdb042cd7bd63ec993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.332181  5270 tablet_bootstrap.cc:654] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:52.333045  5059 tablet_service.cc:1511] Processing CreateTablet for tablet 58bebed3b53f4f178dae221a0daae185 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:22:52.380580  5059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58bebed3b53f4f178dae221a0daae185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.337253  5210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35db327bbc4d4777969286cb1174a85d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.383536  5207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee83a16af0a40998fc58f3e882c6080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.387446  5206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb1b479abeca4781986d8821b704c31f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.336771  4990 tablet_service.cc:1511] Processing CreateTablet for tablet 30a1d0ab099f4c37a8f1fbe48688ac24 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:22:52.383590  5271 tablet_bootstrap.cc:492] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:52.312043  5065 tablet_service.cc:1511] Processing CreateTablet for tablet 9ee83a16af0a40998fc58f3e882c6080 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:22:52.374235  5135 tablet_service.cc:1511] Processing CreateTablet for tablet 35db327bbc4d4777969286cb1174a85d (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:22:52.380117  5134 tablet_service.cc:1511] Processing CreateTablet for tablet 63938e2a81834411b8018a2eb6c6818f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:22:52.375089  5131 tablet_service.cc:1511] Processing CreateTablet for tablet 58bebed3b53f4f178dae221a0daae185 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:22:52.411047  5101 heartbeater.cc:499] Master 127.1.123.126:38285 was elected leader, sending a full tablet report...
I20260709 12:22:52.418196  5065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee83a16af0a40998fc58f3e882c6080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.427547  5272 tablet_bootstrap.cc:492] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:52.430114  4987 tablet_service.cc:1511] Processing CreateTablet for tablet 9d055165bf9e4bfdb042cd7bd63ec993 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
W20260709 12:22:52.413621  4885 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 101555us
W20260709 12:22:52.436801  4886 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 57769.7us
I20260709 12:22:52.444972  4990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30a1d0ab099f4c37a8f1fbe48688ac24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.433511  5139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db646a2919184c1682011b96d25d0cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.430155  4988 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfc1cfb224a4fd989b97af5dcf7da23 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:22:52.452770  5135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35db327bbc4d4777969286cb1174a85d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.461161  5271 tablet_bootstrap.cc:654] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:52.449469  5058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be74bdf01fe4190992575e84da14996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.441955  4982 tablet_service.cc:1511] Processing CreateTablet for tablet a6f50ca0c2db44278637fcdffffc03a2 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:22:52.475492  4983 tablet_service.cc:1511] Processing CreateTablet for tablet 63938e2a81834411b8018a2eb6c6818f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:22:52.476092  5131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58bebed3b53f4f178dae221a0daae185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.477988  5133 tablet_service.cc:1511] Processing CreateTablet for tablet eb1b479abeca4781986d8821b704c31f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:22:52.478556  5132 tablet_service.cc:1511] Processing CreateTablet for tablet 7be74bdf01fe4190992575e84da14996 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:22:52.479409  5133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb1b479abeca4781986d8821b704c31f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.480090  5132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be74bdf01fe4190992575e84da14996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.475952  5214 tablet_service.cc:1511] Processing CreateTablet for tablet 7be74bdf01fe4190992575e84da14996 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:22:52.487809  5134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63938e2a81834411b8018a2eb6c6818f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.492645  5214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be74bdf01fe4190992575e84da14996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.495543  5269 tablet_bootstrap.cc:654] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:52.512938  5175 heartbeater.cc:499] Master 127.1.123.126:38285 was elected leader, sending a full tablet report...
I20260709 12:22:52.508282  5270 tablet_bootstrap.cc:492] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:52.517908  5270 ts_tablet_manager.cc:1403] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.374s	user 0.026s	sys 0.033s
I20260709 12:22:52.510392  4987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d055165bf9e4bfdb042cd7bd63ec993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.525147  4983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63938e2a81834411b8018a2eb6c6818f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.528749  4988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfc1cfb224a4fd989b97af5dcf7da23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.552078  5060 tablet_service.cc:1511] Processing CreateTablet for tablet 805537066aa5440cbb0608aa77daafcf (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:22:52.555102  5270 raft_consensus.cc:359] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.555981  5270 raft_consensus.cc:385] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:52.556299  5270 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:52.557785  5060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 805537066aa5440cbb0608aa77daafcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.559805  4985 tablet_service.cc:1511] Processing CreateTablet for tablet 9ee83a16af0a40998fc58f3e882c6080 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:22:52.562144  4986 tablet_service.cc:1511] Processing CreateTablet for tablet eb1b479abeca4781986d8821b704c31f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:22:52.567852  5271 tablet_bootstrap.cc:492] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:52.569998  5140 tablet_service.cc:1511] Processing CreateTablet for tablet 805537066aa5440cbb0608aa77daafcf (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:22:52.568504  5271 ts_tablet_manager.cc:1403] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.185s	user 0.013s	sys 0.002s
I20260709 12:22:52.572237  4982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6f50ca0c2db44278637fcdffffc03a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.572990  4986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb1b479abeca4781986d8821b704c31f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.571748  5140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 805537066aa5440cbb0608aa77daafcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.546298  4984 tablet_service.cc:1511] Processing CreateTablet for tablet 35db327bbc4d4777969286cb1174a85d (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:22:52.581207  4984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35db327bbc4d4777969286cb1174a85d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.557211  5270 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.594512  4985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee83a16af0a40998fc58f3e882c6080. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:22:52.607291  4885 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 56078.8us
I20260709 12:22:52.609829  5271 raft_consensus.cc:359] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:52.610728  5271 raft_consensus.cc:385] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:52.611042  5271 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:52.611898  5271 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:52.616829  4991 tablet_service.cc:1511] Processing CreateTablet for tablet 805537066aa5440cbb0608aa77daafcf (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:22:52.615459  5272 tablet_bootstrap.cc:654] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:52.636015  5249 heartbeater.cc:499] Master 127.1.123.126:38285 was elected leader, sending a full tablet report...
I20260709 12:22:52.661394  4991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 805537066aa5440cbb0608aa77daafcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.698937  5269 tablet_bootstrap.cc:492] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:52.699666  5269 ts_tablet_manager.cc:1403] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.591s	user 0.032s	sys 0.012s
I20260709 12:22:52.703295  5269 raft_consensus.cc:359] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.704114  5269 raft_consensus.cc:385] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:52.704452  5269 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
W20260709 12:22:52.705790  4885 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 59488.4us
I20260709 12:22:52.709271  5270 ts_tablet_manager.cc:1434] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.191s	user 0.012s	sys 0.030s
I20260709 12:22:52.710737  5270 tablet_bootstrap.cc:492] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:52.714391  5271 ts_tablet_manager.cc:1434] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.142s	user 0.013s	sys 0.016s
I20260709 12:22:52.715785  5271 tablet_bootstrap.cc:492] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:52.705327  5269 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.722062  5270 tablet_bootstrap.cc:654] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:52.731748  5271 tablet_bootstrap.cc:654] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:52.737778  5176 tablet.cc:2406] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:22:52.747336  5272 tablet_bootstrap.cc:492] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:52.748006  5272 ts_tablet_manager.cc:1403] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.321s	user 0.014s	sys 0.017s
I20260709 12:22:52.772037  5063 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa8f9fb1ec8411aa7e23f87b827f019 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:22:52.776708  5063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa8f9fb1ec8411aa7e23f87b827f019. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:22:52.755532  4885 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 46816.6us
I20260709 12:22:52.755046  5272 raft_consensus.cc:359] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.783695  5272 raft_consensus.cc:385] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:52.784211  5272 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:52.785321  5272 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.789772  5269 ts_tablet_manager.cc:1434] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.090s	user 0.009s	sys 0.022s
I20260709 12:22:52.791016  5269 tablet_bootstrap.cc:492] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:52.792227  5139 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6732ed2583444ab53a0bd4a87ff1d0 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:22:52.792276  5214 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa8f9fb1ec8411aa7e23f87b827f019 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:22:52.790612  5133 tablet_service.cc:1511] Processing CreateTablet for tablet 3a7f46906b2d49fe83fd7c7729fc9d53 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:22:52.794323  5214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa8f9fb1ec8411aa7e23f87b827f019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.794414  5133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a7f46906b2d49fe83fd7c7729fc9d53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.798974  5269 tablet_bootstrap.cc:654] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:52.807551  5207 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6732ed2583444ab53a0bd4a87ff1d0 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:22:52.809229  5207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6732ed2583444ab53a0bd4a87ff1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.814184  5139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6732ed2583444ab53a0bd4a87ff1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.816859  4983 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa8f9fb1ec8411aa7e23f87b827f019 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:22:52.816803  4982 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6732ed2583444ab53a0bd4a87ff1d0 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:22:52.829793  4983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa8f9fb1ec8411aa7e23f87b827f019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.830768  5271 tablet_bootstrap.cc:492] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:52.830606  5059 tablet_service.cc:1511] Processing CreateTablet for tablet 3a7f46906b2d49fe83fd7c7729fc9d53 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:22:52.831372  5271 ts_tablet_manager.cc:1403] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.116s	user 0.008s	sys 0.014s
I20260709 12:22:52.832302  5059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a7f46906b2d49fe83fd7c7729fc9d53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.833947  4982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6732ed2583444ab53a0bd4a87ff1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.843953  5206 tablet_service.cc:1511] Processing CreateTablet for tablet 3a7f46906b2d49fe83fd7c7729fc9d53 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:22:52.854334  5206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a7f46906b2d49fe83fd7c7729fc9d53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.861931  5272 ts_tablet_manager.cc:1434] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.114s	user 0.005s	sys 0.008s
I20260709 12:22:52.862844  4991 tablet_service.cc:1511] Processing CreateTablet for tablet 59bc6f04e62a431ba2f113d08a129898 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:22:52.864332  4991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59bc6f04e62a431ba2f113d08a129898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.844611  5271 raft_consensus.cc:359] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.867676  5271 raft_consensus.cc:385] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:52.868043  5271 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:52.868919  5270 tablet_bootstrap.cc:492] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:52.863281  5272 tablet_bootstrap.cc:492] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:52.875209  5270 ts_tablet_manager.cc:1403] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.159s	user 0.015s	sys 0.009s
I20260709 12:22:52.882421  5270 raft_consensus.cc:359] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.883272  5270 raft_consensus.cc:385] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:52.883600  5270 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:52.887877  5063 tablet_service.cc:1511] Processing CreateTablet for tablet 59bc6f04e62a431ba2f113d08a129898 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:22:52.889119  5063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59bc6f04e62a431ba2f113d08a129898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.891948  5272 tablet_bootstrap.cc:654] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:52.868939  5271 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.894798  5277 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.895321  5277 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.897882  5276 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.898447  5276 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.904755  5277 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:52.906749  5269 tablet_bootstrap.cc:492] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:52.931051  4984 tablet_service.cc:1511] Processing CreateTablet for tablet 94a5838186ec4283b67fe9aca66d2a22 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:22:52.931124  5214 tablet_service.cc:1511] Processing CreateTablet for tablet 94a5838186ec4283b67fe9aca66d2a22 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:22:52.932981  4984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94a5838186ec4283b67fe9aca66d2a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.910498  5276 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:52.937415  5277 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.938103  5277 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:52.939043  5214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94a5838186ec4283b67fe9aca66d2a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.909672  5270 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:52.941967  5269 ts_tablet_manager.cc:1403] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.151s	user 0.018s	sys 0.005s
I20260709 12:22:52.953004  5277 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:52.954090  5270 ts_tablet_manager.cc:1434] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.001s
I20260709 12:22:52.955730  5269 raft_consensus.cc:359] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:52.956547  5269 raft_consensus.cc:385] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:52.956830  5269 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:52.960618  5269 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:52.989333  4982 tablet_service.cc:1511] Processing CreateTablet for tablet 77f30d7c9f3244999f248bc1aed226c7 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:22:52.990469  5269 ts_tablet_manager.cc:1434] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 12:22:52.991082  4982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f30d7c9f3244999f248bc1aed226c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:52.991809  5269 tablet_bootstrap.cc:492] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:52.992365  5270 tablet_bootstrap.cc:492] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:52.994732  5271 ts_tablet_manager.cc:1434] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.163s	user 0.010s	sys 0.010s
I20260709 12:22:52.996003  5271 tablet_bootstrap.cc:492] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:53.022200  5272 tablet_bootstrap.cc:492] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:53.022897  5272 ts_tablet_manager.cc:1403] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.160s	user 0.006s	sys 0.032s
I20260709 12:22:53.026528  5272 raft_consensus.cc:359] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:53.027346  5272 raft_consensus.cc:385] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:53.027665  5272 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:53.028496  5272 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:53.031350  5272 ts_tablet_manager.cc:1434] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:22:53.030354  5062 tablet_service.cc:1511] Processing CreateTablet for tablet 4935fc8f35664324a253df9804c8074d (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:22:53.036760  5062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4935fc8f35664324a253df9804c8074d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.037132  5282 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.037559  4983 tablet_service.cc:1511] Processing CreateTablet for tablet 4935fc8f35664324a253df9804c8074d (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:22:53.037762  5282 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:53.038985  4983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4935fc8f35664324a253df9804c8074d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.030164  5059 tablet_service.cc:1511] Processing CreateTablet for tablet 7383bc919f9c47378977b98535752581 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:22:53.028959  5063 tablet_service.cc:1511] Processing CreateTablet for tablet 94a5838186ec4283b67fe9aca66d2a22 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:22:53.041488  5272 tablet_bootstrap.cc:492] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:53.036713  5270 tablet_bootstrap.cc:654] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:53.057346  5271 tablet_bootstrap.cc:654] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:53.047991  5059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7383bc919f9c47378977b98535752581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.066964  5282 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:53.078220  5269 tablet_bootstrap.cc:654] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:53.091372  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:53.092710  5063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94a5838186ec4283b67fe9aca66d2a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.092417  5058 tablet_service.cc:1511] Processing CreateTablet for tablet f6a42608cf874aa6943aaf8e349714b9 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:22:53.093175  5206 tablet_service.cc:1511] Processing CreateTablet for tablet 77f30d7c9f3244999f248bc1aed226c7 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:22:53.094092  5058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6a42608cf874aa6943aaf8e349714b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.094566  5206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f30d7c9f3244999f248bc1aed226c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.098687  5207 tablet_service.cc:1511] Processing CreateTablet for tablet 7383bc919f9c47378977b98535752581 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:22:53.099743  4982 tablet_service.cc:1511] Processing CreateTablet for tablet f6a42608cf874aa6943aaf8e349714b9 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:22:53.100227  5207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7383bc919f9c47378977b98535752581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.101177  4982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6a42608cf874aa6943aaf8e349714b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.105017  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:22:53.107227  5114 leader_election.cc:343] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.111166  5279 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.111806  5279 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:53.117904  5210 tablet_service.cc:1511] Processing CreateTablet for tablet f6a42608cf874aa6943aaf8e349714b9 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:22:53.123823  5211 tablet_service.cc:1511] Processing CreateTablet for tablet 85f48767a7ee419197aff24610838e25 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:22:53.124555  5210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6a42608cf874aa6943aaf8e349714b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.124456  5057 tablet_service.cc:1511] Processing CreateTablet for tablet e4be6683658549c99c727b84f9cd9774 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:22:53.125521  5211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f48767a7ee419197aff24610838e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.126485  5057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4be6683658549c99c727b84f9cd9774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.145120  5213 tablet_service.cc:1511] Processing CreateTablet for tablet e4be6683658549c99c727b84f9cd9774 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:22:53.147353  5213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4be6683658549c99c727b84f9cd9774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.154646  5279 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:53.155365  5279 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.162436  5282 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.167729  5282 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:53.173596  4991 tablet_service.cc:1511] Processing CreateTablet for tablet 15944aba8caf4c1497501efd45dc0430 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:22:53.174513  5272 tablet_bootstrap.cc:654] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:53.175112  4991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15944aba8caf4c1497501efd45dc0430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.174746  5065 tablet_service.cc:1511] Processing CreateTablet for tablet 15944aba8caf4c1497501efd45dc0430 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:22:53.182443  5065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15944aba8caf4c1497501efd45dc0430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.185168  5282 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:53.169324  5279 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:53.191366  5279 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:53.196049  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:53.196370  4984 tablet_service.cc:1511] Processing CreateTablet for tablet 85f48767a7ee419197aff24610838e25 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:22:53.198710  4984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f48767a7ee419197aff24610838e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.225700  5276 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.226392  5276 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:53.228969  5276 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:53.234195  5139 tablet_service.cc:1511] Processing CreateTablet for tablet 59bc6f04e62a431ba2f113d08a129898 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:22:53.235708  5139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59bc6f04e62a431ba2f113d08a129898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.247068  5214 tablet_service.cc:1511] Processing CreateTablet for tablet c1c232e09b414797838d5e57d028ca97 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
W20260709 12:22:53.252250  5189 leader_election.cc:343] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.270028  5133 tablet_service.cc:1511] Processing CreateTablet for tablet 77f30d7c9f3244999f248bc1aed226c7 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:22:53.270987  5269 tablet_bootstrap.cc:492] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:53.271574  5269 ts_tablet_manager.cc:1403] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.280s	user 0.011s	sys 0.021s
I20260709 12:22:53.271633  5133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f30d7c9f3244999f248bc1aed226c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.276934  5269 raft_consensus.cc:359] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:53.277806  5269 raft_consensus.cc:385] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:53.266456  5062 tablet_service.cc:1511] Processing CreateTablet for tablet c1c232e09b414797838d5e57d028ca97 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:22:53.281096  5062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c232e09b414797838d5e57d028ca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.287194  5134 tablet_service.cc:1511] Processing CreateTablet for tablet 7383bc919f9c47378977b98535752581 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:22:53.287405  5136 tablet_service.cc:1511] Processing CreateTablet for tablet 4935fc8f35664324a253df9804c8074d (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:22:53.288141  5140 tablet_service.cc:1511] Processing CreateTablet for tablet e4be6683658549c99c727b84f9cd9774 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:22:53.288879  5134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7383bc919f9c47378977b98535752581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.290565  5137 tablet_service.cc:1511] Processing CreateTablet for tablet 85f48767a7ee419197aff24610838e25 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:22:53.290927  5138 tablet_service.cc:1511] Processing CreateTablet for tablet 15944aba8caf4c1497501efd45dc0430 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:22:53.291887  5137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f48767a7ee419197aff24610838e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.293591  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:53.297824  5271 tablet_bootstrap.cc:492] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:53.298454  5271 ts_tablet_manager.cc:1403] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.303s	user 0.012s	sys 0.007s
I20260709 12:22:53.252889  5270 tablet_bootstrap.cc:492] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:53.306859  5136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4935fc8f35664324a253df9804c8074d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:22:53.308815  5190 leader_election.cc:343] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.309403  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:53.310380  5276 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:53.310873  5212 tablet_service.cc:1511] Processing CreateTablet for tablet 82e74ddaf2154bc1ba1aeb5775d6e514 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:22:53.311888  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:53.312024  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:53.312637  5132 tablet_service.cc:1511] Processing CreateTablet for tablet 82e74ddaf2154bc1ba1aeb5775d6e514 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
W20260709 12:22:53.314903  5115 leader_election.cc:343] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.317112  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:53.316084  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:22:53.311395  5140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4be6683658549c99c727b84f9cd9774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.323184  5132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82e74ddaf2154bc1ba1aeb5775d6e514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.320724  5271 raft_consensus.cc:359] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:53.314893  4982 tablet_service.cc:1511] Processing CreateTablet for tablet c1c232e09b414797838d5e57d028ca97 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:22:53.332902  5271 raft_consensus.cc:385] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:53.333364  5271 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:53.330407  5138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15944aba8caf4c1497501efd45dc0430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.278139  5269 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:53.319908  5270 ts_tablet_manager.cc:1403] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.328s	user 0.025s	sys 0.006s
I20260709 12:22:53.338907  4986 tablet_service.cc:1511] Processing CreateTablet for tablet 82e74ddaf2154bc1ba1aeb5775d6e514 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:22:53.341501  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:53.342875  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:53.343621  5076 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:53.344385  5272 tablet_bootstrap.cc:492] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:53.344995  5272 ts_tablet_manager.cc:1403] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.309s	user 0.014s	sys 0.010s
I20260709 12:22:53.345970  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:53.345082  5269 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:53.348629  5272 raft_consensus.cc:359] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:53.349457  5272 raft_consensus.cc:385] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:53.352413  5272 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:53.356228  5214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c232e09b414797838d5e57d028ca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.359298  5212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82e74ddaf2154bc1ba1aeb5775d6e514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.360481  5286 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:53.363266  4982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c232e09b414797838d5e57d028ca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.369059  4986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82e74ddaf2154bc1ba1aeb5775d6e514. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:22:53.347918  4965 leader_election.cc:343] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.357141  5271 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:53.377801  5271 ts_tablet_manager.cc:1434] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.079s	user 0.010s	sys 0.000s
I20260709 12:22:53.384011  5271 tablet_bootstrap.cc:492] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
W20260709 12:22:53.384454  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 69994.9us
W20260709 12:22:53.410630  5115 leader_election.cc:343] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.391319  5270 raft_consensus.cc:359] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:53.412199  5270 raft_consensus.cc:385] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:53.411931  5057 tablet_service.cc:1511] Processing CreateTablet for tablet a8bbb28bc9f5410d95b20827beeb87cc (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 12:22:53.351706  5189 leader_election.cc:343] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.401031  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:53.415239  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:22:53.411947  5115 leader_election.cc:343] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.400061  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:53.416326  5276 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:22:53.416703  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 62942.1us
W20260709 12:22:53.417527  5041 leader_election.cc:343] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.418464  5276 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.419416  5276 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:53.418756  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:53.420384  5286 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:53.423130  5135 tablet_service.cc:1511] Processing CreateTablet for tablet a8bbb28bc9f5410d95b20827beeb87cc (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:22:53.424566  5135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8bbb28bc9f5410d95b20827beeb87cc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:22:53.418277  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 66614.2us
I20260709 12:22:53.430027  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:22:53.431785  4966 leader_election.cc:343] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.432583  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:53.432698  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:53.433360  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:53.429339  5276 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:22:53.434847  4967 leader_election.cc:343] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.435425  4967 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:53.436324  5282 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:22:53.436786  4967 leader_election.cc:343] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.437280  4967 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:53.438071  5282 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:53.392303  5272 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:53.439481  5000 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:53.442279  5224 raft_consensus.cc:2393] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 1: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:22:53.445078  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:53.451017  5076 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:53.451748  4991 tablet_service.cc:1511] Processing CreateTablet for tablet 719a453defab43d4a1232c664dbaae9c (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:22:53.453042  5269 ts_tablet_manager.cc:1434] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.181s	user 0.012s	sys 0.000s
I20260709 12:22:53.453238  4991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719a453defab43d4a1232c664dbaae9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.455716  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:53.446285  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:53.461182  5302 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:53.461700  5302 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.462037  5302 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:53.465837  5063 tablet_service.cc:1511] Processing CreateTablet for tablet 719a453defab43d4a1232c664dbaae9c (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:22:53.471441  5063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719a453defab43d4a1232c664dbaae9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.472313  5076 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:53.473968  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:22:53.415249  5207 tablet_service.cc:1511] Processing CreateTablet for tablet a8bbb28bc9f5410d95b20827beeb87cc (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:22:53.478197  5057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8bbb28bc9f5410d95b20827beeb87cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.479516  5269 tablet_bootstrap.cc:492] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:53.482471  5207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8bbb28bc9f5410d95b20827beeb87cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.491588  5269 tablet_bootstrap.cc:654] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:53.441377  5272 ts_tablet_manager.cc:1434] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.096s	user 0.005s	sys 0.006s
I20260709 12:22:53.493045  5272 tablet_bootstrap.cc:492] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:53.487007  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:53.445662  5276 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:53.488716  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:53.502162  5279 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:53.504021  5276 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:53.504627  5276 raft_consensus.cc:697] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:53.412634  5270 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:53.493142  5206 tablet_service.cc:1511] Processing CreateTablet for tablet 75210ca16c9d4d0e91d54b6ffbbd75d4 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:22:53.476894  5140 tablet_service.cc:1511] Processing CreateTablet for tablet 719a453defab43d4a1232c664dbaae9c (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 12:22:53.505340  5189 leader_election.cc:343] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.490477  5302 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:53.524923  5206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75210ca16c9d4d0e91d54b6ffbbd75d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.526795  4984 tablet_service.cc:1511] Processing CreateTablet for tablet 75210ca16c9d4d0e91d54b6ffbbd75d4 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:22:53.528998  4984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75210ca16c9d4d0e91d54b6ffbbd75d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.538631  5136 tablet_service.cc:1511] Processing CreateTablet for tablet cf661d015f0e4fc19e57069da58329e3 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:22:53.539480  5058 tablet_service.cc:1511] Processing CreateTablet for tablet 75210ca16c9d4d0e91d54b6ffbbd75d4 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:22:53.541152  5058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75210ca16c9d4d0e91d54b6ffbbd75d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.543246  5302 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:53.524923  5140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719a453defab43d4a1232c664dbaae9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.544440  5136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf661d015f0e4fc19e57069da58329e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.544539  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:53.545389  5000 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:53.553773  5270 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:53.555151  5223 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:53.562242  5212 tablet_service.cc:1511] Processing CreateTablet for tablet cf661d015f0e4fc19e57069da58329e3 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:22:53.563891  5212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf661d015f0e4fc19e57069da58329e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.564999  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:53.567291  5223 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:53.568099  5272 tablet_bootstrap.cc:654] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:53.568372  4983 tablet_service.cc:1511] Processing CreateTablet for tablet cf661d015f0e4fc19e57069da58329e3 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:22:53.557237  5276 consensus_queue.cc:237] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:53.570053  4983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf661d015f0e4fc19e57069da58329e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.577242  5135 tablet_service.cc:1511] Processing CreateTablet for tablet c86395378750473e80c76902ad29fbe7 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:22:53.578428  5210 tablet_service.cc:1511] Processing CreateTablet for tablet c86395378750473e80c76902ad29fbe7 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:22:53.579107  5135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86395378750473e80c76902ad29fbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.579878  5210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86395378750473e80c76902ad29fbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.584234  5269 tablet_bootstrap.cc:492] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:53.584937  5269 ts_tablet_manager.cc:1403] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.106s	user 0.020s	sys 0.021s
I20260709 12:22:53.589570  4987 tablet_service.cc:1511] Processing CreateTablet for tablet 552d065b39ba44699c339e80bcc02744 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:22:53.590871  5062 tablet_service.cc:1511] Processing CreateTablet for tablet c86395378750473e80c76902ad29fbe7 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:22:53.591346  4987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552d065b39ba44699c339e80bcc02744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.600601  5065 tablet_service.cc:1511] Processing CreateTablet for tablet 552d065b39ba44699c339e80bcc02744 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:22:53.610149  5302 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.610881  5302 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:53.611387  5138 tablet_service.cc:1511] Processing CreateTablet for tablet 552d065b39ba44699c339e80bcc02744 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:22:53.613134  5138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552d065b39ba44699c339e80bcc02744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.612780  5214 tablet_service.cc:1511] Processing CreateTablet for tablet 1339c2a0d7f344d09f483e6d965adcc8 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:22:53.617779  5062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86395378750473e80c76902ad29fbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.618064  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:22:53.597921  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 124047us
I20260709 12:22:53.621707  4991 tablet_service.cc:1511] Processing CreateTablet for tablet 1339c2a0d7f344d09f483e6d965adcc8 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:22:53.617784  5271 tablet_bootstrap.cc:654] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:53.623387  4991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1339c2a0d7f344d09f483e6d965adcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.625689  5065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552d065b39ba44699c339e80bcc02744. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:22:53.625741  5041 leader_election.cc:343] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.633508  5302 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:53.637950  5223 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:53.641578  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:22:53.643186  5302 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:53.643893  5302 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:22:53.646453  5302 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:53.659011  5270 ts_tablet_manager.cc:1434] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.309s	user 0.014s	sys 0.001s
W20260709 12:22:53.669749  4885 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 46954.4us
I20260709 12:22:53.670905  5000 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:53.678469  5133 tablet_service.cc:1511] Processing CreateTablet for tablet 003133c544c3449a827e3758faa02f02 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:22:53.680186  5133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 003133c544c3449a827e3758faa02f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.692878  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:22:53.695560  5042 leader_election.cc:343] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:53.696179  5042 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:53.697116  5279 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:53.698876  5214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1339c2a0d7f344d09f483e6d965adcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.698567  5211 tablet_service.cc:1511] Processing CreateTablet for tablet 003133c544c3449a827e3758faa02f02 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:22:53.713346  5269 raft_consensus.cc:359] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:53.714296  5269 raft_consensus.cc:385] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:53.714596  5269 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:53.717003  5211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 003133c544c3449a827e3758faa02f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.725309  5272 tablet_bootstrap.cc:492] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
W20260709 12:22:53.711442  4884 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45010.5us
I20260709 12:22:53.726017  5272 ts_tablet_manager.cc:1403] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.233s	user 0.016s	sys 0.015s
I20260709 12:22:53.725096  4898 catalog_manager.cc:5697] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:53.731621  5063 tablet_service.cc:1511] Processing CreateTablet for tablet 5b780c93046447618cab07ac8b85cdfd (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:22:53.733359  5063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b780c93046447618cab07ac8b85cdfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.729429  5139 tablet_service.cc:1511] Processing CreateTablet for tablet 5b780c93046447618cab07ac8b85cdfd (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:22:53.736060  5270 tablet_bootstrap.cc:492] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:53.736871  5139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b780c93046447618cab07ac8b85cdfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.742424  5269 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:53.751292  5269 ts_tablet_manager.cc:1434] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.166s	user 0.009s	sys 0.006s
I20260709 12:22:53.750954  5210 tablet_service.cc:1511] Processing CreateTablet for tablet 5b780c93046447618cab07ac8b85cdfd (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:22:53.748023  5271 tablet_bootstrap.cc:492] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:53.752835  5057 tablet_service.cc:1511] Processing CreateTablet for tablet 1339c2a0d7f344d09f483e6d965adcc8 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:22:53.765027  4982 tablet_service.cc:1511] Processing CreateTablet for tablet e77063f082034d8bbc280357b7b66bb7 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:22:53.766669  4982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77063f082034d8bbc280357b7b66bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.773669  5272 raft_consensus.cc:359] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:53.774626  5272 raft_consensus.cc:385] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:53.775035  5272 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:53.774713  5059 tablet_service.cc:1511] Processing CreateTablet for tablet e77063f082034d8bbc280357b7b66bb7 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:22:53.775890  5272 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:53.778546  5057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1339c2a0d7f344d09f483e6d965adcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.780112  5210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b780c93046447618cab07ac8b85cdfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.780987  4984 tablet_service.cc:1511] Processing CreateTablet for tablet 003133c544c3449a827e3758faa02f02 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:22:53.792129  5286 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.792693  5269 tablet_bootstrap.cc:492] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:53.770534  5271 ts_tablet_manager.cc:1403] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.387s	user 0.006s	sys 0.013s
I20260709 12:22:53.794801  5140 tablet_service.cc:1511] Processing CreateTablet for tablet e77063f082034d8bbc280357b7b66bb7 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:22:53.792356  5270 tablet_bootstrap.cc:654] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:53.831525  5059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77063f082034d8bbc280357b7b66bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.828646  5286 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:53.845671  5140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77063f082034d8bbc280357b7b66bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.840194  4984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 003133c544c3449a827e3758faa02f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.854820  4983 tablet_service.cc:1511] Processing CreateTablet for tablet c5751113d37b44af87d3d912da388078 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:22:53.858822  5286 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:53.859941  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:53.860265  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:53.858678  4983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5751113d37b44af87d3d912da388078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.861474  5000 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:22:53.863511  5115 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:53.861593  5076 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:22:53.865302  5286 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:53.865783  5286 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.866149  5286 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:53.870182  5063 tablet_service.cc:1511] Processing CreateTablet for tablet c5751113d37b44af87d3d912da388078 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:22:53.871721  5063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5751113d37b44af87d3d912da388078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.875258  5214 tablet_service.cc:1511] Processing CreateTablet for tablet 5deada35125f4c008544b2ff246b039c (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:22:53.876919  5214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5deada35125f4c008544b2ff246b039c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.877728  5206 tablet_service.cc:1511] Processing CreateTablet for tablet c5751113d37b44af87d3d912da388078 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:22:53.850059  5271 raft_consensus.cc:359] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:53.880059  5206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5751113d37b44af87d3d912da388078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.880059  5271 raft_consensus.cc:385] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:53.880612  5271 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:53.883904  5272 ts_tablet_manager.cc:1434] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.157s	user 0.009s	sys 0.000s
I20260709 12:22:53.885246  5272 tablet_bootstrap.cc:492] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:53.884984  5062 tablet_service.cc:1511] Processing CreateTablet for tablet 6c91c376e83f484baca078b37cbf00db (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:22:53.892874  5138 tablet_service.cc:1511] Processing CreateTablet for tablet 6c91c376e83f484baca078b37cbf00db (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:22:53.894507  5138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c91c376e83f484baca078b37cbf00db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.897444  5139 tablet_service.cc:1511] Processing CreateTablet for tablet 5deada35125f4c008544b2ff246b039c (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:22:53.898962  5139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5deada35125f4c008544b2ff246b039c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.902769  5211 tablet_service.cc:1511] Processing CreateTablet for tablet 6c91c376e83f484baca078b37cbf00db (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:22:53.903486  5286 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:53.904533  5211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c91c376e83f484baca078b37cbf00db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.906279  4991 tablet_service.cc:1511] Processing CreateTablet for tablet 5deada35125f4c008544b2ff246b039c (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:22:53.909626  4991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5deada35125f4c008544b2ff246b039c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.911813  4987 tablet_service.cc:1511] Processing CreateTablet for tablet 01e6663f57ec46609e0512d07df6f705 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:22:53.913455  4987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e6663f57ec46609e0512d07df6f705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.913796  5062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c91c376e83f484baca078b37cbf00db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.926154  5269 tablet_bootstrap.cc:654] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:53.881635  5271 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:53.929327  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:53.930506  5000 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:53.944710  5000 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:53.953449  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:53.954545  5076 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:53.968127  5286 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:53.979640  5115 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:53.987349  5065 tablet_service.cc:1511] Processing CreateTablet for tablet 01e6663f57ec46609e0512d07df6f705 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:22:53.988858  5065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e6663f57ec46609e0512d07df6f705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.988930  5133 tablet_service.cc:1511] Processing CreateTablet for tablet 01e6663f57ec46609e0512d07df6f705 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:22:53.990353  5133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e6663f57ec46609e0512d07df6f705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:53.992585  5076 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:53.999501  5211 tablet_service.cc:1511] Processing CreateTablet for tablet 0141a059bf6949c49e5e0091604786e6 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:22:54.001072  5211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0141a059bf6949c49e5e0091604786e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.002902  5270 tablet_bootstrap.cc:492] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:54.003279  5269 tablet_bootstrap.cc:492] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:54.003492  5270 ts_tablet_manager.cc:1403] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.269s	user 0.026s	sys 0.037s
I20260709 12:22:54.003870  5269 ts_tablet_manager.cc:1403] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.213s	user 0.014s	sys 0.006s
I20260709 12:22:54.013660  5270 raft_consensus.cc:359] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.014551  5270 raft_consensus.cc:385] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.014197  4982 tablet_service.cc:1511] Processing CreateTablet for tablet 0141a059bf6949c49e5e0091604786e6 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:22:54.014940  5270 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.015762  4982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0141a059bf6949c49e5e0091604786e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.015797  5270 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.011392  5286 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:54.017215  5286 raft_consensus.cc:697] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:22:54.018584  5286 consensus_queue.cc:237] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.021291  5271 ts_tablet_manager.cc:1434] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.191s	user 0.013s	sys 0.009s
I20260709 12:22:54.022600  5271 tablet_bootstrap.cc:492] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:54.010339  5272 tablet_bootstrap.cc:654] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.024258  5269 raft_consensus.cc:359] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.028445  5270 ts_tablet_manager.cc:1434] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.025s	user 0.010s	sys 0.000s
I20260709 12:22:54.029814  5270 tablet_bootstrap.cc:492] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:54.034257  5057 tablet_service.cc:1511] Processing CreateTablet for tablet 0141a059bf6949c49e5e0091604786e6 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:22:54.025137  5269 raft_consensus.cc:385] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.037737  5269 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.054104  5138 tablet_service.cc:1511] Processing CreateTablet for tablet 2725c773e7344020b5752861d00b493e (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:22:54.055740  5138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2725c773e7344020b5752861d00b493e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.060596  5057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0141a059bf6949c49e5e0091604786e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.060927  5206 tablet_service.cc:1511] Processing CreateTablet for tablet 2725c773e7344020b5752861d00b493e (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:22:54.061180  5059 tablet_service.cc:1511] Processing CreateTablet for tablet 6a61a09c949a4cc9959bd9cfbe00992a (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:22:54.062455  5206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2725c773e7344020b5752861d00b493e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.062600  5059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a61a09c949a4cc9959bd9cfbe00992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.065665  5269 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.068439  5269 ts_tablet_manager.cc:1434] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.064s	user 0.004s	sys 0.004s
I20260709 12:22:54.083592  4982 tablet_service.cc:1511] Processing CreateTablet for tablet 2725c773e7344020b5752861d00b493e (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:22:54.086824  4982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2725c773e7344020b5752861d00b493e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.085338  5269 tablet_bootstrap.cc:492] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:54.089372  5140 tablet_service.cc:1511] Processing CreateTablet for tablet 6a61a09c949a4cc9959bd9cfbe00992a (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:22:54.102862  5140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a61a09c949a4cc9959bd9cfbe00992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.095230  5271 tablet_bootstrap.cc:654] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.097337  5272 tablet_bootstrap.cc:492] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:54.106418  5272 ts_tablet_manager.cc:1403] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.222s	user 0.018s	sys 0.004s
I20260709 12:22:54.099143  5270 tablet_bootstrap.cc:654] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.114691  4984 tablet_service.cc:1511] Processing CreateTablet for tablet 4516325f133e4ea8beacc1968f96ee9b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:22:54.116242  4984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4516325f133e4ea8beacc1968f96ee9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.116202  5272 raft_consensus.cc:359] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.117134  5272 raft_consensus.cc:385] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.114392  5210 tablet_service.cc:1511] Processing CreateTablet for tablet 6a61a09c949a4cc9959bd9cfbe00992a (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:22:54.119403  5210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a61a09c949a4cc9959bd9cfbe00992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.132006  5062 tablet_service.cc:1511] Processing CreateTablet for tablet 4516325f133e4ea8beacc1968f96ee9b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:22:54.133635  5062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4516325f133e4ea8beacc1968f96ee9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.117431  5272 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
W20260709 12:22:54.140319  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:54.141309  5302 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:22:54.153214  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:54.153995  5272 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.156174  5272 ts_tablet_manager.cc:1434] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 12:22:54.157902  5272 tablet_bootstrap.cc:492] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:54.135663  4897 catalog_manager.cc:5697] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:54.162670  5276 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:54.172992  5270 tablet_bootstrap.cc:492] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:54.174898  5270 ts_tablet_manager.cc:1403] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.145s	user 0.009s	sys 0.009s
I20260709 12:22:54.180312  5270 raft_consensus.cc:359] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.181135  5270 raft_consensus.cc:385] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.181447  5302 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:54.197242  4899 catalog_manager.cc:5697] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:54.181454  5270 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.193506  5138 tablet_service.cc:1511] Processing CreateTablet for tablet 4516325f133e4ea8beacc1968f96ee9b (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:22:54.207513  5138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4516325f133e4ea8beacc1968f96ee9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.215901  5269 tablet_bootstrap.cc:654] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.221169  5214 tablet_service.cc:1511] Processing CreateTablet for tablet 4da8de53d0db43f9a92906a98dfda98f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:22:54.218362  5270 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.223646  5214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da8de53d0db43f9a92906a98dfda98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.236419  5270 ts_tablet_manager.cc:1434] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.002s
I20260709 12:22:54.240124  4982 tablet_service.cc:1511] Processing CreateTablet for tablet 4da8de53d0db43f9a92906a98dfda98f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:22:54.241505  4982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da8de53d0db43f9a92906a98dfda98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.238181  5270 tablet_bootstrap.cc:492] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:54.254015  5271 tablet_bootstrap.cc:492] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:54.254704  5271 ts_tablet_manager.cc:1403] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.232s	user 0.023s	sys 0.007s
I20260709 12:22:54.256057  5065 tablet_service.cc:1511] Processing CreateTablet for tablet 4da8de53d0db43f9a92906a98dfda98f (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:22:54.258714  5133 tablet_service.cc:1511] Processing CreateTablet for tablet 3665d1ab30ab4c8d9b7304c26a0cd138 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:22:54.260236  5133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3665d1ab30ab4c8d9b7304c26a0cd138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.242136  5276 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.263537  5276 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.264235  5211 tablet_service.cc:1511] Processing CreateTablet for tablet 3665d1ab30ab4c8d9b7304c26a0cd138 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:22:54.264362  5271 raft_consensus.cc:359] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.265203  5271 raft_consensus.cc:385] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.265529  5271 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.265944  5272 tablet_bootstrap.cc:700] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Time spent opening tablet: real 0.100s	user 0.001s	sys 0.001s
I20260709 12:22:54.266702  5272 tablet_bootstrap.cc:654] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.269802  5211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3665d1ab30ab4c8d9b7304c26a0cd138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.269896  5271 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.278368  4991 tablet_service.cc:1511] Processing CreateTablet for tablet 3665d1ab30ab4c8d9b7304c26a0cd138 (DEFAULT_TABLE table=test-workload [id=7926611e50b544118a4999e26923a1a2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:22:54.279963  4991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3665d1ab30ab4c8d9b7304c26a0cd138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.283417  5271 ts_tablet_manager.cc:1434] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:22:54.284626  5271 tablet_bootstrap.cc:492] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:54.293336  5065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da8de53d0db43f9a92906a98dfda98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:54.297823  5286 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.303287  5286 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.307363  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:54.313212  5286 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:54.315258  5276 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:54.314496  5270 tablet_bootstrap.cc:654] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:54.300881  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 57722us
W20260709 12:22:54.325522  5188 leader_election.cc:343] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.329804  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:54.340636  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:54.340718  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:54.345118  5224 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:22:54.350770  5269 tablet_bootstrap.cc:492] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:54.351495  5269 ts_tablet_manager.cc:1403] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.282s	user 0.019s	sys 0.022s
W20260709 12:22:54.369352  5115 leader_election.cc:343] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.374588  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:54.375555  5286 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:54.375984  5286 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.376303  5286 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.381191  5272 tablet_bootstrap.cc:492] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:54.382555  5272 ts_tablet_manager.cc:1403] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.225s	user 0.023s	sys 0.016s
I20260709 12:22:54.383208  5286 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:22:54.370639  5190 leader_election.cc:343] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.386040  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:54.386374  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:54.387049  5224 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.403218  5338 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:54.405381  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:22:54.408367  5115 leader_election.cc:343] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.401979  5286 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:22:54.415372  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 62062.7us
I20260709 12:22:54.419744  5272 raft_consensus.cc:359] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.420782  5272 raft_consensus.cc:385] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.423369  5336 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.423745  5269 raft_consensus.cc:359] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.424135  5336 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.424635  5269 raft_consensus.cc:385] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.421128  5272 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.427300  5282 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.428056  5282 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.429019  5336 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:54.430457  5282 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:54.430599  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:54.431649  5271 tablet_bootstrap.cc:654] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.432549  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:54.432751  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:54.433359  5076 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
W20260709 12:22:54.434559  5041 leader_election.cc:343] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.426286  5269 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.437131  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:54.437858  5149 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:22:54.438172  5270 tablet_bootstrap.cc:492] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:54.437951  5269 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.438809  5270 ts_tablet_manager.cc:1403] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.201s	user 0.019s	sys 0.016s
I20260709 12:22:54.440014  4965 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:54.441653  5269 ts_tablet_manager.cc:1434] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.090s	user 0.011s	sys 0.001s
I20260709 12:22:54.442793  5269 tablet_bootstrap.cc:492] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:54.443189  5282 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:54.449875  5282 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.450479  5282 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.445497  5272 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.454125  5000 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:54.443974  5270 raft_consensus.cc:359] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.454823  5270 raft_consensus.cc:385] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.455163  5270 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.456091  5270 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.457840  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:22:54.458657  5270 ts_tablet_manager.cc:1434] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 12:22:54.459288  5336 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:54.459702  5336 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.438128  5224 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:54.460146  5336 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.461282  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:54.462981  5286 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:54.463642  5286 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:22:54.464804  5286 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.450568  5269 tablet_bootstrap.cc:654] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.459837  5270 tablet_bootstrap.cc:492] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:54.480788  5341 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.480873  5336 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.482568  5282 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.481642  5341 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.486186  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:22:54.489312  5041 leader_election.cc:343] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.490782  5272 ts_tablet_manager.cc:1434] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.108s	user 0.016s	sys 0.007s
I20260709 12:22:54.492691  5341 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:54.491166  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:54.493093  5336 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:54.492933  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:54.493698  5076 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:54.495575  5189 leader_election.cc:343] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.492624  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:54.497648  5272 tablet_bootstrap.cc:492] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:54.498090  5149 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:54.498989  5271 tablet_bootstrap.cc:492] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:54.503252  5271 ts_tablet_manager.cc:1403] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.219s	user 0.020s	sys 0.019s
I20260709 12:22:54.500520  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:54.504298  5269 tablet_bootstrap.cc:492] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:54.503859  4896 catalog_manager.cc:5697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:54.499768  5270 tablet_bootstrap.cc:654] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.515544  5271 raft_consensus.cc:359] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.516111  5272 tablet_bootstrap.cc:654] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.507287  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:54.508857  5282 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:54.519224  5341 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:54.519661  5341 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.520062  5341 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.514063  5000 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.526007  5269 ts_tablet_manager.cc:1403] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.083s	user 0.012s	sys 0.007s
I20260709 12:22:54.510969  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:54.529922  5149 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.534303  5269 raft_consensus.cc:359] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.516471  5271 raft_consensus.cc:385] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.536230  5271 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.537482  5271 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.535621  5269 raft_consensus.cc:385] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.545905  5269 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.546979  5269 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.550292  5271 ts_tablet_manager.cc:1434] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 12:22:54.551985  5271 tablet_bootstrap.cc:492] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:54.554005  5341 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.555406  5286 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:54.556468  5341 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:54.557794  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:54.559052  5269 ts_tablet_manager.cc:1434] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.033s	user 0.003s	sys 0.006s
I20260709 12:22:54.560173  5269 tablet_bootstrap.cc:492] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:54.561995  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:22:54.562270  5189 leader_election.cc:343] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.562733  5150 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.567070  5076 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:54.572677  4965 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:54.574035  5282 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:54.574700  5282 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:22:54.575722  5271 tablet_bootstrap.cc:654] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.575665  5282 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.582136  5270 tablet_bootstrap.cc:492] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:54.582741  5270 ts_tablet_manager.cc:1403] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.123s	user 0.025s	sys 0.025s
I20260709 12:22:54.583238  5269 tablet_bootstrap.cc:654] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.585934  5149 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:54.587280  5272 tablet_bootstrap.cc:492] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:54.587037  5270 raft_consensus.cc:359] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.587821  5272 ts_tablet_manager.cc:1403] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.091s	user 0.022s	sys 0.017s
I20260709 12:22:54.587831  5270 raft_consensus.cc:385] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.588510  5270 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.591043  5000 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:54.592345  5150 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:54.592473  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:22:54.602135  5336 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:54.602658  5336 raft_consensus.cc:697] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:22:54.603474  5336 consensus_queue.cc:237] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.605660  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:54.593930  5270 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.606952  5341 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:54.607486  5341 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:54.608800  5270 ts_tablet_manager.cc:1434] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:22:54.608423  5341 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.610287  5270 tablet_bootstrap.cc:492] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:54.599457  5272 raft_consensus.cc:359] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.600664  5271 tablet_bootstrap.cc:492] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:54.615769  5272 raft_consensus.cc:385] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.616700  5272 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.618175  5272 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.616021  5271 ts_tablet_manager.cc:1403] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.064s	user 0.018s	sys 0.000s
I20260709 12:22:54.619366  5320 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:54.625065  5282 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.623967  5271 raft_consensus.cc:359] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.626653  5271 raft_consensus.cc:385] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.625633  5282 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.628940  5272 ts_tablet_manager.cc:1434] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.002s
I20260709 12:22:54.637841  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:54.630336  5271 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.641074  4896 catalog_manager.cc:5697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:54.638680  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:54.645807  5150 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:22:54.642161  5271 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:22:54.642724  4965 leader_election.cc:343] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.630537  4897 catalog_manager.cc:5697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:54.648574  5271 ts_tablet_manager.cc:1434] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.028s	user 0.003s	sys 0.004s
I20260709 12:22:54.649725  5271 tablet_bootstrap.cc:492] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:54.650931  4967 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:22:54.652206  5343 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:54.652635  5343 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.653002  5343 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.654336  5270 tablet_bootstrap.cc:654] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.643818  5272 tablet_bootstrap.cc:492] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:54.656828  5336 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.665863  5336 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.668213  5271 tablet_bootstrap.cc:654] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.672049  5272 tablet_bootstrap.cc:654] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.672524  5269 tablet_bootstrap.cc:492] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:54.673094  5269 ts_tablet_manager.cc:1403] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.113s	user 0.020s	sys 0.025s
I20260709 12:22:54.679067  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:54.679728  5336 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:54.680612  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:54.639760  5282 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:54.681358  5224 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:54.681419  5269 raft_consensus.cc:359] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.681396  4899 catalog_manager.cc:5697] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:54.684594  5269 raft_consensus.cc:385] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.684959  5269 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.685726  5269 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:22:54.683483  5042 leader_election.cc:343] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.687950  5269 ts_tablet_manager.cc:1434] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:22:54.688980  5269 tablet_bootstrap.cc:492] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:54.688771  5041 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:22:54.689621  5270 tablet_bootstrap.cc:492] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:54.690312  5270 ts_tablet_manager.cc:1403] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.080s	user 0.018s	sys 0.011s
I20260709 12:22:54.691910  5336 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:54.692413  5336 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.692806  5336 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.694063  5270 raft_consensus.cc:359] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.694286  5343 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.694975  5270 raft_consensus.cc:385] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.695369  5270 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.696384  5270 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.698767  5270 ts_tablet_manager.cc:1434] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:22:54.699640  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:54.699997  5270 tablet_bootstrap.cc:492] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
W20260709 12:22:54.703497  4965 leader_election.cc:343] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.704779  5272 tablet_bootstrap.cc:492] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:54.705428  5272 ts_tablet_manager.cc:1403] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.062s	user 0.009s	sys 0.009s
I20260709 12:22:54.705929  5343 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:54.707930  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:54.708218  5270 tablet_bootstrap.cc:654] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.708664  5150 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.708590  5272 raft_consensus.cc:359] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.712564  5354 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.713122  5354 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.715620  5354 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:54.718027  5150 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:54.719448  5271 tablet_bootstrap.cc:492] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:54.719305  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:54.719616  4967 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:22:54.720119  5224 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:22:54.710935  5336 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.721145  5343 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:54.721901  5343 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:22:54.720081  5271 ts_tablet_manager.cc:1403] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.005s
I20260709 12:22:54.721421  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:54.723131  5001 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:22:54.722978  5343 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.724717  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:54.725996  5336 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:54.709441  5272 raft_consensus.cc:385] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.726801  5272 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.728570  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:54.729422  5224 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.730876  5341 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.731623  5341 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.733858  5354 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:54.734434  5354 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.734956  5354 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.737067  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:54.748907  5224 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:54.748507  5271 raft_consensus.cc:359] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.749672  5272 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.750149  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:22:54.751571  5042 leader_election.cc:343] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.752504  5269 tablet_bootstrap.cc:654] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.753135  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:54.758001  5041 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:22:54.758388  5150 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
W20260709 12:22:54.758611  5188 leader_election.cc:343] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.759284  5271 raft_consensus.cc:385] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.759689  5271 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.753446  5354 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.754611  5341 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:54.760609  5271 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.761379  5336 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:54.761502  5272 ts_tablet_manager.cc:1434] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.008s
I20260709 12:22:54.762024  5336 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:22:54.762507  5354 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:54.763005  5336 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.766623  5271 ts_tablet_manager.cc:1434] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.044s	user 0.016s	sys 0.002s
I20260709 12:22:54.767715  5271 tablet_bootstrap.cc:492] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:54.768693  5270 tablet_bootstrap.cc:492] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:54.769286  5270 ts_tablet_manager.cc:1403] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.004s
I20260709 12:22:54.772614  5270 raft_consensus.cc:359] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.773360  5270 raft_consensus.cc:385] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.773818  5270 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.774533  5270 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.775453  5271 tablet_bootstrap.cc:654] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.775915  4899 catalog_manager.cc:5697] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:54.776911  5270 ts_tablet_manager.cc:1434] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:54.777978  5270 tablet_bootstrap.cc:492] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:54.778437  5190 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:22:54.779697  5341 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:54.780081  5341 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.780447  5341 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.784543  5270 tablet_bootstrap.cc:654] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.787850  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:54.788578  5224 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.793983  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:54.794744  5001 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.795809  5272 tablet_bootstrap.cc:492] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:54.799371  5341 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.804140  5269 tablet_bootstrap.cc:492] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:54.805125  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:54.806277  5269 ts_tablet_manager.cc:1403] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.118s	user 0.015s	sys 0.011s
I20260709 12:22:54.805358  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:54.807883  5150 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:54.808916  5188 leader_election.cc:343] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:54.816103  5272 tablet_bootstrap.cc:654] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.815502  5341 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:54.822088  5271 tablet_bootstrap.cc:492] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:54.823132  5271 ts_tablet_manager.cc:1403] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.056s	user 0.032s	sys 0.013s
I20260709 12:22:54.825510  5271 raft_consensus.cc:359] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.826323  5271 raft_consensus.cc:385] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.826743  5271 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.828251  5271 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.832242  4899 catalog_manager.cc:5697] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:54.835505  5001 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:54.836977  5271 ts_tablet_manager.cc:1434] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.004s
I20260709 12:22:54.837080  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:54.838121  5354 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:54.834331  5150 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:54.839413  5269 raft_consensus.cc:359] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.840145  5269 raft_consensus.cc:385] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.840209  5190 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:22:54.840561  5269 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.838233  5271 tablet_bootstrap.cc:492] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:54.841291  5269 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:54.842895  5341 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:54.842922  5354 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:22:54.843779  5341 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:54.844715  5341 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.844564  5354 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.846885  5224 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:54.852104  5269 ts_tablet_manager.cc:1434] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.001s
I20260709 12:22:54.853299  5269 tablet_bootstrap.cc:492] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
W20260709 12:22:54.856292  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:54.858294  5272 tablet_bootstrap.cc:492] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:54.858810  5271 tablet_bootstrap.cc:654] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.858834  5272 ts_tablet_manager.cc:1403] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.019s
I20260709 12:22:54.862324  5272 raft_consensus.cc:359] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.863049  5272 raft_consensus.cc:385] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.863320  5272 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.866022  5272 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.869504  5272 ts_tablet_manager.cc:1434] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:22:54.870575  5272 tablet_bootstrap.cc:492] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:54.871784  5269 tablet_bootstrap.cc:654] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.874465  5270 tablet_bootstrap.cc:492] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:54.875084  5270 ts_tablet_manager.cc:1403] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.097s	user 0.013s	sys 0.017s
I20260709 12:22:54.878275  5270 raft_consensus.cc:359] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.879112  5270 raft_consensus.cc:385] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.879423  5270 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.880316  5270 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.888121  5270 ts_tablet_manager.cc:1434] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:22:54.890748  5269 tablet_bootstrap.cc:492] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:54.891485  5269 ts_tablet_manager.cc:1403] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.005s
I20260709 12:22:54.892057  5270 tablet_bootstrap.cc:492] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:54.899597  5269 raft_consensus.cc:359] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.900195  5269 raft_consensus.cc:385] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.899276  4896 catalog_manager.cc:5697] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:54.900439  5269 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.894555  5271 tablet_bootstrap.cc:492] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:54.901302  5269 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.902029  5271 ts_tablet_manager.cc:1403] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.007s
I20260709 12:22:54.903815  5269 ts_tablet_manager.cc:1434] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 12:22:54.896751  4899 catalog_manager.cc:5697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:54.905179  5271 raft_consensus.cc:359] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.896183  5272 tablet_bootstrap.cc:654] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.906015  5271 raft_consensus.cc:385] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.906369  5271 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.907362  5271 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.909765  5269 tablet_bootstrap.cc:492] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:54.910117  5271 ts_tablet_manager.cc:1434] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:22:54.910245  5270 tablet_bootstrap.cc:654] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.911237  5271 tablet_bootstrap.cc:492] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:54.920523  5269 tablet_bootstrap.cc:654] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.918777  5271 tablet_bootstrap.cc:654] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.935730  5354 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.936491  5354 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.939379  5354 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:54.940564  5269 tablet_bootstrap.cc:492] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:54.940779  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:54.941325  5269 ts_tablet_manager.cc:1403] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.003s
I20260709 12:22:54.941663  5075 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:22:54.942564  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:54.943125  5114 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:54.943256  5001 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:22:54.944167  5354 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:54.944548  5354 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.944880  5354 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.949736  5343 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.950356  5343 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.952844  5343 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:54.954614  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:54.956611  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:54.957968  5150 raft_consensus.cc:2393] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 1: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:22:54.958630  5354 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.959812  4967 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:54.960597  5271 tablet_bootstrap.cc:492] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:54.960731  5343 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.961139  5271 ts_tablet_manager.cc:1403] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.050s	user 0.027s	sys 0.010s
I20260709 12:22:54.961601  5270 tablet_bootstrap.cc:492] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:54.962134  5270 ts_tablet_manager.cc:1403] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.070s	user 0.011s	sys 0.016s
I20260709 12:22:54.963126  5272 tablet_bootstrap.cc:492] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:54.963596  5272 ts_tablet_manager.cc:1403] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.093s	user 0.013s	sys 0.011s
I20260709 12:22:54.964453  5271 raft_consensus.cc:359] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.965152  5271 raft_consensus.cc:385] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.965075  5270 raft_consensus.cc:359] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.965611  5271 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.965948  5270 raft_consensus.cc:385] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.966667  5271 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.971537  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:54.971735  5343 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:54.972415  5001 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.972944  5272 raft_consensus.cc:359] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.973690  5272 raft_consensus.cc:385] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.973986  5272 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.974730  5272 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:54.976845  5271 ts_tablet_manager.cc:1434] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:22:54.977983  5271 tablet_bootstrap.cc:492] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:54.980253  5269 raft_consensus.cc:359] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.981058  5269 raft_consensus.cc:385] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:54.981374  5269 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:54.982246  5269 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:54.982632  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:54.983355  5075 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.983877  5341 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.984377  5341 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:54.985350  5272 ts_tablet_manager.cc:1434] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.002s
I20260709 12:22:54.986747  5341 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:54.987831  5269 ts_tablet_manager.cc:1434] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.001s
I20260709 12:22:54.988876  5269 tablet_bootstrap.cc:492] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:54.989305  5271 tablet_bootstrap.cc:654] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.993398  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:54.994218  5076 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:54.994266  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:54.994951  5150 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:54.995837  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:22:54.996541  5269 tablet_bootstrap.cc:654] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:54.997097  5224 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:54.997639  5338 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:54.998031  5338 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.998234  5272 tablet_bootstrap.cc:492] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:54.998387  5338 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.000864  5271 tablet_bootstrap.cc:492] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:55.001515  5271 ts_tablet_manager.cc:1403] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.000s
I20260709 12:22:55.005374  5272 tablet_bootstrap.cc:654] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.007030  5341 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:55.013419  5286 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:55.019239  5271 raft_consensus.cc:359] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.020282  5271 raft_consensus.cc:385] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.020659  5271 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.019876  5338 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.024364  5336 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:55.025811  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:55.026463  5076 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.026491  5343 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.027448  5343 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:55.029846  5343 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:55.031157  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:55.031298  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:55.031989  5271 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.034790  5271 ts_tablet_manager.cc:1434] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.002s
I20260709 12:22:55.035290  4966 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:22:55.036314  5343 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.040771  5075 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:55.041906  5354 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:55.044445  5114 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:55.045913  5320 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:55.046397  5320 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:22:55.046864  5001 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:54.966303  5270 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.047276  5320 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.049880  5270 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.053273  5271 tablet_bootstrap.cc:492] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:55.061841  5270 ts_tablet_manager.cc:1434] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.099s	user 0.008s	sys 0.000s
I20260709 12:22:55.063124  5270 tablet_bootstrap.cc:492] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:55.064790  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:55.065516  5150 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.071146  5338 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:55.079514  5269 tablet_bootstrap.cc:492] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:55.080343  5269 ts_tablet_manager.cc:1403] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.092s	user 0.024s	sys 0.029s
I20260709 12:22:55.082201  5272 tablet_bootstrap.cc:492] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:55.082904  5272 ts_tablet_manager.cc:1403] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.085s	user 0.013s	sys 0.009s
I20260709 12:22:55.085707  5271 tablet_bootstrap.cc:654] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.092880  5269 raft_consensus.cc:359] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.093779  5269 raft_consensus.cc:385] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.094197  5269 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.094983  5269 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.086897  5272 raft_consensus.cc:359] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.096486  5272 raft_consensus.cc:385] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.096889  5272 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.089905  5270 tablet_bootstrap.cc:654] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.097566  5269 ts_tablet_manager.cc:1434] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 12:22:55.099176  5269 tablet_bootstrap.cc:492] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:55.080542  5076 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:55.107218  5272 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.110584  5272 ts_tablet_manager.cc:1434] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 12:22:55.111694  5272 tablet_bootstrap.cc:492] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:55.114563  5282 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:55.119112  5272 tablet_bootstrap.cc:654] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.122452  5333 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:55.137490  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:22:55.138963  5341 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:55.139683  5341 raft_consensus.cc:697] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:55.141177  5341 consensus_queue.cc:237] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.143671  5150 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:55.146418  5320 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.147989  5320 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.151394  5270 tablet_bootstrap.cc:492] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:55.152412  5270 ts_tablet_manager.cc:1403] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.089s	user 0.024s	sys 0.018s
I20260709 12:22:55.156437  5333 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.157064  5333 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.159533  5333 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:55.173489  5320 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:55.175249  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:22:55.177150  5115 leader_election.cc:343] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.173759  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:22:55.179539  5114 leader_election.cc:343] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.180200  5114 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:22:55.181332  5320 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.220808  5270 raft_consensus.cc:359] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.221977  5270 raft_consensus.cc:385] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.222332  5270 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.192512  5343 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.058s
I20260709 12:22:55.219183  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:22:55.260864  4965 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 96075.8us
I20260709 12:22:55.254762  5333 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.264662  5382 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.265137  5269 tablet_bootstrap.cc:654] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.239878  5270 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.293907  5270 ts_tablet_manager.cc:1434] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.141s	user 0.010s	sys 0.000s
I20260709 12:22:55.295147  5270 tablet_bootstrap.cc:492] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:55.298800  5272 tablet_bootstrap.cc:492] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:55.299767  5272 ts_tablet_manager.cc:1403] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.188s	user 0.026s	sys 0.048s
I20260709 12:22:55.301895  5336 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.302596  5336 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:22:55.286803  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:55.284247  5224 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:55.316843  5391 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.317590  5391 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:55.287109  5333 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.341297  5271 tablet_bootstrap.cc:492] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:55.342157  5271 ts_tablet_manager.cc:1403] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.289s	user 0.017s	sys 0.009s
I20260709 12:22:55.310343  5382 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:55.349684  5340 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.350788  5382 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:55.352131  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:22:55.353461  5336 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:55.354630  5391 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:55.350420  5340 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:22:55.342051  5042 consensus_peers.cc:597] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:55.346524  5272 raft_consensus.cc:359] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.364662  5272 raft_consensus.cc:385] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.365003  5272 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.368378  5270 tablet_bootstrap.cc:654] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.365700  5272 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.370215  5271 raft_consensus.cc:359] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.371347  5271 raft_consensus.cc:385] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.371765  5271 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.372807  5271 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.355552  5336 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:55.381412  5333 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:55.386026  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:55.377308  5336 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.386778  5336 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.386075  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:55.393071  5340 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:55.393154  4998 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.395184  5271 ts_tablet_manager.cc:1434] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.053s	user 0.009s	sys 0.000s
I20260709 12:22:55.396723  5271 tablet_bootstrap.cc:492] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:55.397184  5272 ts_tablet_manager.cc:1434] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.002s
I20260709 12:22:55.393795  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:55.391212  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:55.400866  5272 tablet_bootstrap.cc:492] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:55.400425  4997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:55.402300  5001 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.407822  5340 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:55.409178  5075 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.398680  5333 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.420717  4966 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:55.416774  4997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:55.424013  4995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:22:55.425954  5115 leader_election.cc:343] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.428056  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:55.429018  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:55.429589  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:55.430464  5150 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 1: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:22:55.432097  5190 leader_election.cc:343] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.433285  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:22:55.434187  5042 leader_election.cc:343] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:55.434428  5189 leader_election.cc:343] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.434962  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:55.437044  5338 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.436800  5286 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:55.439775  5382 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:22:55.441449  5115 leader_election.cc:343] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.421176  5391 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:22:55.414341  5000 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:55.416970  5333 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.443764  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:55.444880  5320 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.446676  5333 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:55.447396  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:55.448585  5148 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 1: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:22:55.452741  5400 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.056s
I20260709 12:22:55.454420  5074 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.455334  5320 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.456025  5320 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.466159  5286 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:55.474284  5320 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:55.478754  5272 tablet_bootstrap.cc:654] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.478085  4896 catalog_manager.cc:5697] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:55.483112  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:55.485211  5336 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:22:55.485647  5114 leader_election.cc:343] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.487787  5336 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:55.493619  5269 tablet_bootstrap.cc:492] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:55.494127  5269 ts_tablet_manager.cc:1403] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.395s	user 0.019s	sys 0.026s
W20260709 12:22:55.490249  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 82728.4us
I20260709 12:22:55.529908  5269 raft_consensus.cc:359] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.530823  5269 raft_consensus.cc:385] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.531111  5282 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:55.531322  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:55.532208  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:55.532831  5000 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.532866  5041 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:55.533432  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:55.534160  5224 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.508323  5341 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:22:55.531132  5269 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.556079  5269 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.572633  5269 ts_tablet_manager.cc:1434] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.078s	user 0.009s	sys 0.000s
I20260709 12:22:55.573772  5269 tablet_bootstrap.cc:492] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:55.576349  5271 tablet_bootstrap.cc:654] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.578737  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:55.582149  5270 tablet_bootstrap.cc:492] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:55.582880  5270 ts_tablet_manager.cc:1403] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.288s	user 0.014s	sys 0.017s
W20260709 12:22:55.567574  5041 leader_election.cc:343] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.590083  5041 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:22:55.593420  5115 leader_election.cc:343] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.594010  5115 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:22:55.594789  5041 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:55.595005  5338 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:55.599289  5354 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.594797  5320 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:22:55.600997  5405 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.604000  5404 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.605901  5401 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.610925  5272 tablet_bootstrap.cc:492] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:55.611588  5272 ts_tablet_manager.cc:1403] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.214s	user 0.019s	sys 0.009s
I20260709 12:22:55.630095  5333 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.585183  5224 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:55.639219  5270 raft_consensus.cc:359] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.640755  5270 raft_consensus.cc:385] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.641151  5270 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.651544  5270 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.663800  5272 raft_consensus.cc:359] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.664762  5272 raft_consensus.cc:385] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.665031  5272 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.665807  5272 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.667749  5272 ts_tablet_manager.cc:1434] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.055s	user 0.003s	sys 0.003s
I20260709 12:22:55.668705  5272 tablet_bootstrap.cc:492] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:55.674659  5270 ts_tablet_manager.cc:1434] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.091s	user 0.007s	sys 0.003s
I20260709 12:22:55.677440  5270 tablet_bootstrap.cc:492] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:55.670373  5382 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.680977  5399 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:55.688982  5409 log.cc:927] Time spent T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.117s	user 0.001s	sys 0.001s
I20260709 12:22:55.676864  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:22:55.684017  5382 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.706091  5341 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.702209  5399 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.711344  5404 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:55.718607  5397 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:22:55.725826  5341 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.742341  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:22:55.744593  5188 leader_election.cc:343] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.751677  5272 tablet_bootstrap.cc:654] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.724838  5000 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:55.753566  4897 catalog_manager.cc:5697] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:55.755106  5271 tablet_bootstrap.cc:492] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:55.755640  5271 ts_tablet_manager.cc:1403] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.359s	user 0.013s	sys 0.008s
I20260709 12:22:55.758538  5399 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:55.758845  5271 raft_consensus.cc:359] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.759675  5271 raft_consensus.cc:385] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.759436  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:55.760010  5271 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.761040  5404 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:22:55.761422  5189 leader_election.cc:343] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.762517  5343 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.760854  5271 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.763768  5269 tablet_bootstrap.cc:654] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.763098  5343 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.770551  5404 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.772207  5343 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:55.773651  5341 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:55.779127  5271 ts_tablet_manager.cc:1434] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.001s
I20260709 12:22:55.780611  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:55.798650  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:55.799752  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:22:55.800239  5188 leader_election.cc:343] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.800781  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:55.802066  5150 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.804198  5394 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.798888  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:55.794776  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:22:55.815402  4967 leader_election.cc:343] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.819046  5382 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:55.828534  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:55.830507  5115 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:22:55.835363  5190 leader_election.cc:343] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.836115  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:55.838946  5397 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.823858  5072 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:22:55.854637  5394 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:55.866752  5272 tablet_bootstrap.cc:492] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:55.867635  5272 ts_tablet_manager.cc:1403] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.199s	user 0.014s	sys 0.014s
I20260709 12:22:55.871413  4965 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:22:55.872710  5401 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:55.873149  5401 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.873507  5401 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.876506  5272 raft_consensus.cc:359] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.881798  5272 raft_consensus.cc:385] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.882263  5272 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.885896  5271 tablet_bootstrap.cc:492] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:55.888873  5270 tablet_bootstrap.cc:654] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.889679  5272 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:55.901397  5269 tablet_bootstrap.cc:492] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:55.902122  5269 ts_tablet_manager.cc:1403] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.329s	user 0.026s	sys 0.003s
I20260709 12:22:55.906663  5424 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.907322  5424 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.908571  5272 ts_tablet_manager.cc:1434] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.038s	user 0.010s	sys 0.001s
I20260709 12:22:55.909996  5272 tablet_bootstrap.cc:492] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:55.918124  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:55.918866  5424 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:22:55.920097  5115 leader_election.cc:343] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.921154  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:55.923194  5382 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:22:55.925582  5114 leader_election.cc:343] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.926122  5114 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:55.927085  5382 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:55.927596  5270 tablet_bootstrap.cc:492] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:55.927358  5269 raft_consensus.cc:359] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.928076  5269 raft_consensus.cc:385] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.928385  5269 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.925935  5401 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.931350  5401 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:55.928079  5270 ts_tablet_manager.cc:1403] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.251s	user 0.010s	sys 0.009s
I20260709 12:22:55.933046  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:55.933737  5072 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:55.933931  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:55.934196  5269 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:22:55.935370  4967 leader_election.cc:343] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:55.937870  5269 ts_tablet_manager.cc:1434] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.001s
I20260709 12:22:55.938596  5270 raft_consensus.cc:359] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.939234  5269 tablet_bootstrap.cc:492] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:55.939358  5270 raft_consensus.cc:385] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.939863  5270 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.940657  5270 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.942420  5270 ts_tablet_manager.cc:1434] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:22:55.943133  5270 tablet_bootstrap.cc:492] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:55.947054  5269 tablet_bootstrap.cc:654] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.948796  5270 tablet_bootstrap.cc:654] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.933303  5271 tablet_bootstrap.cc:654] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:55.959523  5072 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:55.961390  4965 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:22:55.962538  5401 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:55.963054  5401 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:22:55.964015  5401 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:55.966395  5271 tablet_bootstrap.cc:492] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:55.967110  5271 ts_tablet_manager.cc:1403] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.003s
I20260709 12:22:55.971519  5271 raft_consensus.cc:359] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.972503  5271 raft_consensus.cc:385] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.972856  5271 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.973850  5271 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:55.978118  5270 tablet_bootstrap.cc:492] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:55.978961  5270 ts_tablet_manager.cc:1403] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.006s
I20260709 12:22:55.983218  5270 raft_consensus.cc:359] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:55.984251  5270 raft_consensus.cc:385] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:55.987246  5271 ts_tablet_manager.cc:1434] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:22:55.988868  5271 tablet_bootstrap.cc:492] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:55.984627  5270 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:55.991600  5270 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:55.994850  5270 ts_tablet_manager.cc:1434] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.001s
I20260709 12:22:55.996299  5270 tablet_bootstrap.cc:492] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:55.990518  4898 catalog_manager.cc:5697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:56.001456  5272 tablet_bootstrap.cc:654] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.009629  5271 tablet_bootstrap.cc:654] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.016929  5270 tablet_bootstrap.cc:654] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.027487  5271 tablet_bootstrap.cc:492] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:56.028033  5271 ts_tablet_manager.cc:1403] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.007s
I20260709 12:22:56.029246  5269 tablet_bootstrap.cc:492] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:56.029844  5269 ts_tablet_manager.cc:1403] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.091s	user 0.032s	sys 0.021s
I20260709 12:22:56.032127  5271 raft_consensus.cc:359] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.032855  5271 raft_consensus.cc:385] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.033094  5271 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.035935  5271 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.038440  5271 ts_tablet_manager.cc:1434] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:22:56.039713  4895 catalog_manager.cc:5697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:56.043788  5399 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.044566  5399 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.048981  5399 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:56.049866  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:22:56.052119  5189 leader_election.cc:343] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.052767  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:56.053467  5271 tablet_bootstrap.cc:492] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:56.053503  5149 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:56.056057  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:56.057286  5399 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:56.033053  5269 raft_consensus.cc:359] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.057694  5399 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.057920  5269 raft_consensus.cc:385] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.058215  5399 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.058298  5269 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.059417  5269 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.062107  5269 ts_tablet_manager.cc:1434] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.003s
I20260709 12:22:56.063796  5271 tablet_bootstrap.cc:654] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.065114  5270 tablet_bootstrap.cc:492] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:56.065765  5270 ts_tablet_manager.cc:1403] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.070s	user 0.011s	sys 0.022s
I20260709 12:22:56.068889  5270 raft_consensus.cc:359] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.069628  5270 raft_consensus.cc:385] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.069888  5270 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.070756  5269 tablet_bootstrap.cc:492] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:56.078073  5269 tablet_bootstrap.cc:654] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.057658  5272 tablet_bootstrap.cc:492] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:56.081245  5272 ts_tablet_manager.cc:1403] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.172s	user 0.025s	sys 0.000s
I20260709 12:22:56.081398  5270 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.083989  5270 ts_tablet_manager.cc:1434] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:22:56.085444  5270 tablet_bootstrap.cc:492] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:56.085147  5272 raft_consensus.cc:359] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.085949  5272 raft_consensus.cc:385] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.086252  5272 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.087100  5272 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.089775  5272 ts_tablet_manager.cc:1434] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:22:56.091411  5272 tablet_bootstrap.cc:492] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:56.093029  5270 tablet_bootstrap.cc:654] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.092903  5404 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.094213  5404 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.096534  5404 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:56.097281  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:56.098008  5224 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:56.100009  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:22:56.100220  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:56.100958  4999 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:56.101460  5405 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:56.101899  5405 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.102254  5405 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.100519  5272 tablet_bootstrap.cc:654] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.104507  5271 tablet_bootstrap.cc:492] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:56.104971  5271 ts_tablet_manager.cc:1403] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.052s	user 0.008s	sys 0.011s
I20260709 12:22:56.108168  5271 raft_consensus.cc:359] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.108821  5271 raft_consensus.cc:385] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.087634  5399 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.111940  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:56.112586  5149 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.109058  5271 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.119059  5271 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.124840  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:22:56.128489  5189 leader_election.cc:343] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.131457  5405 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.142853  5382 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.143756  5382 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.151216  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:56.152319  5269 tablet_bootstrap.cc:492] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:56.152925  5269 ts_tablet_manager.cc:1403] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.090s	user 0.009s	sys 0.004s
W20260709 12:22:56.153409  5114 leader_election.cc:343] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.155364  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:56.156088  5224 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:22:56.156162  5269 raft_consensus.cc:359] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.156889  5269 raft_consensus.cc:385] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.157195  5269 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.158221  5399 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:56.157620  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:22:56.159631  5271 ts_tablet_manager.cc:1434] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.054s	user 0.004s	sys 0.003s
I20260709 12:22:56.158655  5382 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:56.160915  5269 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.161872  5401 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.163528  5401 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.164369  5271 tablet_bootstrap.cc:492] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:56.164706  5269 ts_tablet_manager.cc:1434] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:22:56.166020  5269 tablet_bootstrap.cc:492] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:56.166729  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:56.167474  5223 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:22:56.169433  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:56.170899  4966 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:22:56.171514  4965 leader_election.cc:343] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.172209  5343 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:56.158713  5149 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:56.172669  5343 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.173137  5343 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.173906  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:56.174989  5340 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:56.175504  5340 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:56.176705  5269 tablet_bootstrap.cc:654] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.176388  5340 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.161624  5424 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:56.177554  5401 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:56.181253  5424 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.183471  5272 tablet_bootstrap.cc:492] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:56.183947  5272 ts_tablet_manager.cc:1403] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.093s	user 0.018s	sys 0.030s
I20260709 12:22:56.184557  5391 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.185847  5391 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.186331  5405 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:56.187505  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:56.188143  4999 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.188238  5391 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:56.182822  5424 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.189375  5272 raft_consensus.cc:359] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.189816  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:56.190131  5272 raft_consensus.cc:385] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.190594  5272 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.191218  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:56.191494  5272 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:22:56.191788  4967 leader_election.cc:343] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.191859  5223 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.196218  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:56.196969  5072 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:22:56.198601  4965 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:22:56.200246  5401 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:56.200649  5401 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.201027  5401 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.204146  4999 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:56.205677  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:56.206459  5272 ts_tablet_manager.cc:1434] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:22:56.206692  5405 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:56.207283  5405 raft_consensus.cc:697] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:22:56.207623  5272 tablet_bootstrap.cc:492] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:56.208159  5405 consensus_queue.cc:237] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.210081  5223 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:56.216044  5270 tablet_bootstrap.cc:492] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:56.216632  5270 ts_tablet_manager.cc:1403] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.131s	user 0.024s	sys 0.018s
I20260709 12:22:56.216665  5272 tablet_bootstrap.cc:654] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.217013  5424 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.220247  5270 raft_consensus.cc:359] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.221439  5270 raft_consensus.cc:385] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.221832  5270 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.221484  5271 tablet_bootstrap.cc:654] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.222146  5343 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.226019  5270 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.229754  5270 ts_tablet_manager.cc:1434] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.005s
I20260709 12:22:56.230892  5270 tablet_bootstrap.cc:492] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:56.234433  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:22:56.237103  5114 leader_election.cc:343] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.232966  5272 tablet_bootstrap.cc:492] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:56.237399  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:56.238276  5272 ts_tablet_manager.cc:1403] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.031s	user 0.006s	sys 0.012s
I20260709 12:22:56.238471  5224 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.239104  5271 tablet_bootstrap.cc:492] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:56.239622  5271 ts_tablet_manager.cc:1403] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.076s	user 0.007s	sys 0.005s
I20260709 12:22:56.241788  5272 raft_consensus.cc:359] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.242625  5272 raft_consensus.cc:385] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.243245  5272 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.244051  5272 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.234957  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:56.245641  5223 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.245797  5424 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:56.248487  5343 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:56.250933  5272 ts_tablet_manager.cc:1434] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:22:56.251751  5401 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.252099  4896 catalog_manager.cc:5697] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:56.254150  5401 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:56.256094  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:56.257275  5269 tablet_bootstrap.cc:492] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:56.257856  5269 ts_tablet_manager.cc:1403] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.092s	user 0.010s	sys 0.002s
I20260709 12:22:56.258436  5223 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:56.259785  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:22:56.260002  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:22:56.261030  4965 leader_election.cc:343] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.261148  5424 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:22:56.261570  4967 leader_election.cc:343] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.262152  5424 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:22:56.263414  5424 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.264671  5333 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:56.266970  5224 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:56.267035  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:56.268241  5072 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.268607  5401 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.269184  5401 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.272812  5401 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:56.272720  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:56.272720  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:56.273586  5075 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:22:56.273869  5149 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:22:56.275089  4965 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:56.275365  4966 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:22:56.276214  5391 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:56.276643  5391 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.276979  5391 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.279188  5343 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:56.252136  5272 tablet_bootstrap.cc:492] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:56.285692  5391 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.288607  5272 tablet_bootstrap.cc:654] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.289564  5401 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.291040  5401 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.295468  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:56.296500  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:56.297135  5075 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:56.297412  4967 leader_election.cc:343] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.298735  5271 raft_consensus.cc:359] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.299455  5271 raft_consensus.cc:385] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.299736  5271 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.300446  5271 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.304510  5270 tablet_bootstrap.cc:654] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.299026  5269 raft_consensus.cc:359] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.308602  5269 raft_consensus.cc:385] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.309092  5269 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.309389  5354 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.310863  5354 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.315379  5354 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:56.307888  5271 ts_tablet_manager.cc:1434] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.000s
I20260709 12:22:56.317206  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:56.318547  5073 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 1: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:22:56.294941  5072 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:56.321848  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:56.324921  5270 tablet_bootstrap.cc:492] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:56.325440  5270 ts_tablet_manager.cc:1403] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.095s	user 0.010s	sys 0.000s
I20260709 12:22:56.325575  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:56.326886  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:56.327488  5149 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.332088  5405 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:22:56.335042  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:56.337498  5269 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.340082  5269 ts_tablet_manager.cc:1434] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.082s	user 0.002s	sys 0.004s
I20260709 12:22:56.341285  5269 tablet_bootstrap.cc:492] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:56.337556  5272 tablet_bootstrap.cc:492] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:56.342329  5272 ts_tablet_manager.cc:1403] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.090s	user 0.013s	sys 0.000s
I20260709 12:22:56.345404  5272 raft_consensus.cc:359] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.328610  5270 raft_consensus.cc:359] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.335366  5354 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.346210  5149 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:56.347688  4967 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:56.348304  5343 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:22:56.335996  4965 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:22:56.348981  5391 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:56.346225  5272 raft_consensus.cc:385] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.349222  5454 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:56.349469  5272 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.350318  5453 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:56.350373  5343 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.350553  5272 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.351512  5270 raft_consensus.cc:385] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.351893  5270 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.318549  5271 tablet_bootstrap.cc:492] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:56.297729  5401 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:56.353596  5075 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:56.324131  4896 catalog_manager.cc:5697] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:56.310037  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:56.370201  5354 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:56.363894  4898 catalog_manager.cc:5697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:56.369197  5269 tablet_bootstrap.cc:654] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.373750  5270 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.377408  5271 tablet_bootstrap.cc:654] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.367172  5224 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:22:56.375567  5399 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.390990  5399 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.393821  5270 ts_tablet_manager.cc:1434] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.001s
I20260709 12:22:56.396108  5270 tablet_bootstrap.cc:492] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:56.404309  4966 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:22:56.408120  5401 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:56.414420  5341 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.415042  5341 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.408586  5401 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.399860  5453 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:22:56.419086  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:56.423398  5401 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.421288  5454 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:22:56.443466  5272 ts_tablet_manager.cc:1434] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.101s	user 0.034s	sys 0.035s
I20260709 12:22:56.444612  5272 tablet_bootstrap.cc:492] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:56.447938  5341 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:56.448770  5399 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:56.450217  5454 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.455433  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:56.456223  5149 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:56.444217  5075 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:56.457398  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:56.461817  5150 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:56.463167  5190 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:56.464201  5190 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:56.465073  5341 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:56.465461  5341 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.465591  5399 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:56.465893  5341 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.466229  5399 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.465790  5401 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.466693  5399 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.470429  5271 tablet_bootstrap.cc:492] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:56.471071  5271 ts_tablet_manager.cc:1403] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.153s	user 0.015s	sys 0.014s
I20260709 12:22:56.473645  5459 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:56.474192  5271 raft_consensus.cc:359] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.475003  5271 raft_consensus.cc:385] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.475306  5271 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.476543  5271 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.502174  5271 ts_tablet_manager.cc:1434] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 12:22:56.494113  5424 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.506706  5271 tablet_bootstrap.cc:492] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:56.509153  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:56.509871  5150 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.505650  5424 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.514431  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:56.519023  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:56.519712  5223 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:22:56.520850  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:56.521719  5354 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:56.522073  5354 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.522245  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:56.522466  5354 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.522955  5223 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.526053  5424 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:56.505905  5401 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:56.498226  5453 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:22:56.537156  5188 leader_election.cc:343] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.538965  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:56.541350  5401 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:22:56.549958  5115 leader_election.cc:343] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.546440  5341 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.558780  5269 tablet_bootstrap.cc:492] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:56.559420  5269 ts_tablet_manager.cc:1403] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.218s	user 0.007s	sys 0.010s
I20260709 12:22:56.562578  5269 raft_consensus.cc:359] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.563350  5269 raft_consensus.cc:385] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.563524  5272 tablet_bootstrap.cc:654] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.563645  5269 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.564751  5269 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.569123  5354 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.570434  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:56.571288  5148 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.571529  5354 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:56.572516  5271 tablet_bootstrap.cc:654] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.572749  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:56.573424  5224 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.574406  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:56.575399  5223 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
W20260709 12:22:56.575986  5115 leader_election.cc:343] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.576689  4966 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:56.577948  5401 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:56.578639  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:56.578806  5401 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:22:56.579312  5072 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.579774  5401 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.580996  5150 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:56.588085  5000 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.592586  5269 ts_tablet_manager.cc:1434] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 12:22:56.562364  5399 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.591954  5270 tablet_bootstrap.cc:654] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.593973  5269 tablet_bootstrap.cc:492] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:56.605221  5399 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:56.608357  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:56.609167  5075 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.598415  5341 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:56.608438  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:56.624221  5461 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.636518  5461 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.626315  5341 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:22:56.642774  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:56.643553  5147 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:56.642927  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:56.645687  4967 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:56.627259  5148 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:56.647267  5190 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:56.633158  5272 tablet_bootstrap.cc:492] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:56.648171  5341 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:56.648545  5272 ts_tablet_manager.cc:1403] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.204s	user 0.012s	sys 0.003s
I20260709 12:22:56.648725  5341 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:56.649757  5341 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.630056  5224 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:56.628650  5271 tablet_bootstrap.cc:492] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:56.651885  5271 ts_tablet_manager.cc:1403] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.006s
I20260709 12:22:56.652554  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:56.653522  5459 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:56.653728  5424 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:56.654007  5459 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.628021  5150 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.654384  5424 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:22:56.654524  5459 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:56.655406  5424 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.657264  5271 raft_consensus.cc:359] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.659096  5271 raft_consensus.cc:385] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.659482  5271 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.635527  5075 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:56.630056  5072 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:56.661315  5188 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:22:56.662318  5399 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:56.662809  5399 raft_consensus.cc:697] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:56.663738  5399 consensus_queue.cc:237] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.645691  5461 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:56.658368  5272 raft_consensus.cc:359] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.670056  5272 raft_consensus.cc:385] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.672039  5272 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.616039  5269 tablet_bootstrap.cc:654] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.674374  5271 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.678864  5354 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:56.688835  5272 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.698727  5465 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:56.699695  5270 tablet_bootstrap.cc:492] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:56.700326  5270 ts_tablet_manager.cc:1403] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.305s	user 0.011s	sys 0.011s
I20260709 12:22:56.709200  5271 ts_tablet_manager.cc:1434] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 12:22:56.715042  5272 ts_tablet_manager.cc:1434] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 12:22:56.716722  5272 tablet_bootstrap.cc:492] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:56.719655  4895 catalog_manager.cc:5697] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:56.715178  5150 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:56.713322  5459 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.724187  5270 raft_consensus.cc:359] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.734709  5270 raft_consensus.cc:385] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.735168  5270 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.737423  5000 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:22:56.738770  5459 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:56.736287  5270 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.742254  5271 tablet_bootstrap.cc:492] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:56.749286  5270 ts_tablet_manager.cc:1434] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.003s
I20260709 12:22:56.750466  5270 tablet_bootstrap.cc:492] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:56.751727  5405 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.752365  5405 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.756923  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:56.757040  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:56.759004  5270 tablet_bootstrap.cc:654] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.757161  5271 tablet_bootstrap.cc:654] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.748140  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:56.755203  5405 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:56.765981  5480 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.777908  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:56.780386  5461 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.780939  5461 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.783200  5461 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:56.778730  4895 catalog_manager.cc:5697] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:56.784569  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:56.785567  5221 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 1: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:22:56.784711  5269 tablet_bootstrap.cc:492] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:56.779604  5480 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:56.786517  5269 ts_tablet_manager.cc:1403] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.193s	user 0.017s	sys 0.013s
I20260709 12:22:56.787087  5465 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:56.788043  5465 raft_consensus.cc:3055] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:56.788400  5465 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:56.784688  4895 catalog_manager.cc:5697] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:56.790094  5269 raft_consensus.cc:359] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.789865  5480 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:56.791177  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:56.789474  5465 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.792084  5072 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 1: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:22:56.790850  5269 raft_consensus.cc:385] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.794240  5269 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.797787  5269 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.797979  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:56.800366  5269 ts_tablet_manager.cc:1434] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.003s
I20260709 12:22:56.793735  5075 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.802409  5041 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:56.807374  5269 tablet_bootstrap.cc:492] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:56.794056  5465 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:56.808569  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:56.793744  4967 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:56.829020  5461 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:56.834015  5404 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:56.836280  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:56.837332  5480 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.834084  5272 tablet_bootstrap.cc:654] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:56.840077  5479 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.128s	user 0.000s	sys 0.006s
W20260709 12:22:56.843676  4967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 50032.2us
I20260709 12:22:56.846100  4967 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:56.850584  5459 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.852919  5401 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:56.853605  5269 tablet_bootstrap.cc:654] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.794950  4895 catalog_manager.cc:5697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:56.865746  5480 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:56.875424  5405 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:56.877947  5271 tablet_bootstrap.cc:492] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:56.878203  5270 tablet_bootstrap.cc:492] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:56.878525  5271 ts_tablet_manager.cc:1403] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.168s	user 0.016s	sys 0.015s
I20260709 12:22:56.878749  5270 ts_tablet_manager.cc:1403] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.129s	user 0.023s	sys 0.020s
I20260709 12:22:56.881409  5459 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:56.882113  5382 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:56.885291  5271 raft_consensus.cc:359] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.886173  5271 raft_consensus.cc:385] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.886482  5271 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.887279  5271 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:22:56.893502  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 58625.9us
I20260709 12:22:56.895851  5271 ts_tablet_manager.cc:1434] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:22:56.897121  5271 tablet_bootstrap.cc:492] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:56.898118  5270 raft_consensus.cc:359] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.899171  5270 raft_consensus.cc:385] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.899500  5270 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.899861  5404 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:56.912439  5271 tablet_bootstrap.cc:654] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.921592  5272 tablet_bootstrap.cc:492] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:56.922189  5272 ts_tablet_manager.cc:1403] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.206s	user 0.018s	sys 0.002s
I20260709 12:22:56.926363  5270 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:56.928267  5072 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.932271  5271 tablet_bootstrap.cc:492] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:56.932315  5343 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:56.933080  5271 ts_tablet_manager.cc:1403] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.002s
I20260709 12:22:56.936218  5271 raft_consensus.cc:359] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.937021  5271 raft_consensus.cc:385] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.937307  5271 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.938125  5271 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:56.925905  5480 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.938526  4898 catalog_manager.cc:5697] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:56.939347  5480 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.940279  5343 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:56.940608  5271 ts_tablet_manager.cc:1434] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:22:56.940760  4898 catalog_manager.cc:5697] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:56.941808  5271 tablet_bootstrap.cc:492] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:56.944684  5480 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:56.945473  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:56.946959  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:56.948102  5223 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:22:56.949939  5271 tablet_bootstrap.cc:654] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:56.950030  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:56.951114  5480 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:22:56.951272  5115 leader_election.cc:343] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:56.951916  5480 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.951083  5343 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:56.952427  5480 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.933684  5269 tablet_bootstrap.cc:492] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:56.962126  5269 ts_tablet_manager.cc:1403] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.161s	user 0.032s	sys 0.000s
I20260709 12:22:56.973451  5480 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.989187  5272 raft_consensus.cc:359] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:56.990101  5272 raft_consensus.cc:385] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:56.990411  5272 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:56.992189  5454 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:56.994097  4899 catalog_manager.cc:5697] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:56.995721  5495 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:22:57.003124  5270 ts_tablet_manager.cc:1434] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.124s	user 0.005s	sys 0.004s
I20260709 12:22:57.003216  5454 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:57.004447  5270 tablet_bootstrap.cc:492] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:57.005380  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:57.010921  5454 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:57.011010  5221 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:56.991321  5272 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.015621  5272 ts_tablet_manager.cc:1434] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.000s
I20260709 12:22:57.016671  5272 tablet_bootstrap.cc:492] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:57.023236  5272 tablet_bootstrap.cc:654] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.026058  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:57.026825  5000 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.036921  5480 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:57.038160  5270 tablet_bootstrap.cc:654] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.039613  5221 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:57.011263  5269 raft_consensus.cc:359] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.043867  5269 raft_consensus.cc:385] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.051863  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:57.044911  5269 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.052996  5272 tablet_bootstrap.cc:492] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:57.054379  5272 ts_tablet_manager.cc:1403] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.009s
I20260709 12:22:57.054903  5382 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:57.054127  5269 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.057619  5272 raft_consensus.cc:359] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.058378  5272 raft_consensus.cc:385] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.058688  5272 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.059665  5269 ts_tablet_manager.cc:1434] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.097s	user 0.008s	sys 0.000s
I20260709 12:22:57.060811  5269 tablet_bootstrap.cc:492] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:57.055624  5382 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:22:57.061406  5270 tablet_bootstrap.cc:492] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:57.061949  5270 ts_tablet_manager.cc:1403] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.003s
I20260709 12:22:57.061882  5382 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.069496  5269 tablet_bootstrap.cc:654] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.073823  5270 raft_consensus.cc:359] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.074705  5270 raft_consensus.cc:385] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.075018  5270 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.075745  5270 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.078249  5272 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.082386  5272 ts_tablet_manager.cc:1434] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.002s
I20260709 12:22:57.083521  5272 tablet_bootstrap.cc:492] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:57.084424  5465 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:57.088361  5465 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:57.079069  5270 ts_tablet_manager.cc:1434] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:22:57.091727  5272 tablet_bootstrap.cc:654] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.092337  5270 tablet_bootstrap.cc:492] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:57.095474  5424 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.096012  5424 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.098299  5424 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:57.101364  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:22:57.103211  5114 leader_election.cc:343] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:57.103430  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:57.104206  5221 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:22:57.105458  5115 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:22:57.107265  5467 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:57.119166  5465 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:57.141830  5000 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:57.143244  5271 tablet_bootstrap.cc:492] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:57.144014  5271 ts_tablet_manager.cc:1403] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.202s	user 0.040s	sys 0.034s
I20260709 12:22:57.133061  5454 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:22:57.146704  5270 tablet_bootstrap.cc:654] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.125459  5424 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:57.149700  5424 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.150094  5424 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.169443  5271 raft_consensus.cc:359] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.170509  5271 raft_consensus.cc:385] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.170822  5271 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.204828  5271 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.222396  5272 tablet_bootstrap.cc:492] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:57.223152  5272 ts_tablet_manager.cc:1403] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.140s	user 0.014s	sys 0.028s
I20260709 12:22:57.227278  5272 raft_consensus.cc:359] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.228116  5272 raft_consensus.cc:385] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.228446  5272 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.216122  4898 catalog_manager.cc:5697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.230859  5272 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.246232  5269 tablet_bootstrap.cc:492] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:57.246948  5269 ts_tablet_manager.cc:1403] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.186s	user 0.011s	sys 0.018s
I20260709 12:22:57.183848  5424 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.250209  5269 raft_consensus.cc:359] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.251106  5269 raft_consensus.cc:385] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.250697  5354 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:57.251425  5269 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.252270  5269 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.254799  5269 ts_tablet_manager.cc:1434] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:22:57.255857  5269 tablet_bootstrap.cc:492] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:57.256729  5424 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:57.255985  5271 ts_tablet_manager.cc:1434] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.112s	user 0.008s	sys 0.002s
I20260709 12:22:57.259927  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:57.260704  5221 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.261010  5480 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.261732  5480 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.264320  5480 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:57.265838  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:57.277783  5271 tablet_bootstrap.cc:492] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:57.278249  5075 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.267423  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:57.275254  5272 ts_tablet_manager.cc:1434] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.001s
I20260709 12:22:57.282519  5272 tablet_bootstrap.cc:492] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:57.275527  5341 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.284984  5341 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.288691  5341 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:57.289588  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:57.290267  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:57.290407  4999 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:57.291102  5073 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
W20260709 12:22:57.291646  5115 leader_election.cc:343] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:57.292244  5272 tablet_bootstrap.cc:654] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.291981  5189 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:57.268055  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:22:57.297024  5115 leader_election.cc:343] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:57.297636  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:57.298623  5382 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:57.300347  5480 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:22:57.303551  5341 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:57.309725  5341 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.309983  5269 tablet_bootstrap.cc:654] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.310328  5341 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.318737  5270 tablet_bootstrap.cc:492] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:57.319376  5270 ts_tablet_manager.cc:1403] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.227s	user 0.023s	sys 0.015s
I20260709 12:22:57.323212  5270 raft_consensus.cc:359] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.324155  5270 raft_consensus.cc:385] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.324496  5270 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.332721  5269 tablet_bootstrap.cc:492] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:57.333508  5269 ts_tablet_manager.cc:1403] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.078s	user 0.017s	sys 0.021s
I20260709 12:22:57.336773  5269 raft_consensus.cc:359] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.337513  5269 raft_consensus.cc:385] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.337810  5269 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.332651  5515 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.338528  5269 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.338629  5515 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.341871  5221 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:57.342480  5515 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:57.344069  5269 ts_tablet_manager.cc:1434] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.006s
I20260709 12:22:57.345098  5269 tablet_bootstrap.cc:492] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:57.348094  5115 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:57.349870  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:57.351724  5269 tablet_bootstrap.cc:654] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.352388  5515 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:57.353072  5515 raft_consensus.cc:697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:22:57.354068  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:57.354024  5515 consensus_queue.cc:237] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.355175  5075 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:22:57.356594  5271 tablet_bootstrap.cc:654] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:57.358364  5114 leader_election.cc:343] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:57.359122  5115 leader_election.cc:343] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:57.359683  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:22:57.360726  5480 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:57.328660  5270 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.359251  5341 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.365374  5467 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.378221  5467 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.391682  5341 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:57.396238  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:57.397229  4999 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.405814  5467 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:57.407415  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:57.410192  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:22:57.410498  5188 leader_election.cc:343] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:57.411778  5189 leader_election.cc:343] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:57.412271  5189 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:57.413892  5341 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:57.421294  4999 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:57.421993  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:57.431852  5269 tablet_bootstrap.cc:492] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:57.432461  5269 ts_tablet_manager.cc:1403] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.088s	user 0.027s	sys 0.007s
I20260709 12:22:57.441985  5189 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:57.442364  5270 ts_tablet_manager.cc:1434] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.123s	user 0.012s	sys 0.002s
I20260709 12:22:57.443416  5270 tablet_bootstrap.cc:492] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:57.447752  5399 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:57.448522  5399 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:57.449610  5399 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.432969  5075 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.451000  5269 raft_consensus.cc:359] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.460397  5269 raft_consensus.cc:385] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.460693  5269 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.463690  5270 tablet_bootstrap.cc:654] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:57.466965  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:57.481179  5269 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.482412  5272 tablet_bootstrap.cc:492] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:57.484135  5272 ts_tablet_manager.cc:1403] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.202s	user 0.017s	sys 0.035s
I20260709 12:22:57.487363  4898 catalog_manager.cc:5697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.504648  5269 ts_tablet_manager.cc:1434] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 12:22:57.506841  5269 tablet_bootstrap.cc:492] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:57.525569  5075 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:57.532235  5271 tablet_bootstrap.cc:492] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:57.532876  5271 ts_tablet_manager.cc:1403] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.255s	user 0.013s	sys 0.009s
I20260709 12:22:57.536989  5270 tablet_bootstrap.cc:492] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:57.537647  5270 ts_tablet_manager.cc:1403] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.094s	user 0.039s	sys 0.029s
I20260709 12:22:57.540220  5271 raft_consensus.cc:359] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.541281  5272 raft_consensus.cc:359] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.541801  5271 raft_consensus.cc:385] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.541352  5269 tablet_bootstrap.cc:654] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.541702  5480 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:57.542449  5271 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.543416  5271 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.540650  5270 raft_consensus.cc:359] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.545967  5271 ts_tablet_manager.cc:1434] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.005s
I20260709 12:22:57.546268  5270 raft_consensus.cc:385] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.546650  5270 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.547466  5271 tablet_bootstrap.cc:492] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:57.547375  5270 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.549697  5270 ts_tablet_manager.cc:1434] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:22:57.550457  5272 raft_consensus.cc:385] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.550731  5270 tablet_bootstrap.cc:492] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:57.550812  5272 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.551913  5272 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.554987  5272 ts_tablet_manager.cc:1434] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.000s
I20260709 12:22:57.556058  5272 tablet_bootstrap.cc:492] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:57.557964  5271 tablet_bootstrap.cc:654] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.560665  5270 tablet_bootstrap.cc:654] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.563689  5269 tablet_bootstrap.cc:492] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:57.564355  5269 ts_tablet_manager.cc:1403] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.002s
I20260709 12:22:57.567767  5272 tablet_bootstrap.cc:654] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.567529  5269 raft_consensus.cc:359] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.568511  5269 raft_consensus.cc:385] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.568960  5269 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.570240  5269 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.572994  5270 tablet_bootstrap.cc:492] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:57.573562  5269 ts_tablet_manager.cc:1434] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:22:57.573421  5461 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.573594  5270 ts_tablet_manager.cc:1403] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.012s
I20260709 12:22:57.574833  5269 tablet_bootstrap.cc:492] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:57.574895  5461 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.578191  5270 raft_consensus.cc:359] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.578924  5270 raft_consensus.cc:385] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.579209  5270 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.579942  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:57.580067  5270 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.581295  5354 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:22:57.581811  4966 leader_election.cc:343] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:57.582541  5270 ts_tablet_manager.cc:1434] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:22:57.583436  5461 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:57.583873  5270 tablet_bootstrap.cc:492] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:57.584571  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:57.589033  5269 tablet_bootstrap.cc:654] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:57.589862  4967 leader_election.cc:343] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:57.590466  4967 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:57.591377  5270 tablet_bootstrap.cc:654] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.598991  5401 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:57.619990  5404 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.620880  5404 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.626461  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:57.627601  4999 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:57.629976  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:57.631031  5489 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:57.631459  5489 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.631822  5489 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.640132  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:22:57.646409  5042 leader_election.cc:343] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:57.641323  5404 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:57.645694  4899 catalog_manager.cc:5697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.670003  5269 tablet_bootstrap.cc:492] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:57.672431  5272 tablet_bootstrap.cc:492] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:57.672900  5269 ts_tablet_manager.cc:1403] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.096s	user 0.014s	sys 0.000s
I20260709 12:22:57.673446  5272 ts_tablet_manager.cc:1403] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.118s	user 0.014s	sys 0.005s
I20260709 12:22:57.655627  5489 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.677275  5272 raft_consensus.cc:359] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.690963  5272 raft_consensus.cc:385] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.691354  5272 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.692164  5272 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.693950  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:57.683139  5269 raft_consensus.cc:359] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.684840  5270 tablet_bootstrap.cc:492] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:57.694828  4999 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.695060  5269 raft_consensus.cc:385] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.685527  5271 tablet_bootstrap.cc:492] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:57.695391  5270 ts_tablet_manager.cc:1403] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.112s	user 0.016s	sys 0.004s
I20260709 12:22:57.695806  5271 ts_tablet_manager.cc:1403] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.149s	user 0.035s	sys 0.047s
I20260709 12:22:57.695456  5269 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.698838  5270 raft_consensus.cc:359] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.698881  5269 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.699753  5270 raft_consensus.cc:385] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.700081  5270 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.701052  5270 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.701783  5269 ts_tablet_manager.cc:1434] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:22:57.702913  5269 tablet_bootstrap.cc:492] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:57.703277  5270 ts_tablet_manager.cc:1434] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:57.704303  5270 tablet_bootstrap.cc:492] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:57.708525  4999 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:57.709887  5272 ts_tablet_manager.cc:1434] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:22:57.711756  5270 tablet_bootstrap.cc:654] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.711647  5271 raft_consensus.cc:359] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.715130  5271 raft_consensus.cc:385] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.715423  5272 tablet_bootstrap.cc:492] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:57.715925  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:57.715435  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:57.715449  5271 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.725335  5270 tablet_bootstrap.cc:492] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
W20260709 12:22:57.725818  5042 leader_election.cc:343] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:57.713615  5489 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:57.725960  5270 ts_tablet_manager.cc:1403] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.003s
I20260709 12:22:57.725649  5271 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.726609  5404 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:57.727227  5404 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:22:57.728535  5271 ts_tablet_manager.cc:1434] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:22:57.728230  5404 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.730330  5271 tablet_bootstrap.cc:492] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:57.712625  5269 tablet_bootstrap.cc:654] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.737859  5272 tablet_bootstrap.cc:654] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.743144  5489 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.743783  5489 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.746235  5489 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:57.747403  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:22:57.749122  5041 leader_election.cc:343] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:57.749584  5269 tablet_bootstrap.cc:492] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:57.750759  5271 tablet_bootstrap.cc:654] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.748927  5270 raft_consensus.cc:359] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.752490  5269 ts_tablet_manager.cc:1403] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.000s
I20260709 12:22:57.752712  5270 raft_consensus.cc:385] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.753649  5270 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.755815  5269 raft_consensus.cc:359] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.756381  5270 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.761420  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:57.762293  5150 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:57.763872  5270 ts_tablet_manager.cc:1434] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:22:57.765277  5270 tablet_bootstrap.cc:492] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:57.766953  5269 raft_consensus.cc:385] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.769955  5269 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.772626  5271 tablet_bootstrap.cc:492] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:57.770891  5269 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.773492  5271 ts_tablet_manager.cc:1403] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.002s
I20260709 12:22:57.775338  5269 ts_tablet_manager.cc:1434] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
I20260709 12:22:57.776470  5269 tablet_bootstrap.cc:492] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:57.777987  5271 raft_consensus.cc:359] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.778936  5271 raft_consensus.cc:385] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.779207  5271 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.779907  5271 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.781711  5042 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:22:57.783034  5489 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:57.783434  5489 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.783795  5489 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.794775  5269 tablet_bootstrap.cc:654] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.802431  5270 tablet_bootstrap.cc:654] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.796927  5489 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.806751  5271 ts_tablet_manager.cc:1434] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.032s	user 0.003s	sys 0.003s
I20260709 12:22:57.808625  5271 tablet_bootstrap.cc:492] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:57.807574  5489 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:57.823648  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:57.825944  5150 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.830263  4899 catalog_manager.cc:5697] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.842468  5150 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:57.861492  5042 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:57.861961  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:57.862545  5404 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:57.862726  5221 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.863129  5404 raft_consensus.cc:697] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:22:57.853818  5489 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.864177  5404 consensus_queue.cc:237] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:57.864355  5489 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.869028  5272 tablet_bootstrap.cc:492] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:57.869666  5272 ts_tablet_manager.cc:1403] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.155s	user 0.012s	sys 0.005s
I20260709 12:22:57.869644  5399 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.872998  5399 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.875202  5272 raft_consensus.cc:359] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.876313  5270 tablet_bootstrap.cc:492] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:57.876473  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:57.875991  5272 raft_consensus.cc:385] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.877137  5270 ts_tablet_manager.cc:1403] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.112s	user 0.011s	sys 0.005s
I20260709 12:22:57.877502  5272 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.878391  5272 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.879026  5399 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:57.879272  5001 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:57.880887  5270 raft_consensus.cc:359] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.881747  5270 raft_consensus.cc:385] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.882105  5270 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.883037  5270 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.884578  5272 ts_tablet_manager.cc:1434] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:22:57.885730  5272 tablet_bootstrap.cc:492] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:57.887634  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:57.888410  5270 ts_tablet_manager.cc:1434] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:22:57.891618  5341 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.892145  5341 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.894687  5341 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:57.895532  5401 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.896309  5401 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:57.897965  5341 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:57.898155  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:57.898442  5341 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.868662  5489 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:57.900607  5270 tablet_bootstrap.cc:492] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:57.901973  5271 tablet_bootstrap.cc:654] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.898883  5341 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.910554  5269 tablet_bootstrap.cc:492] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:57.912312  5269 ts_tablet_manager.cc:1403] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.136s	user 0.031s	sys 0.010s
I20260709 12:22:57.902830  5001 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:57.917932  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:57.917973  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:57.919543  5075 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:22:57.921439  5539 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.922133  5539 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:57.922554  4965 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:57.923216  5354 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.923853  5461 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:57.923763  5354 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:57.924317  5461 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.924839  5539 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:57.926321  5354 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:57.927340  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:57.928079  5224 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:22:57.928434  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:57.929395  5272 tablet_bootstrap.cc:654] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.924742  5461 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.930500  5401 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:57.944161  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:57.944327  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:57.945608  5399 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:57.940075  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:57.946028  5399 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.946460  5399 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:57.947072  5190 leader_election.cc:343] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:57.947415  5115 leader_election.cc:343] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:57.936939  5042 leader_election.cc:343] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:57.950729  5269 raft_consensus.cc:359] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:58.007277  5269 raft_consensus.cc:385] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.007736  5269 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.009060  5221 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:58.010317  5270 tablet_bootstrap.cc:654] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.015345  5467 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.018050  5341 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:58.008822  5269 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:58.018590  5461 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:58.022891  5272 tablet_bootstrap.cc:492] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:58.030139  5270 tablet_bootstrap.cc:492] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:58.030543  5272 ts_tablet_manager.cc:1403] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.145s	user 0.014s	sys 0.001s
I20260709 12:22:58.030716  5270 ts_tablet_manager.cc:1403] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.140s	user 0.014s	sys 0.001s
I20260709 12:22:58.019747  5544 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.031697  5341 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:58.031610  5544 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.032888  5461 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:58.024232  5467 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.035318  5467 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:58.037242  5272 raft_consensus.cc:359] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:58.038096  5544 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:58.038417  5272 raft_consensus.cc:385] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:22:58.085345  4965 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 133428us
I20260709 12:22:58.085266  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:58.099385  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:58.247751  4999 raft_consensus.cc:2393] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 1: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:22:58.104837  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:22:58.108911  5188 leader_election.cc:343] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:58.250716  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 142178us
I20260709 12:22:58.117422  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:58.123421  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:58.254166  5150 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:58.028966  5269 ts_tablet_manager.cc:1434] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.116s	user 0.008s	sys 0.000s
I20260709 12:22:58.256325  5190 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:58.258119  5269 tablet_bootstrap.cc:492] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:58.023627  5271 tablet_bootstrap.cc:492] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:58.262774  5271 ts_tablet_manager.cc:1403] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.454s	user 0.012s	sys 0.010s
I20260709 12:22:58.024024  5399 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.268662  5399 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:58.145031  5545 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.269804  5545 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:58.272523  5545 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:58.153868  5542 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.273489  5542 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:58.277062  5542 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:58.170257  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:22:58.195559  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 243055us
I20260709 12:22:58.303162  5222 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:58.137097  5270 raft_consensus.cc:359] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.304836  5270 raft_consensus.cc:385] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.305241  5270 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.306152  5270 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.306730  5341 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:58.308458  5270 ts_tablet_manager.cc:1434] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.277s	user 0.011s	sys 0.000s
I20260709 12:22:58.200843  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:22:58.309376  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 53444.3us
I20260709 12:22:58.310011  5545 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:58.310426  5545 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.310777  5545 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.309636  4897 catalog_manager.cc:5697] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:58.311817  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 59043.7us
I20260709 12:22:58.219935  5515 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:22:58.319021  5515 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.319612  5270 tablet_bootstrap.cc:492] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:58.157856  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:58.321120  5146 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
W20260709 12:22:58.293573  5115 leader_election.cc:343] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.322393  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:58.323628  5424 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:58.326525  5271 raft_consensus.cc:359] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.327023  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:58.327557  5271 raft_consensus.cc:385] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.327893  5271 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.326555  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:58.329071  5150 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:58.328733  5271 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.216626  5382 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:58.309392  5223 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.375092  5515 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:58.376617  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:58.390280  5271 ts_tablet_manager.cc:1434] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.127s	user 0.011s	sys 0.001s
I20260709 12:22:58.384008  5149 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:58.435071  5272 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.453795  5515 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.454746  5515 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:58.461684  5546 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:22:58.464303  5188 leader_election.cc:343] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.476063  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:58.485750  5399 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:58.486300  5399 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.486732  5399 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.477219  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:58.467898  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:58.489974  5272 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:58.490881  5073 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.491293  4998 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:58.492358  5270 tablet_bootstrap.cc:654] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:58.495143  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 51237.7us
I20260709 12:22:58.469918  5546 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.505599  5271 tablet_bootstrap.cc:492] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:58.509008  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:58.510111  5075 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:22:58.503087  5546 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:58.548893  5515 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:58.564306  5272 ts_tablet_manager.cc:1434] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.533s	user 0.009s	sys 0.003s
I20260709 12:22:58.549275  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:22:58.528518  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 97532us
I20260709 12:22:58.549275  5557 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:58.566104  5272 ts_tablet_manager.cc:1467] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Tablet startup took 535ms
I20260709 12:22:58.574345  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:58.576052  5146 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.587476  5269 tablet_bootstrap.cc:654] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.589236  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:58.590369  5001 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 1: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:22:58.591594  5000 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:22:58.608069  5073 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:58.582577  5042 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:58.612236  5223 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:58.578053  5557 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:58.628170  5542 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:58.628674  5542 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.629036  5542 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.629112  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:58.639384  5557 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:22:58.578445  5272 ts_tablet_manager.cc:1469] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Trace:
0709 12:22:53.005909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.017359 (+ 11450us) service_pool.cc:224] Handling call
0709 12:22:53.038451 (+ 21092us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:22:53.038577 (+   126us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:22:53.193824 (+155247us) tablet_metadata.cc:665] Metadata flushed
0709 12:22:53.331348 (+137524us) tablet_replica.cc:206] Creating consensus instance
0709 12:22:53.337237 (+  5889us) inbound_call.cc:177] Queueing success response
0709 12:22:57.885356 (+4548119us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:22:58.030914 (+145558us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:22:58.034146 (+  3232us) tablet_replica.cc:261] Starting instrumentation
0709 12:22:58.034937 (+   791us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":9463,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":178,"tablet-open.queue_time_us":4553416}
I20260709 12:22:58.644001  5272 tablet_bootstrap.cc:492] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:58.648252  5545 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.651546  5554 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:58.653072  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:58.651589  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:58.656001  5554 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.656440  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:58.659420  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:58.660190  5149 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.663993  5399 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:58.668365  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:58.669152  4999 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:22:58.669852  4997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:58.670280  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:58.670620  4997 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.671679  5222 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 1: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:22:58.675057  4995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:58.675752  4995 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.680415  4966 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:58.682375  5552 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:58.682940  5552 raft_consensus.cc:697] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:22:58.682252  5554 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:58.683951  5552 consensus_queue.cc:237] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:58.681262  4997 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:58.699694  5341 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.718757  5341 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.720005  5270 tablet_bootstrap.cc:492] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:58.720683  5270 ts_tablet_manager.cc:1403] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.401s	user 0.020s	sys 0.032s
I20260709 12:22:58.725392  5271 tablet_bootstrap.cc:654] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:58.741946  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 85691.6us
I20260709 12:22:58.727638  5404 raft_consensus.cc:2764] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:22:58.743278  5341 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:58.725483  5270 raft_consensus.cc:359] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:58.762701  5270 raft_consensus.cc:385] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.763156  5270 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
W20260709 12:22:58.685614  5042 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.1.123.126:38285, user_credentials={real_user=slave}} blocked reactor thread for 53411.5us
I20260709 12:22:58.764827  5270 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:58.770205  4995 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:58.776361  5542 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:58.786347  5272 tablet_bootstrap.cc:654] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.736898  5149 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:58.794984  5542 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:58.797626  5270 ts_tablet_manager.cc:1434] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.077s	user 0.016s	sys 0.000s
I20260709 12:22:58.799010  5270 tablet_bootstrap.cc:492] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:58.802687  4995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:58.806039  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:58.811686  5041 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:22:58.812865  5542 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:58.813274  5542 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.813668  5542 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.685462  5546 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.815742  5220 raft_consensus.cc:2368] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:22:58.818003  4995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:58.819969  5546 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:58.823553  5542 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:58.824499  5404 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:58.825392  5115 leader_election.cc:400] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:22:58.826673  5115 leader_election.cc:403] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:58.827302  5554 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:22:58.827790  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:58.828711  5073 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 1: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:22:58.829820  5189 leader_election.cc:343] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.831416  5569 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:58.832564  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:58.833592  5569 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:58.833725  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:22:58.834102  5569 raft_consensus.cc:697] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:58.835332  5569 consensus_queue.cc:237] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:58.837822  5554 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
W20260709 12:22:58.825296  5041 leader_election.cc:343] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.839522  5404 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:58.869331  5146 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:58.780623  5545 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:58.819991  4995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:58.890113  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:58.890851  5149 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.893939  5544 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:58.894616  5544 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:58.895586  5544 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:22:58.897456  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:58.899914  5269 tablet_bootstrap.cc:492] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:58.904534  5269 ts_tablet_manager.cc:1403] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.647s	user 0.024s	sys 0.012s
I20260709 12:22:58.902364  5399 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:58.908522  5269 raft_consensus.cc:359] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:58.915298  5269 raft_consensus.cc:385] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.915802  5269 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.935142  5269 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:58.928426  5545 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:58.924499  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 90117.3us
I20260709 12:22:58.931700  5542 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:22:58.950541  5189 leader_election.cc:343] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.953790  5557 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:58.937847  5404 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:22:58.960955  5269 ts_tablet_manager.cc:1434] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.056s	user 0.003s	sys 0.005s
I20260709 12:22:58.962401  5269 tablet_bootstrap.cc:492] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:58.964035  5190 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:58.964696  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:58.939260  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:22:58.970856  5042 leader_election.cc:343] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.978300  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:22:58.980924  5567 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:22:58.981618  4995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:58.943351  5545 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:58.993413  4995 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.995589  5545 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:58.981314  5553 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:59.003453  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:59.005241  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:58.968549  5569 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.009068  5270 tablet_bootstrap.cc:654] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.980605  5146 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
W20260709 12:22:59.013291  5188 leader_election.cc:343] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.007736  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:59.014465  5399 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:22:59.020493  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:59.024161  5149 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:59.025116  5220 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 1: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:22:59.018811  5569 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:59.028798  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:59.034528  4998 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:59.039225  5272 tablet_bootstrap.cc:492] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:59.039880  5272 ts_tablet_manager.cc:1403] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.396s	user 0.026s	sys 0.010s
I20260709 12:22:59.037833  4895 catalog_manager.cc:5697] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.043018  5148 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:22:59.047421  5271 tablet_bootstrap.cc:492] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:59.052618  5271 ts_tablet_manager.cc:1403] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.592s	user 0.028s	sys 0.014s
I20260709 12:22:59.053642  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:22:59.074395  5189 leader_election.cc:343] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.078311  4995 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:59.083143  5042 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:22:59.094030  5269 tablet_bootstrap.cc:654] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.063458  5569 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:59.033571  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:59.099138  5272 raft_consensus.cc:359] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.100662  5272 raft_consensus.cc:385] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.101078  5272 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
W20260709 12:22:59.111836  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 81056.5us
I20260709 12:22:59.111708  5041 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:59.113827  5556 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.114358  5556 raft_consensus.cc:697] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:22:59.114552  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:59.115393  5556 consensus_queue.cc:237] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.120179  5272 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.106695  5557 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.110193  5341 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:59.126432  4998 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:22:59.125517  5557 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:22:59.123711  5341 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.147126  5271 raft_consensus.cc:359] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.147876  5271 raft_consensus.cc:385] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.148176  5271 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.140609  5341 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.147310  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:22:59.149292  5189 leader_election.cc:343] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.159083  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:59.138042  5557 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.163306  5552 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.165246  5568 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:59.166009  5568 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:59.166119  5546 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:59.167135  5568 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:59.171867  5270 tablet_bootstrap.cc:492] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:59.172554  5270 ts_tablet_manager.cc:1403] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.374s	user 0.009s	sys 0.028s
I20260709 12:22:59.180544  5270 raft_consensus.cc:359] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.181525  5270 raft_consensus.cc:385] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.182017  5270 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.201896  5271 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.200676  5270 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.221987  5272 ts_tablet_manager.cc:1434] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.182s	user 0.006s	sys 0.013s
I20260709 12:22:59.230485  5270 ts_tablet_manager.cc:1434] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.058s	user 0.016s	sys 0.000s
I20260709 12:22:59.235291  5272 tablet_bootstrap.cc:492] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:59.240645  5149 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.250373  5341 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:59.251870  5272 tablet_bootstrap.cc:654] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.252781  5341 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:59.254138  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:59.254449  5550 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.254873  5147 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.265388  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:22:59.279956  5189 leader_election.cc:343] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.278311  5149 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:59.286363  5147 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:22:59.290359  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:59.291801  5546 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.292295  5546 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:22:59.293160  5546 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:59.294591  5270 tablet_bootstrap.cc:492] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:59.299422  5271 ts_tablet_manager.cc:1434] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.246s	user 0.008s	sys 0.001s
I20260709 12:22:59.311822  5269 tablet_bootstrap.cc:492] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:59.312676  5269 ts_tablet_manager.cc:1403] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.351s	user 0.011s	sys 0.019s
I20260709 12:22:59.319708  5269 raft_consensus.cc:359] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:59.326505  5269 raft_consensus.cc:385] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.326936  5271 tablet_bootstrap.cc:492] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:59.326913  5269 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.347805  5341 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.352397  5269 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:59.361620  5553 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.362251  5553 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:59.363844  5271 tablet_bootstrap.cc:654] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.364894  5553 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:59.365790  5550 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.367774  5550 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.368528  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:59.370399  5550 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:59.371326  4997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:59.372195  4997 raft_consensus.cc:2393] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 1: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:22:59.376214  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:59.376963  5073 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:22:59.378547  4965 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:22:59.379894  5552 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:59.380390  5552 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.380777  5552 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.387365  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:59.389820  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:59.392644  5269 ts_tablet_manager.cc:1434] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.003s
W20260709 12:22:59.392923  4967 leader_election.cc:343] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.393759  5553 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.394096  5269 tablet_bootstrap.cc:492] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:59.395960  5568 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.405253  4899 catalog_manager.cc:5697] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.409965  5552 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.411598  4899 catalog_manager.cc:5697] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.412680  5552 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:59.414237  5568 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.415126  5270 tablet_bootstrap.cc:654] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.414811  5568 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.422617  5568 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:22:59.426357  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:22:59.399431  4898 catalog_manager.cc:5697] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.425127  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:59.427075  5075 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:59.430279  5190 leader_election.cc:343] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.436514  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:59.441718  4898 catalog_manager.cc:5697] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.444917  5269 tablet_bootstrap.cc:654] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:59.455936  4967 leader_election.cc:343] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.458819  5272 tablet_bootstrap.cc:492] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:59.459744  5272 ts_tablet_manager.cc:1403] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.225s	user 0.012s	sys 0.007s
I20260709 12:22:59.461974  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:59.466439  5399 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:22:59.470463  5188 leader_election.cc:343] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.471076  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:22:59.473196  5399 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:59.474750  5075 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:22:59.481156  4965 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:22:59.489486  5553 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:59.494233  5550 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.495124  5550 raft_consensus.cc:697] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:22:59.496153  5550 consensus_queue.cc:237] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.499328  5272 raft_consensus.cc:359] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:59.500202  5272 raft_consensus.cc:385] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.500495  5272 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.505954  5568 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.508196  5271 tablet_bootstrap.cc:492] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:59.508836  5271 ts_tablet_manager.cc:1403] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.192s	user 0.017s	sys 0.010s
I20260709 12:22:59.519107  5272 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:59.525943  5269 tablet_bootstrap.cc:492] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:59.526647  5269 ts_tablet_manager.cc:1403] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.133s	user 0.020s	sys 0.014s
I20260709 12:22:59.526809  5272 ts_tablet_manager.cc:1434] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.002s
I20260709 12:22:59.539381  5557 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.540169  5269 raft_consensus.cc:359] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.545673  4895 catalog_manager.cc:5697] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.546854  5269 raft_consensus.cc:385] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.551880  5269 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.552151  5271 raft_consensus.cc:359] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.552954  5271 raft_consensus.cc:385] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.552706  5269 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.553253  5271 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.555015  5272 tablet_bootstrap.cc:492] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:59.555426  5269 ts_tablet_manager.cc:1434] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:22:59.556587  5269 tablet_bootstrap.cc:492] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:59.559733  5271 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.563433  5271 ts_tablet_manager.cc:1434] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 12:22:59.564774  5271 tablet_bootstrap.cc:492] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:59.566136  5269 tablet_bootstrap.cc:654] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.551425  5270 tablet_bootstrap.cc:492] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:59.572293  5270 ts_tablet_manager.cc:1403] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.278s	user 0.021s	sys 0.007s
I20260709 12:22:59.573747  5272 tablet_bootstrap.cc:654] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.576370  5271 tablet_bootstrap.cc:654] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.567072  4895 catalog_manager.cc:5697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.585722  4895 catalog_manager.cc:5697] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.586437  5556 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.588119  5542 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.588624  5542 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:59.589267  5270 raft_consensus.cc:359] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.590158  5270 raft_consensus.cc:385] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.590426  5270 raft_consensus.cc:740] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.590965  5542 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:59.591368  5270 consensus_queue.cc:260] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.593261  5270 ts_tablet_manager.cc:1434] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 12:22:59.594511  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:59.595244  5224 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:59.596361  5270 tablet_bootstrap.cc:492] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:59.596153  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:59.598076  4998 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:22:59.599850  5041 leader_election.cc:343] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.600873  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:59.601778  5515 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.602319  5515 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.602677  5542 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:59.603221  5542 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.603598  5542 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.605386  5568 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.612505  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:22:59.614218  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:22:59.615212  5069 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 1: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:22:59.619851  5115 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:59.620950  5382 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.624696  5542 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:59.630198  5515 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:22:59.631593  5147 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:59.647282  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:59.648092  5224 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:59.653223  5610 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.055s	user 0.000s	sys 0.017s
I20260709 12:22:59.653484  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:22:59.654881  5568 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:22:59.655339  5041 leader_election.cc:343] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.656368  5271 tablet_bootstrap.cc:492] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:59.656927  5271 ts_tablet_manager.cc:1403] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.092s	user 0.019s	sys 0.017s
W20260709 12:22:59.657871  5589 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.009s
I20260709 12:22:59.664127  4998 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.667716  5270 tablet_bootstrap.cc:654] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.670084  5269 tablet_bootstrap.cc:492] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:22:59.660254  5271 raft_consensus.cc:359] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.671044  5271 raft_consensus.cc:385] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.671379  5271 raft_consensus.cc:740] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.673067  5224 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:59.673885  5271 consensus_queue.cc:260] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.676718  5269 ts_tablet_manager.cc:1403] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.114s	user 0.007s	sys 0.007s
I20260709 12:22:59.679822  5271 ts_tablet_manager.cc:1434] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.005s
I20260709 12:22:59.681142  5271 tablet_bootstrap.cc:492] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:22:59.682753  5382 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:59.684072  5272 tablet_bootstrap.cc:492] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:22:59.647533  5542 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:22:59.687882  5553 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.687866  5272 ts_tablet_manager.cc:1403] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.133s	user 0.011s	sys 0.006s
I20260709 12:22:59.688745  5269 raft_consensus.cc:359] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.689640  5269 raft_consensus.cc:385] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.697206  5399 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.689913  5269 raft_consensus.cc:740] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.709827  5271 tablet_bootstrap.cc:654] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.706586  5269 consensus_queue.cc:260] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.709496  4899 catalog_manager.cc:5697] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.712441  5269 ts_tablet_manager.cc:1434] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 12:22:59.713624  5269 tablet_bootstrap.cc:492] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:22:59.716562  5542 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.717262  5542 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.718019  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:22:59.720203  5557 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.719694  5542 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:59.720855  5557 raft_consensus.cc:697] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:22:59.726912  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:22:59.731413  5150 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:59.733270  5042 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:59.734253  5404 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:59.734750  5404 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.722045  5557 consensus_queue.cc:237] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:59.734974  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:22:59.735098  5404 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.735860  5220 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:22:59.727171  5542 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:22:59.740800  5272 raft_consensus.cc:359] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.756549  5272 raft_consensus.cc:385] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.756918  5272 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.757526  5272 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:22:59.762692  5272 ts_tablet_manager.cc:1434] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.000s
I20260709 12:22:59.787351  5269 tablet_bootstrap.cc:654] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.790941  5270 tablet_bootstrap.cc:492] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:59.791570  5270 ts_tablet_manager.cc:1403] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.196s	user 0.015s	sys 0.004s
I20260709 12:22:59.799265  5272 tablet_bootstrap.cc:492] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:22:59.808507  5270 raft_consensus.cc:359] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:59.809113  5404 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.809383  5270 raft_consensus.cc:385] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.811060  5270 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.820576  5404 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:22:59.821611  5270 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:22:59.822108  5544 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.827637  5270 ts_tablet_manager.cc:1434] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.001s
I20260709 12:22:59.828965  5270 tablet_bootstrap.cc:492] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:22:59.831300  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:22:59.834661  5150 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.831842  4897 catalog_manager.cc:5697] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:59.836046  5546 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.838295  5271 tablet_bootstrap.cc:492] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:22:59.838831  5271 ts_tablet_manager.cc:1403] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.158s	user 0.016s	sys 0.019s
I20260709 12:22:59.837807  5272 tablet_bootstrap.cc:654] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.844412  5270 tablet_bootstrap.cc:654] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.850484  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:22:59.851305  5222 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.854740  5271 raft_consensus.cc:359] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:59.855572  5271 raft_consensus.cc:385] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.855854  5271 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.862874  5271 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:59.880589  5150 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:22:59.882485  5042 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:22:59.884495  5553 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.885083  5553 raft_consensus.cc:697] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:22:59.886327  5553 consensus_queue.cc:237] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:22:59.890019  5271 ts_tablet_manager.cc:1434] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.051s	user 0.009s	sys 0.000s
I20260709 12:22:59.896621  5270 tablet_bootstrap.cc:492] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:22:59.897482  5270 ts_tablet_manager.cc:1403] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.069s	user 0.006s	sys 0.009s
I20260709 12:22:59.902930  5270 raft_consensus.cc:359] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:59.903798  5270 raft_consensus.cc:385] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.904095  5270 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.906708  5270 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:22:59.947901  5271 tablet_bootstrap.cc:492] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
W20260709 12:22:59.958010  5625 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:22:59.963514  5222 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:00.003561  5270 ts_tablet_manager.cc:1434] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.106s	user 0.020s	sys 0.018s
I20260709 12:23:00.004943  5270 tablet_bootstrap.cc:492] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:23:00.010282  5269 tablet_bootstrap.cc:492] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:00.010958  5269 ts_tablet_manager.cc:1403] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.298s	user 0.013s	sys 0.020s
I20260709 12:23:00.033701  5553 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.042119  5553 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:00.048019  5603 consensus_queue.cc:1048] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:00.048139  5270 tablet_bootstrap.cc:654] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.052256  5272 tablet_bootstrap.cc:492] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:00.052999  5272 ts_tablet_manager.cc:1403] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.254s	user 0.019s	sys 0.011s
I20260709 12:23:00.053121  5603 consensus_queue.cc:1048] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:00.056723  5272 raft_consensus.cc:359] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.057487  5272 raft_consensus.cc:385] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.057700  5271 tablet_bootstrap.cc:654] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.061882  5553 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:00.061908  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:00.062937  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:00.063187  4996 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:23:00.064750  5041 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:00.065703  5042 leader_election.cc:343] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:00.066409  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:00.068447  5557 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.069815  5557 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.070254  5557 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.070485  5568 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.071066  5568 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.073321  5568 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:00.075110  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:00.080199  5188 leader_election.cc:343] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.057821  5272 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.099388  4899 catalog_manager.cc:5697] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:00.077505  5269 raft_consensus.cc:359] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:00.103149  5269 raft_consensus.cc:385] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.103725  5269 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.105182  5272 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.107280  5272 ts_tablet_manager.cc:1434] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.054s	user 0.004s	sys 0.004s
I20260709 12:23:00.108577  5269 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:00.114753  5630 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.118355  5630 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.120723  5630 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:00.121165  5272 tablet_bootstrap.cc:492] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:00.123775  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:00.124497  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:00.125607  5115 leader_election.cc:343] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.130148  5269 ts_tablet_manager.cc:1434] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.119s	user 0.008s	sys 0.001s
I20260709 12:23:00.131351  5269 tablet_bootstrap.cc:492] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:23:00.141124  5556 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.142074  5556 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:00.146379  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:00.148878  5272 tablet_bootstrap.cc:654] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:00.150794  5042 leader_election.cc:343] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.152418  5556 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:00.152995  5269 tablet_bootstrap.cc:654] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.153829  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:00.155222  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:00.161497  5189 leader_election.cc:343] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:00.196933  5115 leader_election.cc:343] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.199069  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:00.216035  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:00.170775  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:00.215256  5382 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:00.231808  5557 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:00.255604  5272 tablet_bootstrap.cc:492] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:00.259977  5272 ts_tablet_manager.cc:1403] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.148s	user 0.020s	sys 0.039s
I20260709 12:23:00.236449  5568 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:00.259982  5270 tablet_bootstrap.cc:492] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:23:00.265128  5270 ts_tablet_manager.cc:1403] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.260s	user 0.057s	sys 0.051s
W20260709 12:23:00.246598  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 85157.5us
W20260709 12:23:00.275182  5632 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:23:00.294760  5557 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:00.296233  5272 raft_consensus.cc:359] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.297109  5272 raft_consensus.cc:385] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.298717  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:00.297394  5272 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
W20260709 12:23:00.300209  5042 leader_election.cc:343] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.300832  5272 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.307174  5271 tablet_bootstrap.cc:492] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:00.307871  5271 ts_tablet_manager.cc:1403] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.367s	user 0.022s	sys 0.002s
W20260709 12:23:00.308460  5041 leader_election.cc:343] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.309064  5041 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:00.309883  5553 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:00.312156  5271 raft_consensus.cc:359] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:00.312922  5271 raft_consensus.cc:385] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.313249  5271 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.314213  5271 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:00.323206  5270 raft_consensus.cc:359] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.324095  5270 raft_consensus.cc:385] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.324385  5270 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.327986  5272 ts_tablet_manager.cc:1434] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.001s
I20260709 12:23:00.329602  5269 tablet_bootstrap.cc:492] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:00.330143  5269 ts_tablet_manager.cc:1403] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.199s	user 0.012s	sys 0.008s
I20260709 12:23:00.329934  5270 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.333441  5270 ts_tablet_manager.cc:1434] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 12:23:00.329183  5272 tablet_bootstrap.cc:492] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:00.333232  5269 raft_consensus.cc:359] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.337702  5269 raft_consensus.cc:385] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.339131  5271 ts_tablet_manager.cc:1434] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 12:23:00.339980  5270 tablet_bootstrap.cc:492] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:23:00.338109  5269 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.353240  5269 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.357415  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:00.358268  4996 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:00.360869  5041 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:00.364481  5603 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.365110  5603 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.367374  5603 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:00.373076  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:00.373929  5569 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.374506  5569 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.376739  5569 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:00.379495  5640 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.381865  5544 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.396075  5269 ts_tablet_manager.cc:1434] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.002s
I20260709 12:23:00.401114  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:00.404618  5556 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:23:00.404765  5542 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:23:00.402017  4997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:00.383668  5271 tablet_bootstrap.cc:492] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:00.410955  5269 tablet_bootstrap.cc:492] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:23:00.384477  5640 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.410257  5270 tablet_bootstrap.cc:654] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.409345  5544 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.401901  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:00.423694  4966 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:00.429647  5188 leader_election.cc:343] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.430009  5603 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:00.441887  5271 tablet_bootstrap.cc:654] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.442193  5544 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:00.443722  5189 leader_election.cc:343] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.458459  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:00.446162  5640 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:00.446723  5556 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.460920  5544 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:00.461037  5556 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.463943  5556 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:00.466312  4997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:00.467219  4997 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:23:00.447544  5269 tablet_bootstrap.cc:654] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.468755  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:00.473608  5190 leader_election.cc:343] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.480253  5556 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:00.409313  5272 tablet_bootstrap.cc:654] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.447926  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:00.451893  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:00.487042  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:00.486799  5271 tablet_bootstrap.cc:492] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:00.488085  5070 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:23:00.488571  5271 ts_tablet_manager.cc:1403] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.130s	user 0.013s	sys 0.014s
I20260709 12:23:00.489511  4965 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:00.403967  4996 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:00.457074  5270 tablet_bootstrap.cc:492] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:23:00.490604  5640 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.491001  5640 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.491079  5270 ts_tablet_manager.cc:1403] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.151s	user 0.015s	sys 0.006s
I20260709 12:23:00.491432  5640 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.494451  5041 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:00.494614  5270 raft_consensus.cc:359] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.495476  5270 raft_consensus.cc:385] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.495788  5270 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.496682  4996 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.496548  5270 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.499058  5270 ts_tablet_manager.cc:1434] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:00.500252  5270 tablet_bootstrap.cc:492] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:23:00.506455  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:00.507228  5222 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:23:00.507998  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:00.508327  5189 leader_election.cc:343] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.508791  5223 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:23:00.508826  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:00.510113  5270 tablet_bootstrap.cc:654] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.510437  5399 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:00.512084  5542 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.512542  5542 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.512966  5542 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.518586  5553 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:00.522779  5270 tablet_bootstrap.cc:492] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:23:00.523402  5270 ts_tablet_manager.cc:1403] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.010s
I20260709 12:23:00.524174  5553 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:00.525506  5553 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:00.526428  5270 raft_consensus.cc:359] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.527148  5270 raft_consensus.cc:385] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.527434  5270 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.528262  5270 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.530944  5270 ts_tablet_manager.cc:1434] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:00.532116  5270 tablet_bootstrap.cc:492] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:23:00.534693  5271 raft_consensus.cc:359] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:00.535488  5271 raft_consensus.cc:385] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.535766  5271 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.536434  5271 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:00.539208  5270 tablet_bootstrap.cc:654] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.540712  5272 tablet_bootstrap.cc:492] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:00.541267  5272 ts_tablet_manager.cc:1403] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.212s	user 0.011s	sys 0.010s
I20260709 12:23:00.544353  5272 raft_consensus.cc:359] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.544792  5269 tablet_bootstrap.cc:492] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:00.545960  5269 ts_tablet_manager.cc:1403] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.135s	user 0.010s	sys 0.004s
I20260709 12:23:00.555243  5542 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.562134  5271 ts_tablet_manager.cc:1434] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.000s
I20260709 12:23:00.563576  5271 tablet_bootstrap.cc:492] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:00.568229  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:00.567932  5269 raft_consensus.cc:359] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:00.568902  5224 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.569348  4899 catalog_manager.cc:5697] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:00.568940  5269 raft_consensus.cc:385] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.572460  5269 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.579052  5224 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:00.582625  5269 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:00.545244  5272 raft_consensus.cc:385] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.579546  5271 tablet_bootstrap.cc:654] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.590342  5269 ts_tablet_manager.cc:1434] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.002s
I20260709 12:23:00.592335  5269 tablet_bootstrap.cc:492] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:23:00.584208  5272 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.599375  5272 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.607620  5269 tablet_bootstrap.cc:654] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.601745  5640 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.598223  5542 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:00.626729  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:00.622551  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:00.629010  5272 ts_tablet_manager.cc:1434] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.087s	user 0.004s	sys 0.004s
I20260709 12:23:00.629845  5070 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 1: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:00.631314  4997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:00.632625  4997 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 1: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:00.618700  5640 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:00.655421  5272 tablet_bootstrap.cc:492] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:00.666080  5270 tablet_bootstrap.cc:492] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:23:00.666929  5270 ts_tablet_manager.cc:1403] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.135s	user 0.027s	sys 0.010s
I20260709 12:23:00.675526  5480 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.676234  5480 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:00.678767  5480 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:00.682457  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:00.662146  5542 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:00.699914  5399 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.723138  5542 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:00.719228  5399 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.736984  5399 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:00.745978  5544 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.734390  4997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:00.746253  5640 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.777022  5271 tablet_bootstrap.cc:492] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:00.778173  5271 ts_tablet_manager.cc:1403] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.214s	user 0.016s	sys 0.001s
I20260709 12:23:00.772850  4997 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 1: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:00.757719  5544 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.777985  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:00.759871  5640 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.785037  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:00.788779  5223 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 1: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:00.799986  5270 raft_consensus.cc:359] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:00.800760  5270 raft_consensus.cc:385] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.800997  5270 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.802330  5269 tablet_bootstrap.cc:492] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:00.802845  5269 ts_tablet_manager.cc:1403] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.211s	user 0.032s	sys 0.041s
I20260709 12:23:00.803495  5270 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:00.762184  5542 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:00.808776  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:00.816219  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:00.810642  5382 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.825698  5270 ts_tablet_manager.cc:1434] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.158s	user 0.007s	sys 0.002s
I20260709 12:23:00.831710  5544 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:00.838747  5270 tablet_bootstrap.cc:492] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:23:00.842377  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:00.847259  5150 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:23:00.843393  5269 raft_consensus.cc:359] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:00.847959  5269 raft_consensus.cc:385] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.848275  5269 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.846256  5640 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:00.842842  4999 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.849422  5269 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:00.851881  5269 ts_tablet_manager.cc:1434] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.001s
I20260709 12:23:00.853688  5382 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:00.854079  5269 tablet_bootstrap.cc:492] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
W20260709 12:23:00.854043  5190 leader_election.cc:343] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.862169  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:00.862535  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:00.864320  5399 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.864743  5399 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.865088  5399 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.866438  5271 raft_consensus.cc:359] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.880200  5271 raft_consensus.cc:385] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.859483  5272 tablet_bootstrap.cc:654] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.880592  5271 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.885875  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:00.885208  5399 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:00.887228  5189 leader_election.cc:343] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.888839  5189 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:00.888908  5271 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.889863  5546 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:00.892913  5399 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:00.893045  5546 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.893642  5546 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:00.894268  4966 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:00.895150  5640 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:00.897320  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:00.899569  4966 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:00.901126  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:00.903167  5546 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:00.904138  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:00.905503  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:00.905768  5188 leader_election.cc:343] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.906270  5150 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.906913  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:00.908823  5190 leader_election.cc:343] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.909322  5190 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:00.911585  5544 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:00.912814  5640 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:00.912709  5189 leader_election.cc:343] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:00.914835  5189 leader_election.cc:343] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.919014  5150 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:00.913014  5271 ts_tablet_manager.cc:1434] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.134s	user 0.009s	sys 0.000s
I20260709 12:23:00.920893  5271 tablet_bootstrap.cc:492] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:00.924024  5480 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.924633  5480 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.906924  5269 tablet_bootstrap.cc:654] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.929006  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:00.929890  5222 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:23:00.926818  5480 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:00.932116  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:00.933066  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:00.933844  5382 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.934440  5544 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:00.934515  5382 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.935120  5544 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:00.935181  5382 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.935828  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:00.934902  4898 catalog_manager.cc:5697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:00.936090  5544 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.880697  5270 tablet_bootstrap.cc:654] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.942701  5272 tablet_bootstrap.cc:492] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:00.943339  5272 ts_tablet_manager.cc:1403] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.313s	user 0.013s	sys 0.007s
W20260709 12:23:00.944067  5115 leader_election.cc:343] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.946653  5272 raft_consensus.cc:359] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:00.947476  5272 raft_consensus.cc:385] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.949388  5272 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.950290  5272 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:00.948649  5382 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:00.953028  5272 ts_tablet_manager.cc:1434] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:00.958875  5272 tablet_bootstrap.cc:492] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:00.968545  5272 tablet_bootstrap.cc:654] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.968613  5659 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:00.978364  5556 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:00.979944  5382 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:00.991786  5270 tablet_bootstrap.cc:492] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:23:00.991595  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:00.992374  5222 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.993155  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:01.001150  4898 catalog_manager.cc:5697] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.004065  5271 tablet_bootstrap.cc:654] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.992388  5270 ts_tablet_manager.cc:1403] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.159s	user 0.009s	sys 0.003s
I20260709 12:23:01.009050  5515 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.009670  5515 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:01.011356  5115 leader_election.cc:343] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.012032  5515 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:01.014281  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:01.015003  5074 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:23:01.018103  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:01.027995  5659 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.028656  5659 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:01.026088  5115 leader_election.cc:343] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.025993  5270 raft_consensus.cc:359] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.032480  5270 raft_consensus.cc:385] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.032832  5270 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.034179  5270 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.035516  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:01.037307  5042 leader_election.cc:343] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.042764  5270 ts_tablet_manager.cc:1434] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.003s
I20260709 12:23:01.045648  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:01.046450  5223 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:23:01.043226  5659 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:01.047741  5041 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:01.049647  5556 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:01.050096  5556 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.050501  5556 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.051734  5270 tablet_bootstrap.cc:492] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:23:01.053826  5114 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:01.054795  5382 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:01.056107  5222 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:01.057044  5269 tablet_bootstrap.cc:492] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:01.057638  5269 ts_tablet_manager.cc:1403] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.204s	user 0.013s	sys 0.021s
I20260709 12:23:01.060376  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:01.061581  5515 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.062052  5515 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:01.062922  5515 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:01.068158  5270 tablet_bootstrap.cc:654] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.055262  5382 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.073421  5382 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.080372  5640 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.081046  5640 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.083114  5556 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.084586  5640 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:01.088199  5404 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.088860  5404 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.088970  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:01.090088  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:01.092501  4967 leader_election.cc:343] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.093449  5404 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:01.094691  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:01.095427  5222 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.097990  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:01.101946  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:01.103893  5041 leader_election.cc:343] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.104558  5272 tablet_bootstrap.cc:492] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:01.105309  5272 ts_tablet_manager.cc:1403] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.147s	user 0.014s	sys 0.008s
W20260709 12:23:01.105954  4965 leader_election.cc:343] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.106490  4965 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:01.105594  5042 leader_election.cc:343] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.109582  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:01.113601  5640 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:01.114264  5556 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:01.115379  5042 leader_election.cc:343] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.115970  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:01.117964  5404 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:01.124225  5222 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:01.129833  5041 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:01.131421  5659 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.122920  5271 tablet_bootstrap.cc:492] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:01.132735  5659 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:01.135010  5271 ts_tablet_manager.cc:1403] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.214s	user 0.019s	sys 0.004s
I20260709 12:23:01.135618  5659 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.144605  5270 tablet_bootstrap.cc:492] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:23:01.145213  5270 ts_tablet_manager.cc:1403] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.097s	user 0.024s	sys 0.046s
I20260709 12:23:01.148123  5270 raft_consensus.cc:359] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.148810  5270 raft_consensus.cc:385] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.149071  5270 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.150859  5270 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.159103  5270 ts_tablet_manager.cc:1434] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:23:01.160389  5270 tablet_bootstrap.cc:492] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:23:01.160488  4899 catalog_manager.cc:5697] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.171144  5271 raft_consensus.cc:359] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.126785  5382 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.077653  5269 raft_consensus.cc:359] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.173698  5269 raft_consensus.cc:385] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.174005  5269 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.174741  5269 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.175642  5270 tablet_bootstrap.cc:654] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.176091  4898 catalog_manager.cc:5697] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.177016  5269 ts_tablet_manager.cc:1434] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.119s	user 0.006s	sys 0.000s
I20260709 12:23:01.143453  5272 raft_consensus.cc:359] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:01.177816  5272 raft_consensus.cc:385] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.178097  5272 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.178879  5272 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:01.181267  5272 ts_tablet_manager.cc:1434] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.076s	user 0.004s	sys 0.000s
I20260709 12:23:01.182372  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:01.183079  5074 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.184348  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:01.185063  5222 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.187682  5270 tablet_bootstrap.cc:492] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:23:01.188181  5270 ts_tablet_manager.cc:1403] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.000s
I20260709 12:23:01.191136  5270 raft_consensus.cc:359] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.191828  5270 raft_consensus.cc:385] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.192111  5270 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.192780  5270 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.171922  5271 raft_consensus.cc:385] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.194845  5271 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.182459  5272 tablet_bootstrap.cc:492] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:01.198168  5382 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:01.210018  5074 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:01.211529  5114 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:01.211936  5222 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:01.219753  5272 tablet_bootstrap.cc:654] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.222837  5271 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.221930  5269 tablet_bootstrap.cc:492] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:23:01.212554  5382 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.238897  5271 ts_tablet_manager.cc:1434] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.103s	user 0.008s	sys 0.001s
I20260709 12:23:01.198587  5270 ts_tablet_manager.cc:1434] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:23:01.239665  5382 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:01.244143  5399 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.244673  5399 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.246960  5399 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:01.249275  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:01.250977  5270 tablet_bootstrap.cc:492] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
W20260709 12:23:01.252231  5188 leader_election.cc:343] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:01.254211  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:01.253006  5382 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.255975  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:01.270797  5271 tablet_bootstrap.cc:492] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:01.272627  5640 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:01.272953  5189 leader_election.cc:343] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.273682  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:01.274379  5269 tablet_bootstrap.cc:654] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.273232  5640 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:01.275502  5399 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:01.280100  5640 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:01.282811  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:01.283787  5150 raft_consensus.cc:2393] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 1: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:01.297034  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:01.293192  5272 tablet_bootstrap.cc:492] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:01.300315  5272 ts_tablet_manager.cc:1403] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.118s	user 0.024s	sys 0.029s
I20260709 12:23:01.307148  5271 tablet_bootstrap.cc:654] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.308379  5272 raft_consensus.cc:359] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.309163  5272 raft_consensus.cc:385] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.309473  5272 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.310416  5272 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.312206  4966 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:01.323763  5640 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.324572  5270 tablet_bootstrap.cc:654] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.324570  5272 ts_tablet_manager.cc:1434] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:23:01.327961  5271 tablet_bootstrap.cc:492] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:01.328549  5271 ts_tablet_manager.cc:1403] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.005s
I20260709 12:23:01.328927  5272 tablet_bootstrap.cc:492] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:01.330497  5269 tablet_bootstrap.cc:492] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:01.331045  5269 ts_tablet_manager.cc:1403] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.133s	user 0.025s	sys 0.012s
I20260709 12:23:01.332295  5271 raft_consensus.cc:359] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.333091  5271 raft_consensus.cc:385] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.333411  5271 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.336014  5640 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:01.336611  5271 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.337141  5269 raft_consensus.cc:359] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.337991  5269 raft_consensus.cc:385] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.338308  5269 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.339196  5269 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.339216  5271 ts_tablet_manager.cc:1434] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:23:01.344444  5382 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.343948  4898 catalog_manager.cc:5697] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.344977  5382 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.348834  5382 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:01.349470  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:01.350251  4998 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:23:01.352088  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:01.352489  5269 ts_tablet_manager.cc:1434] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:23:01.352805  5075 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:23:01.353020  5271 tablet_bootstrap.cc:492] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:01.354619  5115 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:01.355654  5399 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.356170  5399 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.356875  5382 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:01.355978  5269 tablet_bootstrap.cc:492] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:23:01.357314  5382 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.357806  5270 tablet_bootstrap.cc:492] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:23:01.357962  5382 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.358355  5270 ts_tablet_manager.cc:1403] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.116s	user 0.011s	sys 0.002s
I20260709 12:23:01.359237  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:01.360751  5189 leader_election.cc:343] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.363742  5270 raft_consensus.cc:359] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:01.366593  5270 raft_consensus.cc:385] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.366931  5270 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.365718  5272 tablet_bootstrap.cc:654] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.369554  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:01.370954  5270 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:01.371822  5546 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.372293  5546 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:01.375523  5271 tablet_bootstrap.cc:654] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.376768  5546 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:01.377924  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:01.379374  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:01.378018  5382 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:01.391016  5190 leader_election.cc:343] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.391733  5546 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.392891  5075 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
I20260709 12:23:01.393989  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:01.394654  4999 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.394565  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:01.396189  5546 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:01.396582  5546 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.396908  5546 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:01.397300  5189 leader_election.cc:343] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.397821  5189 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:01.398460  5399 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:01.403627  5636 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.392148  5269 tablet_bootstrap.cc:654] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.405781  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:01.411757  5643 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:01.412086  5382 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:01.420912  5075 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.406275  4999 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:01.378901  5270 ts_tablet_manager.cc:1434] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.002s
I20260709 12:23:01.429889  5115 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:01.416190  5480 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.431311  5270 tablet_bootstrap.cc:492] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:23:01.426048  5546 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.431744  5382 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.414923  5569 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:23:01.430923  5480 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.432687  5382 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:01.436367  5382 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.436698  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:01.438123  5480 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:01.439924  5546 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:01.439834  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:01.443982  5222 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 1: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:01.445871  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:01.446816  5480 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.453159  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:01.473973  5515 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.477195  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:01.478053  5070 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:01.502035  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 49707.2us
I20260709 12:23:01.504282  5636 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:23:01.521709  5115 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:01.533896  5189 leader_election.cc:343] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.542357  5070 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:01.540853  5272 tablet_bootstrap.cc:492] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:01.587554  5075 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:01.592869  5272 ts_tablet_manager.cc:1403] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.264s	user 0.026s	sys 0.039s
I20260709 12:23:01.591933  4899 catalog_manager.cc:5697] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:01.600430  5270 tablet_bootstrap.cc:654] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.630080  5480 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:01.648918  5272 raft_consensus.cc:359] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.649921  5272 raft_consensus.cc:385] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.650228  5272 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.656796  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:01.658339  5544 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.658970  5544 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:01.659955  5544 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.666414  5269 tablet_bootstrap.cc:492] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:01.665225  5272 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.666987  5269 ts_tablet_manager.cc:1403] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.311s	user 0.047s	sys 0.069s
I20260709 12:23:01.668358  5271 tablet_bootstrap.cc:492] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:01.668907  5271 ts_tablet_manager.cc:1403] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.318s	user 0.051s	sys 0.050s
I20260709 12:23:01.669154  5272 ts_tablet_manager.cc:1434] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 12:23:01.670463  5272 tablet_bootstrap.cc:492] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:01.670176  5269 raft_consensus.cc:359] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.670943  5269 raft_consensus.cc:385] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.671212  5269 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.676424  5269 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.678860  5269 ts_tablet_manager.cc:1434] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:23:01.679862  5659 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.682703  5659 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.686331  5659 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:01.688608  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:01.675631  5271 raft_consensus.cc:359] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.689467  5222 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:23:01.689620  5271 raft_consensus.cc:385] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.692412  5271 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.693235  5271 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.694548  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:01.683156  5269 tablet_bootstrap.cc:492] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:23:01.714834  5271 ts_tablet_manager.cc:1434] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 12:23:01.716765  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:01.717638  4996 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:23:01.720243  5272 tablet_bootstrap.cc:654] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:01.720718  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 59798.9us
I20260709 12:23:01.703169  5659 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:01.737375  5271 tablet_bootstrap.cc:492] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:01.741196  5404 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.741915  5404 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.744599  5404 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:01.738968  5659 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.748759  5659 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.773826  5693 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.779942  5270 tablet_bootstrap.cc:492] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:23:01.780299  5269 tablet_bootstrap.cc:654] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.780843  5270 ts_tablet_manager.cc:1403] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.350s	user 0.019s	sys 0.015s
I20260709 12:23:01.784484  5270 raft_consensus.cc:359] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:01.788347  5270 raft_consensus.cc:385] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.788828  5270 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.789903  5270 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:01.792794  5270 ts_tablet_manager.cc:1434] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.005s
I20260709 12:23:01.794472  5270 tablet_bootstrap.cc:492] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:23:01.798859  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:01.807474  5271 tablet_bootstrap.cc:654] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.807348  5150 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:23:01.811645  5404 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.823746  5270 tablet_bootstrap.cc:654] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.825450  5272 tablet_bootstrap.cc:492] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:01.826148  5272 ts_tablet_manager.cc:1403] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.156s	user 0.037s	sys 0.027s
I20260709 12:23:01.798593  5659 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.841708  5659 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:01.843080  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:01.844035  5220 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.848629  5269 tablet_bootstrap.cc:492] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:01.849203  5269 ts_tablet_manager.cc:1403] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.166s	user 0.010s	sys 0.008s
I20260709 12:23:01.848930  5272 raft_consensus.cc:359] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.849772  5272 raft_consensus.cc:385] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.850082  5272 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.850804  5272 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.852083  5269 raft_consensus.cc:359] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.852748  5269 raft_consensus.cc:385] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.852984  5269 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.853083  5272 ts_tablet_manager.cc:1434] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.000s
I20260709 12:23:01.854676  5220 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:01.863411  5269 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.865934  5269 ts_tablet_manager.cc:1434] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
I20260709 12:23:01.867211  5269 tablet_bootstrap.cc:492] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:23:01.868235  5272 tablet_bootstrap.cc:492] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
W20260709 12:23:01.871531  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 67082.1us
I20260709 12:23:01.872663  5042 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:01.874045  5659 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:01.874435  5659 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.874783  5659 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.888577  5272 tablet_bootstrap.cc:654] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.913856  5693 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.907107  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:01.918205  4998 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:23:01.921823  5659 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:01.937338  5270 tablet_bootstrap.cc:492] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:23:01.938095  5270 ts_tablet_manager.cc:1403] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.144s	user 0.014s	sys 0.026s
I20260709 12:23:01.957782  5270 raft_consensus.cc:359] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.958591  5270 raft_consensus.cc:385] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.958884  5270 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.959517  5270 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.961582  5270 ts_tablet_manager.cc:1434] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 12:23:01.962738  5270 tablet_bootstrap.cc:492] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Bootstrap starting.
I20260709 12:23:01.965830  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:01.966564  5150 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.970074  5270 tablet_bootstrap.cc:654] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.976568  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:01.977348  5699 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.979466  5404 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.980020  5404 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:01.980993  5404 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:01.982928  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:01.983702  4998 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.983741  5270 tablet_bootstrap.cc:492] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: No bootstrap required, opened a new log
I20260709 12:23:01.984542  5270 ts_tablet_manager.cc:1403] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.004s
I20260709 12:23:01.987967  4996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:01.988726  4996 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.988562  5270 raft_consensus.cc:359] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.989346  5270 raft_consensus.cc:385] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.989708  5270 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.990836  5270 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:01.992551  5270 ts_tablet_manager.cc:1434] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:02.002735  5150 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:02.003774  5659 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:02.004283  5550 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.004830  5550 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:02.006965  5550 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:02.016453  5042 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:02.017634  5659 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:02.018391  5659 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:02.019410  5659 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:02.025842  5546 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.026682  5546 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:02.029726  5546 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:02.041380  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:02.045281  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:02.038457  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 99264.5us
W20260709 12:23:02.064157  5188 leader_election.cc:343] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:02.073482  5271 tablet_bootstrap.cc:492] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:02.074303  5271 ts_tablet_manager.cc:1403] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.337s	user 0.020s	sys 0.021s
I20260709 12:23:02.077819  5686 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:02.079453  5271 raft_consensus.cc:359] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:02.082737  5271 raft_consensus.cc:385] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.083225  5271 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.084062  5271 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:02.087263  5271 ts_tablet_manager.cc:1434] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:23:02.089818  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:02.093276  5271 tablet_bootstrap.cc:492] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:02.106217  5684 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.106992  5684 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:02.108326  5190 leader_election.cc:343] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:02.108953  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:02.110631  5546 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:02.111162  5684 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:02.112934  5000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:02.114931  5189 leader_election.cc:343] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:02.118206  5271 tablet_bootstrap.cc:654] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.121519  5515 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:02.131963  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:02.135746  4965 leader_election.cc:343] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:02.152387  5705 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.070s	user 0.003s	sys 0.000s
W20260709 12:23:02.156651  5190 leader_election.cc:343] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:02.157466  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:02.173839  5399 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:02.210618  5404 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.211413  5404 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:02.226258  5189 consensus_peers.cc:597] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:02.244299  5404 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:02.271162  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:02.272408  5223 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:23:02.282492  5041 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:02.283919  5556 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:02.284368  5556 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.284735  5556 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.136073  4996 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:02.142652  5272 tablet_bootstrap.cc:492] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:02.307379  5272 ts_tablet_manager.cc:1403] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.439s	user 0.019s	sys 0.009s
I20260709 12:23:02.310999  5272 raft_consensus.cc:359] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:02.311848  5272 raft_consensus.cc:385] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.312139  5272 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.312958  5272 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:02.314965  5272 ts_tablet_manager.cc:1434] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:02.316032  5272 tablet_bootstrap.cc:492] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:02.323500  5269 tablet_bootstrap.cc:654] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.339601  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:02.450623  5569 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:02.451464  5223 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:23:02.330360  5272 tablet_bootstrap.cc:654] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.339071  5271 tablet_bootstrap.cc:492] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:02.382498  4998 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:02.452245  5569 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.557224  5271 ts_tablet_manager.cc:1403] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.464s	user 0.037s	sys 0.049s
I20260709 12:23:02.557686  5569 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:02.560411  5569 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:02.460170  5699 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:02.561648  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:02.461817  5272 tablet_bootstrap.cc:492] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:02.564438  5272 ts_tablet_manager.cc:1403] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.249s	user 0.010s	sys 0.004s
W20260709 12:23:02.566586  5189 leader_election.cc:343] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:02.567543  5272 raft_consensus.cc:359] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:02.465389  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:02.568233  5272 raft_consensus.cc:385] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.476598  4966 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:23:02.511381  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 334041us
W20260709 12:23:02.569473  4966 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 93015.7us
I20260709 12:23:02.571072  5550 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:02.571511  5550 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.523862  5269 tablet_bootstrap.cc:492] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:02.560926  5271 raft_consensus.cc:359] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:02.536437  5404 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:02.572407  5269 ts_tablet_manager.cc:1403] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.705s	user 0.025s	sys 0.022s
I20260709 12:23:02.375433  5556 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:02.568570  5272 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.572829  5271 raft_consensus.cc:385] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.573009  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:02.577150  5271 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.576766  5149 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:23:02.580960  5272 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:02.581707  5271 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:02.571913  5550 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.599788  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:02.603406  5272 ts_tablet_manager.cc:1434] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 12:23:02.625737  5272 tablet_bootstrap.cc:492] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:02.602514  4895 catalog_manager.cc:5697] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:02.608256  5269 raft_consensus.cc:359] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:02.603312  5404 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:23:02.613864  5556 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:02.636826  5269 raft_consensus.cc:385] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.637722  5269 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.643982  5269 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:02.647969  5271 ts_tablet_manager.cc:1434] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.090s	user 0.007s	sys 0.000s
I20260709 12:23:02.637382  5636 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:23:02.658951  5271 tablet_bootstrap.cc:492] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:02.669945  5269 ts_tablet_manager.cc:1434] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.001s
I20260709 12:23:02.654464  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:02.677860  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:02.678735  5146 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.679633  5269 tablet_bootstrap.cc:492] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:23:02.688948  5220 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.725632  5686 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:02.726409  5686 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:02.732925  4999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:02.741820  5272 tablet_bootstrap.cc:654] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.752048  5686 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:02.759078  5724 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.763115  5640 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:02.766287  5188 leader_election.cc:343] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:02.785241  5640 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:02.862905  5640 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:02.789966  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:02.782995  5724 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:02.871131  5146 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:02.872808  5271 tablet_bootstrap.cc:654] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.889828  5724 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:02.890923  5716 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:02.891423  5269 tablet_bootstrap.cc:654] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:02.891909  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 128573us
W20260709 12:23:02.924654  5730 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:23:02.968307  4965 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 207142us
W20260709 12:23:02.971874  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48191us
I20260709 12:23:02.974200  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:02.974300  4967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 206361us
I20260709 12:23:02.975467  5224 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
I20260709 12:23:02.877641  5550 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:03.068432  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 74797.6us
I20260709 12:23:03.070997  5041 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:03.106710  5271 tablet_bootstrap.cc:492] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:03.107333  5271 ts_tablet_manager.cc:1403] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.452s	user 0.017s	sys 0.003s
I20260709 12:23:03.111809  5271 raft_consensus.cc:359] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:03.115469  5271 raft_consensus.cc:385] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.115887  5271 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.116616  5271 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:03.117431  5724 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.118104  5724 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.118602  5724 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.120152  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:03.120978  5145 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:23:03.113868  5220 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
W20260709 12:23:03.122152  5729 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 12:23:03.123927  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:03.129841  5550 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:03.118863  5271 ts_tablet_manager.cc:1434] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
W20260709 12:23:03.137115  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 66217.1us
W20260709 12:23:03.205196  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 235096us
I20260709 12:23:03.230643  5271 tablet_bootstrap.cc:492] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:03.270881  5722 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.281920  5041 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:23:03.285583  1522 debug-util.cc:398] Leaking SignalData structure 0x7b080000fc00 after lost signal to thread 5115
I20260709 12:23:03.293342  4967 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:03.257095  5640 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.300853  5272 tablet_bootstrap.cc:492] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:03.302067  5756 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.301497  5272 ts_tablet_manager.cc:1403] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.676s	user 0.020s	sys 0.012s
I20260709 12:23:03.302325  5550 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.309645  5550 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.310032  5550 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.306039  5640 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:03.292476  5722 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.352877  5722 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:03.356312  5272 raft_consensus.cc:359] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.357273  5272 raft_consensus.cc:385] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.357577  5272 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.371562  5269 tablet_bootstrap.cc:492] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:03.356380  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:03.372215  5269 ts_tablet_manager.cc:1403] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.693s	user 0.017s	sys 0.003s
I20260709 12:23:03.372552  5220 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.375404  5269 raft_consensus.cc:359] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:03.376176  5269 raft_consensus.cc:385] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.376456  5269 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.377226  5269 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:03.359548  5756 raft_consensus.cc:697] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:03.379215  5269 ts_tablet_manager.cc:1434] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:03.379137  5756 consensus_queue.cc:237] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:03.380578  5269 tablet_bootstrap.cc:492] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:23:03.384338  5724 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:03.386229  5022 debug-util.cc:398] Leaking SignalData structure 0x7b080037d8c0 after lost signal to thread 1520
W20260709 12:23:03.387637  5022 debug-util.cc:398] Leaking SignalData structure 0x7b0800162ca0 after lost signal to thread 4933
I20260709 12:23:03.390734  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:03.393095  5271 tablet_bootstrap.cc:654] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.365801  4997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:03.399595  5269 tablet_bootstrap.cc:654] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:03.407186  5115 leader_election.cc:343] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:03.403653  5022 debug-util.cc:398] Leaking SignalData structure 0x7b0800084a60 after lost signal to thread 5025
I20260709 12:23:03.365392  5550 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:03.443002  5741 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.443569  5741 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.446967  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:03.448489  5190 leader_election.cc:343] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.451081  5399 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.451642  5399 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:03.456964  5022 debug-util.cc:398] Leaking SignalData structure 0x7b08001d4380 after lost signal to thread 5100
I20260709 12:23:03.457970  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:03.458420  5022 debug-util.cc:398] Leaking SignalData structure 0x7b08002cd460 after lost signal to thread 5174
I20260709 12:23:03.458700  5143 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 0.
W20260709 12:23:03.459734  5022 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc500 after lost signal to thread 5248
I20260709 12:23:03.460004  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:03.460711  5146 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.460884  4997 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.461601  5399 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:03.463599  5741 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:03.468117  5190 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:03.468398  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:03.468969  5075 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.467418  4998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:03.473528  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 180823us
I20260709 12:23:03.480227  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:03.481361  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:03.443585  5550 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:03.483747  5189 leader_election.cc:343] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.495477  5732 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:03.496349  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:03.482064  5741 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.489166  5745 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:03.516685  5075 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
W20260709 12:23:03.509724  5189 leader_election.cc:343] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.517513  5189 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:03.521677  5745 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.524278  5745 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:03.526194  5146 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:03.526996  5546 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.527537  4967 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:03.503590  5741 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.534325  5741 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.496253  5732 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:03.534178  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:03.536172  5640 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:03.536628  5732 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:03.506419  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:03.543848  5001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:03.559979  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:03.561136  5271 tablet_bootstrap.cc:492] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:03.561786  5271 ts_tablet_manager.cc:1403] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.356s	user 0.016s	sys 0.009s
I20260709 12:23:03.556715  5741 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.562417  5716 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:03.562762  5219 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 1: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:23:03.566262  5042 leader_election.cc:343] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.570284  5741 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:03.571807  5269 tablet_bootstrap.cc:492] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:03.572533  5269 ts_tablet_manager.cc:1403] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.192s	user 0.013s	sys 0.015s
W20260709 12:23:03.573853  4965 leader_election.cc:343] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.574002  5724 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:03.576344  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:03.577236  5221 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.583196  4994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:03.565104  5271 raft_consensus.cc:359] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.588430  5271 raft_consensus.cc:385] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.588879  5271 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.589434  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:03.590375  5142 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:03.588433  5189 leader_election.cc:343] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.548771  5550 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.513834  5220 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:03.589965  5271 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.606606  4965 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:03.607780  4965 leader_election.cc:343] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.614444  5114 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:03.615559  5686 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:03.617223  5738 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.617890  5738 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:03.618940  5272 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.623890  5271 ts_tablet_manager.cc:1434] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.062s	user 0.008s	sys 0.000s
I20260709 12:23:03.626055  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:03.626991  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:03.629483  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:03.618805  5738 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:03.607190  5550 raft_consensus.cc:697] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:03.640986  5272 ts_tablet_manager.cc:1434] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.333s	user 0.012s	sys 0.000s
I20260709 12:23:03.643054  4899 catalog_manager.cc:5697] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.645159  5269 raft_consensus.cc:359] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:03.646677  5269 raft_consensus.cc:385] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.647003  5269 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.648605  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:03.651805  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:03.651002  4898 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43218 (request call id 66) took 1047 ms. Trace:
I20260709 12:23:03.652801  4898 rpcz_store.cc:276] 0709 12:23:02.608094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.608371 (+   277us) service_pool.cc:224] Handling call
0709 12:23:03.650928 (+1042557us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142798,"reported_tablets":1}
I20260709 12:23:03.654470  5271 tablet_bootstrap.cc:492] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:03.656590  5272 tablet_bootstrap.cc:492] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:03.658444  4899 catalog_manager.cc:5697] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:03.656385  5269 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:03.666771  4895 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43234 (request call id 68) took 1134 ms. Trace:
I20260709 12:23:03.668161  4895 rpcz_store.cc:276] 0709 12:23:02.532582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.532751 (+   169us) service_pool.cc:224] Handling call
0709 12:23:03.666530 (+1133779us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:02.690529 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:02.693081 (+  2552us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:02.693139 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:23:02.693171 (+    32us) tablet.cc:662] Decoding operations
0709 12:23:02.693770 (+   599us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:02.693902 (+   132us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:02.693953 (+    51us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:23:02.694253 (+   300us) tablet.cc:701] Row locks acquired
0709 12:23:02.694283 (+    30us) write_op.cc:260] PREPARE: finished
0709 12:23:02.694534 (+   251us) write_op.cc:270] Start()
0709 12:23:02.694711 (+   177us) write_op.cc:276] Timestamp: P: 1783599782694440 usec, L: 0
0709 12:23:02.694756 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 12:23:02.695875 (+  1119us) log.cc:844] Serialized 537 byte log entry
0709 12:23:02.886002 (+190127us) op_driver.cc:464] REPLICATION: finished
0709 12:23:03.066789 (+180787us) write_op.cc:301] APPLY: starting
0709 12:23:03.085957 (+ 19168us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:03.097604 (+ 11647us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:03.125276 (+ 27672us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:03.125583 (+   307us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:03.125753 (+   170us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:23:03.125806 (+    53us) write_op.cc:312] APPLY: finished
0709 12:23:03.128804 (+  2998us) log.cc:844] Serialized 22 byte log entry
0709 12:23:03.129026 (+   222us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:03.129129 (+   103us) write_op.cc:454] Released schema lock
0709 12:23:03.129177 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:03.129642 (+   465us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:03.129676 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":22311,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":109861,"apply.run_cpu_time_us":2558,"apply.run_wall_time_us":123175,"num_ops":1,"prepare.queue_time_us":37454,"prepare.run_cpu_time_us":25089,"prepare.run_wall_time_us":46235,"raft.queue_time_us":68516,"raft.run_cpu_time_us":3967,"raft.run_wall_time_us":42295,"replication_time_us":191012,"thread_start_us":184307,"threads_started":4,"wal-append.queue_time_us":82118}]]}
W20260709 12:23:03.688149  5042 leader_election.cc:343] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.705498  5550 consensus_queue.cc:237] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:03.706321  5269 ts_tablet_manager.cc:1434] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.133s	user 0.007s	sys 0.003s
I20260709 12:23:03.707484  5269 tablet_bootstrap.cc:492] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:23:03.725673  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:03.727808  4966 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:03.728855  5640 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.729568  5640 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.731976  5640 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:03.732641  5269 tablet_bootstrap.cc:654] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.734222  5640 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:03.734156  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:03.735174  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:03.732066  5272 tablet_bootstrap.cc:654] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.743578  4966 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:03.753151  5758 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.759276  5271 tablet_bootstrap.cc:654] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.783694  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:03.786696  4966 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:03.788445  5758 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.788637  5142 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:03.789337  5693 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.791110  5190 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:03.797427  5221 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:03.798951  5041 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:03.800434  5724 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.800952  5724 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:03.802109  5724 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:03.805229  5546 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.805855  5546 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:03.806783  5546 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.821110  5718 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.822997  5718 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.868223  5269 tablet_bootstrap.cc:492] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:03.868857  5269 ts_tablet_manager.cc:1403] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.162s	user 0.015s	sys 0.001s
I20260709 12:23:03.881373  4993 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.885133  5718 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:03.887949  4995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:03.888578  5272 tablet_bootstrap.cc:492] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:03.889187  5272 ts_tablet_manager.cc:1403] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.233s	user 0.013s	sys 0.008s
I20260709 12:23:03.894032  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:03.894887  5221 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
W20260709 12:23:03.900379  5115 leader_election.cc:343] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.906946  5272 raft_consensus.cc:359] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.908129  5272 raft_consensus.cc:385] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.908732  5272 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.908005  5269 raft_consensus.cc:359] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:03.913942  5269 raft_consensus.cc:385] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.914428  5269 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.915633  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:03.923091  5718 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.924223  5718 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.924650  5718 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.925737  5272 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.916123  5636 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:23:03.935587  5755 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:03.928951  5269 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:03.951074  5272 ts_tablet_manager.cc:1434] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.062s	user 0.012s	sys 0.000s
I20260709 12:23:03.958281  5269 ts_tablet_manager.cc:1434] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.089s	user 0.011s	sys 0.000s
I20260709 12:23:03.972265  5271 tablet_bootstrap.cc:492] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:03.972316  5272 tablet_bootstrap.cc:492] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:03.972904  5271 ts_tablet_manager.cc:1403] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.326s	user 0.014s	sys 0.018s
I20260709 12:23:03.977416  5269 tablet_bootstrap.cc:492] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f: Bootstrap starting.
I20260709 12:23:03.981458  5718 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:03.983786  5718 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:03.976157  5271 raft_consensus.cc:359] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:03.985214  5271 raft_consensus.cc:385] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.994963  4899 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43222 (request call id 72) took 1332 ms. Trace:
I20260709 12:23:03.985477  5271 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.997865  5271 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:04.004135  4997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:04.005719  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:04.006392  5223 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.007359  5724 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.007752  5271 ts_tablet_manager.cc:1434] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.002s
I20260709 12:23:04.007901  5724 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:04.009699  5546 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.010232  5546 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.012269  5546 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:04.013801  5115 leader_election.cc:343] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.018082  5223 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:04.016120  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:04.016120  4997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:04.022903  5190 leader_election.cc:343] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.023056  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:04.023911  5189 leader_election.cc:343] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.025645  5189 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:03.995507  4899 rpcz_store.cc:276] 0709 12:23:02.663027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.663195 (+   168us) service_pool.cc:224] Handling call
0709 12:23:03.994873 (+1331678us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:03.750682 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:03.751537 (+   855us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:03.751589 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:23:03.751618 (+    29us) tablet.cc:662] Decoding operations
0709 12:23:03.752217 (+   599us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:03.752329 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:03.752373 (+    44us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:23:03.752822 (+   449us) tablet.cc:701] Row locks acquired
0709 12:23:03.752848 (+    26us) write_op.cc:260] PREPARE: finished
0709 12:23:03.753064 (+   216us) write_op.cc:270] Start()
0709 12:23:03.753217 (+   153us) write_op.cc:276] Timestamp: P: 1783599783753008 usec, L: 0
0709 12:23:03.753251 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:23:03.763173 (+  9922us) log.cc:844] Serialized 922 byte log entry
0709 12:23:03.841706 (+ 78533us) op_driver.cc:464] REPLICATION: finished
0709 12:23:03.914470 (+ 72764us) write_op.cc:301] APPLY: starting
0709 12:23:03.916915 (+  2445us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:03.939758 (+ 22843us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:03.941809 (+  2051us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:03.942251 (+   442us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:03.942442 (+   191us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:23:03.942497 (+    55us) write_op.cc:312] APPLY: finished
0709 12:23:03.945668 (+  3171us) log.cc:844] Serialized 28 byte log entry
0709 12:23:03.947898 (+  2230us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:03.950786 (+  2888us) write_op.cc:454] Released schema lock
0709 12:23:03.954421 (+  3635us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:03.971520 (+ 17099us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:03.971561 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":72694,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":32317,"apply.run_cpu_time_us":2758,"apply.run_wall_time_us":97063,"num_ops":2,"prepare.queue_time_us":82963,"prepare.run_cpu_time_us":4128,"prepare.run_wall_time_us":17888,"replication_time_us":84460,"thread_start_us":56948,"threads_started":2}]]}
I20260709 12:23:04.031095  5271 tablet_bootstrap.cc:492] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:04.024605  5724 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:04.027899  5718 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.028978  5569 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.042920  4898 catalog_manager.cc:5697] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.047649  5693 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.048300  5693 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:04.034760  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:04.050688  5693 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:04.051105  5146 raft_consensus.cc:2393] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 1: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:04.052636  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:04.053524  5221 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 1: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:04.055699  4898 catalog_manager.cc:5697] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.032521  4992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:04.054811  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:04.064492  5718 raft_consensus.cc:697] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:04.070473  5041 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:04.075227  5755 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.085155  5114 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:04.082756  5738 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.087677  5738 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.090138  5738 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:04.095049  5272 tablet_bootstrap.cc:654] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.096196  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:04.095556  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:04.101219  5145 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 1: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:04.102603  5718 consensus_queue.cc:237] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.107750  5759 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.109965  4966 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:04.112536  5758 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.119621  4895 catalog_manager.cc:5697] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 1 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.122083  4895 catalog_manager.cc:5697] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:23:04.139822  5758 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.130928  5702 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:23:04.175484  5550 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:04.195215  5269 tablet_bootstrap.cc:654] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.204602  5755 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.209210  5271 tablet_bootstrap.cc:654] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.223879  4897 catalog_manager.cc:5697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.248571  5271 tablet_bootstrap.cc:492] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:04.249238  5271 ts_tablet_manager.cc:1403] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.221s	user 0.014s	sys 0.000s
I20260709 12:23:04.264341  5550 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:04.284700  5272 tablet_bootstrap.cc:492] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:04.285295  5272 ts_tablet_manager.cc:1403] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.313s	user 0.010s	sys 0.018s
I20260709 12:23:04.289115  5272 raft_consensus.cc:359] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.290045  5272 raft_consensus.cc:385] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.290410  5272 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.308146  5636 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:23:04.317356  5271 raft_consensus.cc:359] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.318178  5271 raft_consensus.cc:385] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.318590  5271 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.320463  5271 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.302737  5272 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.328147  5271 ts_tablet_manager.cc:1434] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.079s	user 0.006s	sys 0.000s
I20260709 12:23:04.324004  5550 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.334081  5550 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.336658  5550 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:04.343219  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:04.345120  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:04.346140  5223 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 1: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:04.347501  4966 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:04.348361  5550 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.349040  5269 tablet_bootstrap.cc:492] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f: No bootstrap required, opened a new log
I20260709 12:23:04.349642  5269 ts_tablet_manager.cc:1403] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent bootstrapping tablet: real 0.373s	user 0.021s	sys 0.004s
I20260709 12:23:04.352840  5269 raft_consensus.cc:359] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:04.355880  5269 raft_consensus.cc:385] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.356541  5269 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.300343  5759 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.362865  5693 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.363358  5272 ts_tablet_manager.cc:1434] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.078s	user 0.006s	sys 0.001s
I20260709 12:23:04.363474  5693 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.364585  5272 tablet_bootstrap.cc:492] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:04.374083  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:04.375021  5223 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 0.
I20260709 12:23:04.361299  5745 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:23:04.381637  5693 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:04.349064  5758 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:23:04.394342  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:04.394627  5755 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.395313  5755 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:04.398604  5755 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:04.398826  5515 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.399358  5515 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.399724  5515 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.401006  5550 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.401767  5269 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:04.404511  5269 ts_tablet_manager.cc:1434] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.000s
I20260709 12:23:04.406744  5271 tablet_bootstrap.cc:492] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Bootstrap starting.
I20260709 12:23:04.415455  4992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:04.415983  4993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:04.416178  5755 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.417299  5755 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:04.419225  5041 leader_election.cc:343] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.429112  4898 catalog_manager.cc:5697] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:04.428509  5115 leader_election.cc:343] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.418110  5716 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:04.439785  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:04.440239  5744 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:23:04.446023  5042 leader_election.cc:343] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.446635  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:04.448031  5636 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.454815  4992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:04.456482  5272 tablet_bootstrap.cc:654] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.458841  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:04.463524  5223 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 0.
W20260709 12:23:04.461735  5041 leader_election.cc:343] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.466387  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:04.470124  5636 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.470624  5636 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.471024  5636 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.453290  5755 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:04.517773  5515 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.520196  5515 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:04.531005  4992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:04.534761  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:04.535566  5219 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:04.547190  5115 leader_election.cc:343] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.554562  5271 tablet_bootstrap.cc:654] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.584332  5693 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:04.607285  5219 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:04.611442  5272 tablet_bootstrap.cc:492] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:04.612154  5272 ts_tablet_manager.cc:1403] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.248s	user 0.012s	sys 0.010s
I20260709 12:23:04.616039  5272 raft_consensus.cc:359] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:04.616891  5272 raft_consensus.cc:385] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.617169  5272 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.617877  5272 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:04.619812  5272 ts_tablet_manager.cc:1434] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:04.620882  5272 tablet_bootstrap.cc:492] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:04.594918  5636 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:04.629611  5272 tablet_bootstrap.cc:654] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.631174  5636 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:04.631388  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:04.632956  4993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:04.634824  5271 tablet_bootstrap.cc:492] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: No bootstrap required, opened a new log
I20260709 12:23:04.643137  5271 ts_tablet_manager.cc:1403] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Time spent bootstrapping tablet: real 0.237s	user 0.011s	sys 0.004s
I20260709 12:23:04.644586  5693 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.645208  5693 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:04.646130  5693 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.646294  5271 raft_consensus.cc:359] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:04.647073  5271 raft_consensus.cc:385] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.647360  5271 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.648458  5271 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:04.652988  5272 tablet_bootstrap.cc:492] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:04.653628  5272 ts_tablet_manager.cc:1403] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.008s
I20260709 12:23:04.658916  5271 ts_tablet_manager.cc:1434] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:23:04.660537  5272 raft_consensus.cc:359] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:04.661693  5272 raft_consensus.cc:385] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.662057  5272 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
W20260709 12:23:04.663951  5041 leader_election.cc:343] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.662765  5272 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:04.670243  5272 ts_tablet_manager.cc:1434] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:23:04.671470  5272 tablet_bootstrap.cc:492] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:04.680449  5272 tablet_bootstrap.cc:654] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.687902  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:04.688800  5219 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.701678  5515 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:04.728901  5272 tablet_bootstrap.cc:492] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: No bootstrap required, opened a new log
I20260709 12:23:04.731088  5272 ts_tablet_manager.cc:1403] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.060s	user 0.011s	sys 0.001s
I20260709 12:23:04.730190  5219 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:04.734773  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:04.735958  5636 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.736469  5636 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:04.743247  5272 raft_consensus.cc:359] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.744220  5272 raft_consensus.cc:385] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.744583  5272 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.745419  5272 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:04.748761  5272 ts_tablet_manager.cc:1434] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:23:04.737291  5636 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:04.762143  4898 catalog_manager.cc:5697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.773756  4895 catalog_manager.cc:5697] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.795258  5758 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.795866  5758 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:04.798343  5758 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:04.800927  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:04.801718  5145 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:23:04.803776  4967 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:04.804618  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:04.805363  5071 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 0.
I20260709 12:23:04.807924  5550 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.808465  5550 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.808826  5550 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.832363  5550 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:04.835808  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:04.837188  5145 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.840687  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:04.842262  5071 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:04.844122  4946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:04.836539  5550 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:04.852126  5145 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:04.853670  4967 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:04.854640  5550 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.860466  5071 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:04.855149  5550 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:04.863425  5550 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:04.891045  4897 catalog_manager.cc:5697] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.030611  5758 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.031183  5758 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:05.035351  5758 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:05.036120  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:05.036921  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:05.037258  5218 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 1: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:05.039628  4966 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:05.040693  5758 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.046273  5758 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:05.088129  5758 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.088704  5758 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:05.091709  5758 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:05.092837  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:05.093886  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:05.094853  5219 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 1: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:05.096026  4966 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:05.096648  5758 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.122390  5758 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:05.166501  5693 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:05.238399  5717 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:05.272748  5758 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:05.294308  5550 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:05.319778  5636 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:05.354554  5739 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.619719  1517 tablet_server.cc:179] TabletServer@127.1.123.65:0 shutting down...
W20260709 12:23:06.627873  5041 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:23:06.652418  5041 consensus_peers.cc:597] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:06.655766  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:06.657763  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:06.664320  5189 consensus_peers.cc:597] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:06.666889  5115 consensus_peers.cc:597] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.694737  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:06.696185  1517 tablet_replica.cc:333] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.697324  1517 raft_consensus.cc:2243] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.698032  1517 raft_consensus.cc:2272] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.703312  1517 tablet_replica.cc:333] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.704236  1517 raft_consensus.cc:2243] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.704766  1517 raft_consensus.cc:2272] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.707171  1517 tablet_replica.cc:333] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.707854  1517 raft_consensus.cc:2243] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.708370  1517 raft_consensus.cc:2272] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.710901  1517 tablet_replica.cc:333] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.711585  1517 raft_consensus.cc:2243] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:06.712514  1517 raft_consensus.cc:2272] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.714864  1517 tablet_replica.cc:333] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.716998  1517 raft_consensus.cc:2243] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:06.718128  1517 raft_consensus.cc:2272] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.718492  5041 consensus_peers.cc:597] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.720543  1517 tablet_replica.cc:333] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.721616  1517 raft_consensus.cc:2243] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.722203  1517 raft_consensus.cc:2272] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.724311  1517 tablet_replica.cc:333] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.724972  1517 raft_consensus.cc:2243] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.725466  1517 raft_consensus.cc:2272] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.727384  1517 tablet_replica.cc:333] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.728032  1517 raft_consensus.cc:2243] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.728475  1517 raft_consensus.cc:2272] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.730424  1517 tablet_replica.cc:333] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.731096  1517 raft_consensus.cc:2243] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.731565  1517 raft_consensus.cc:2272] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.733415  1517 tablet_replica.cc:333] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.734452  1517 raft_consensus.cc:2243] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.735044  1517 raft_consensus.cc:2272] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.738965  1517 tablet_replica.cc:333] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.741103  1517 raft_consensus.cc:2243] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:06.742261  1517 raft_consensus.cc:2272] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.745014  1517 tablet_replica.cc:333] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.746066  1517 raft_consensus.cc:2243] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.746666  1517 raft_consensus.cc:2272] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.750424  1517 tablet_replica.cc:333] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.751266  1517 raft_consensus.cc:2243] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.751878  1517 raft_consensus.cc:2272] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.754797  1517 tablet_replica.cc:333] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.755707  1517 raft_consensus.cc:2243] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.756269  1517 raft_consensus.cc:2272] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.756644  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.759210  1517 tablet_replica.cc:333] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.760040  1517 raft_consensus.cc:2243] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.760555  1517 raft_consensus.cc:2272] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.763396  1517 tablet_replica.cc:333] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.764355  1517 raft_consensus.cc:2243] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:23:06.764796  5115 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.765436  1517 raft_consensus.cc:2272] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.765686  5115 consensus_peers.cc:597] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.768749  1517 tablet_replica.cc:333] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.769812  1517 raft_consensus.cc:2243] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.770426  1517 raft_consensus.cc:2272] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.770684  5041 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.773820  1517 tablet_replica.cc:333] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.774755  1517 raft_consensus.cc:2243] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.775367  1517 raft_consensus.cc:2272] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.778055  1517 tablet_replica.cc:333] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.779018  1517 raft_consensus.cc:2243] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.779624  1517 raft_consensus.cc:2272] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.779726  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.782334  1517 tablet_replica.cc:333] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.783241  1517 raft_consensus.cc:2243] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:06.784335  1517 raft_consensus.cc:2272] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.787563  1517 tablet_replica.cc:333] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.788451  1517 raft_consensus.cc:2243] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.789095  1517 raft_consensus.cc:2272] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.791898  1517 tablet_replica.cc:333] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.792825  1517 raft_consensus.cc:2243] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.793366  1517 raft_consensus.cc:2272] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.795979  5189 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.796370  1517 tablet_replica.cc:333] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.801321  1517 raft_consensus.cc:2243] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.803406  1517 raft_consensus.cc:2272] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.807087  1517 tablet_replica.cc:333] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.808115  1517 raft_consensus.cc:2243] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:06.809870  1517 raft_consensus.cc:2272] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.810490  5041 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.813000  1517 tablet_replica.cc:333] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.814695  1517 raft_consensus.cc:2243] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:06.815994  1517 raft_consensus.cc:2272] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.819605  1517 tablet_replica.cc:333] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.820515  1517 raft_consensus.cc:2243] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.821076  1517 raft_consensus.cc:2272] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.821198  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.823822  1517 tablet_replica.cc:333] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.825137  1517 raft_consensus.cc:2243] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.825734  1517 raft_consensus.cc:2272] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.826021  5189 consensus_peers.cc:597] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.828573  1517 tablet_replica.cc:333] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.829593  1517 raft_consensus.cc:2243] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.830173  1517 raft_consensus.cc:2272] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.833263  1517 tablet_replica.cc:333] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.834164  1517 raft_consensus.cc:2243] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.834718  1517 raft_consensus.cc:2272] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.837374  1517 tablet_replica.cc:333] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.838305  1517 raft_consensus.cc:2243] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.839098  1517 raft_consensus.cc:2272] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.842049  1517 tablet_replica.cc:333] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.842922  1517 raft_consensus.cc:2243] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:06.844029  1517 raft_consensus.cc:2272] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.844194  5115 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.847064  1517 tablet_replica.cc:333] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.848186  1517 raft_consensus.cc:2243] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.848922  1517 raft_consensus.cc:2272] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.852696  1517 tablet_replica.cc:333] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.853801  1517 raft_consensus.cc:2243] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.854532  1517 raft_consensus.cc:2272] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.858157  1517 tablet_replica.cc:333] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.859043  1517 raft_consensus.cc:2243] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:06.860194  1517 raft_consensus.cc:2272] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.862655  1517 tablet_replica.cc:333] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.863407  1517 raft_consensus.cc:2243] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:06.864379  1517 raft_consensus.cc:2272] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.866410  1517 tablet_replica.cc:333] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.867027  1517 raft_consensus.cc:2243] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.867587  1517 raft_consensus.cc:2272] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.869863  1517 tablet_replica.cc:333] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.870493  1517 raft_consensus.cc:2243] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.870949  1517 raft_consensus.cc:2272] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.873153  1517 tablet_replica.cc:333] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.873786  1517 raft_consensus.cc:2243] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.874249  1517 raft_consensus.cc:2272] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.876133  1517 tablet_replica.cc:333] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.876721  1517 raft_consensus.cc:2243] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.877197  1517 raft_consensus.cc:2272] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.879520  1517 tablet_replica.cc:333] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.880249  1517 raft_consensus.cc:2243] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.880785  1517 raft_consensus.cc:2272] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.883566  1517 tablet_replica.cc:333] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.884327  1517 raft_consensus.cc:2243] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:06.885411  1517 raft_consensus.cc:2272] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.887730  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.888253  1517 tablet_replica.cc:333] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.889174  1517 raft_consensus.cc:2243] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.889770  1517 raft_consensus.cc:2272] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.892673  1517 tablet_replica.cc:333] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.893378  1517 raft_consensus.cc:2243] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.893932  1517 raft_consensus.cc:2272] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.896657  1517 tablet_replica.cc:333] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.897307  1517 raft_consensus.cc:2243] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:06.898319  1517 raft_consensus.cc:2272] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:06.900864  1517 tablet_replica.cc:333] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: stopping tablet replica
I20260709 12:23:06.901495  1517 raft_consensus.cc:2243] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:06.902064  1517 raft_consensus.cc:2272] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:06.926916  5041 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:06.935694  5115 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:06.938621  5041 consensus_peers.cc:597] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:06.948143  5115 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:06.952284  5041 consensus_peers.cc:597] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:06.956096  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:06.965664  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:07.008742  5041 consensus_peers.cc:597] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:07.031282  5189 consensus_peers.cc:597] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:07.069816  5041 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:07.072687  5041 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:07.086601  5115 consensus_peers.cc:597] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:07.107856  5041 consensus_peers.cc:597] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:07.167326  5041 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:07.169608  5189 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:07.281728  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:07.753393  5717 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:07.754063  5717 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:07.756098  5717 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:07.757344  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:07.761422  5115 leader_election.cc:336] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:07.761904  5115 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:07.762609  5717 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:07.778172  5569 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:07.778879  5569 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:07.781293  5569 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:07.784137  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:07.787142  5189 leader_election.cc:336] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:07.792775  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:07.794278  5546 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:07.801919  5717 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:07.802580  5717 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:07.806137  5717 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:07.806947  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:07.812325  5115 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:07.812958  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:07.813885  5823 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:07.889212  5831 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:07.890004  5831 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:07.892220  5831 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:07.893610  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:07.896896  5041 leader_election.cc:336] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:07.897449  5041 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:07.898411  5831 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:07.928731  5823 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:07.929272  5823 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:07.931411  5823 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:07.932550  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:07.939960  5115 leader_election.cc:336] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:07.940651  5823 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:07.941257  5823 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:07.944710  5823 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:07.947558  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:07.949685  5823 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:07.949858  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:07.955914  5115 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:07.956393  5115 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:07.957098  5823 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:07.986899  5831 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:07.987769  5831 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:07.990234  5831 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:07.991617  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:07.992415  5142 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:07.993458  5042 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:07.994289  5831 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:07.994626  5831 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:07.994937  5831 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:07.995348  5041 leader_election.cc:336] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:23:07.999914  1517 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 5022)
I20260709 12:23:08.002113  5831 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.004663  5831 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:08.005884  5717 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.006475  5717 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.008708  5717 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:08.010423  5041 leader_election.cc:336] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.010499  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:08.011310  5831 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.011915  5831 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.012537  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:08.013155  5142 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.014290  5831 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:08.015807  5115 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.016302  5115 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.017510  5717 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:08.020391  5041 leader_election.cc:336] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.021021  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:08.023089  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:08.024323  5142 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:08.026500  5831 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:08.029639  5042 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.031411  5831 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:08.032779  5831 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:08.033854  5831 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.052126  5546 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.057160  5831 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.053012  5546 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:08.058089  5831 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:08.060774  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:08.061398  5072 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:08.063383  4897 catalog_manager.cc:5697] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 2, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 2 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.064989  5188 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:08.065037  5831 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:08.065873  5569 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:08.066232  5569 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.066593  5569 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.069378  5831 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.070150  5831 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.072475  5831 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:08.078079  5041 leader_election.cc:336] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.079440  5546 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:08.080368  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:08.081707  5189 leader_election.cc:336] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:23:08.082912  5041 leader_election.cc:336] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.089411  5829 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.090256  5829 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:08.093852  5829 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:08.097007  5831 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.097640  5831 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.099854  5831 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:08.100349  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:08.105933  5041 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:23:08.118489  5189 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.126205  5190 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:08.128911  5829 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:08.132131  5831 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.132813  5831 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.135519  5831 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:08.109133  5569 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:08.143802  5041 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.144037  5219 raft_consensus.cc:2393] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:08.143813  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:08.147563  5145 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:08.152287  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.153594  5831 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.156342  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:08.157888  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:08.158581  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:08.159487  5143 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:08.158605  5142 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:08.158478  5569 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:08.167105  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.168540  5819 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:08.168597  5042 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.169180  5819 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.169739  5819 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.171617  5744 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:08.172061  5744 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.172394  5744 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.192976  5744 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.206700  5831 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:08.208636  5839 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.208936  5042 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.209185  5839 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:08.211174  5823 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.211982  5823 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.214745  5823 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:08.217036  5839 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:08.218792  5115 leader_election.cc:336] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.238835  5744 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:08.239563  5744 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:08.239943  5744 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.240322  5744 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.229202  5819 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:08.244422  5189 leader_election.cc:336] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.246381  5834 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.247116  5717 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.247678  5717 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.241063  5823 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.251422  5823 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:08.254570  5744 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.249943  5834 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:08.256740  5717 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:08.257861  5744 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:08.258658  5819 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:08.260257  5717 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.263640  5834 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:08.264474  5189 leader_election.cc:336] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.260795  5823 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:08.265332  5717 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:08.301983  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 106222us
I20260709 12:23:08.323459  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:08.326478  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:08.327708  5218 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
W20260709 12:23:08.329500  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 96170.4us
I20260709 12:23:08.333464  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:08.334673  5069 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:08.340871  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:08.325402  5219 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:08.342981  5819 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.344548  5819 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.347944  5831 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.348737  5819 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:08.350217  5823 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:08.350915  5823 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.351454  5823 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.345342  5717 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:08.356433  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.359454  5839 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.360616  5839 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:08.371942  5834 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:23:08.372370  5041 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.372996  5834 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.348949  5831 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.377863  5834 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.377310  5839 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:08.384480  5831 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:08.382345  5115 leader_election.cc:336] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:23:08.373268  5041 leader_election.cc:336] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:23:08.398774  5115 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:23:08.398494  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 67658.4us
W20260709 12:23:08.401579  5041 leader_election.cc:336] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:23:08.402660  5041 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.412680  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:08.412869  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:08.413805  5115 leader_election.cc:336] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.396903  5823 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:08.415729  5841 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:08.416354  5841 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.416923  5841 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.419000  5823 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:08.428166  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:08.429216  5143 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.429482  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:08.430460  5145 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.430982  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:08.431860  5147 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:08.427915  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:08.434988  5142 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.440173  5072 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:08.441654  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:08.433799  5834 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:08.443814  5188 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.442929  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:08.446274  5837 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:08.447507  5837 raft_consensus.cc:697] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:08.442767  5147 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:08.443814  5218 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:08.449059  5837 consensus_queue.cc:237] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:08.453760  5041 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.457918  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:08.459221  5115 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.458688  5834 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:08.459391  5072 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.466451  5143 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:08.470237  5219 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.491030  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:08.492412  5069 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:08.498140  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:08.498610  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.499315  5069 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:08.498811  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:08.501400  5823 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:08.501977  5823 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.502470  5823 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.502316  5076 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:08.507860  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.507916  5841 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:08.509898  5041 leader_election.cc:336] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:23:08.510802  5041 leader_election.cc:336] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.512584  5841 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:08.512701  5848 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:23:08.513922  5189 leader_election.cc:336] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.522082  5848 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.522621  5848 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.530602  5072 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:08.532130  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.534848  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.538390  5837 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:08.538623  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.539121  5837 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:08.539566  5831 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:08.540725  5042 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.540009  4897 catalog_manager.cc:5697] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 2, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.540860  5837 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:08.540022  5831 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.542033  5841 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:08.542122  5831 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.546736  5846 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:08.547622  5841 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:08.549631  5841 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.558351  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.560204  5847 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:08.560679  5847 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.561105  5847 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.562320  5145 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:08.572145  5114 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.573717  5845 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:08.574209  5845 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:08.574774  5845 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:08.570686  5041 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:08.576370  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:08.577293  5218 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.551134  5848 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.588136  5848 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:08.600277  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.607300  5848 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:08.591337  5847 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.611020  5848 raft_consensus.cc:697] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:08.612840  5219 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:08.613298  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:08.613265  5848 consensus_queue.cc:237] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.614630  5147 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:08.602105  5218 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:08.624346  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:08.624039  5843 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:08.625016  5147 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.624949  5823 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.627414  5823 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:08.600592  5831 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.631716  5072 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:08.631482  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:08.632277  5076 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.637828  5831 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:08.629001  5041 leader_election.cc:336] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.621332  5847 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:08.647058  5142 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:08.655062  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.656061  5843 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:08.656543  5843 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:08.657477  5843 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:08.658250  5845 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.660676  5845 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:08.660507  4897 catalog_manager.cc:5697] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 2, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.670331  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.672236  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:08.672947  5072 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:08.673430  5147 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:08.679361  5844 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:08.680003  5844 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:23:08.681998  5041 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.681290  5844 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:08.682514  4899 catalog_manager.cc:5697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 2, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 2 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.687885  5072 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:08.689330  5114 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:08.691179  5823 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:08.691799  5823 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:23:08.694190  5041 leader_election.cc:336] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.693197  5823 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.704095  5042 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.708133  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:08.709195  5846 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:08.709890  5846 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:08.711172  5846 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.714890  5848 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:08.721199  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:08.722349  5142 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 2: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:23:08.725597  5115 leader_election.cc:336] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.726495  5115 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:08.729039  5115 leader_election.cc:336] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.732488  5845 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:08.733724  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:08.734953  5841 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:08.725597  5848 raft_consensus.cc:697] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:08.745899  5076 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:08.747936  4899 catalog_manager.cc:5697] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 2, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 2 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.750079  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:08.750803  5115 leader_election.cc:336] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:08.750115  4899 catalog_manager.cc:5697] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 2, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 2 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:08.749632  5848 consensus_queue.cc:237] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.758031  4897 catalog_manager.cc:5697] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 2, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.755589  5823 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:08.777067  5823 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:08.781221  5823 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:08.799963  4899 catalog_manager.cc:5697] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 2, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 2 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.804668  4899 catalog_manager.cc:5697] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 2, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 2 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.808915  4897 catalog_manager.cc:5697] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 2, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:08.814219  4897 catalog_manager.cc:5697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 2, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:08.886857  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:08.900974  5076 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:08.902770  5837 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:08.958456  5115 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:08.963848  5041 consensus_peers.cc:597] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:08.999157  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:09.000771  1517 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 5022)
I20260709 12:23:09.024515  5076 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:09.026168  5829 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:09.039731  5041 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:09.060220  5115 consensus_peers.cc:597] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.089805  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.104499  5041 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:09.106684  5041 consensus_peers.cc:597] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.147233  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:23:09.149358  5145 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:09.150427  5076 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:09.150859  5848 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:09.151779  5823 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:09.152424  5041 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:23:09.173274  5147 raft_consensus.cc:1275] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:09.176317  5846 consensus_queue.cc:1048] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:09.202898  5142 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:09.203735  5219 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:09.213511  5041 consensus_peers.cc:597] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:09.217252  5219 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:09.233125  5848 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:09.235539  5846 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:09.240746  5847 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:09.259241  5115 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:09.269652  5041 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:09.270278  5041 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.271881  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.304932  5115 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:23:09.307708  5076 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:09.308845  5041 consensus_peers.cc:597] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:23:09.311082  5847 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:09.314133  5041 consensus_peers.cc:597] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.330871  5115 consensus_peers.cc:597] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:09.331503  5115 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.350075  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.356197  5189 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.350569  5115 consensus_peers.cc:597] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.367712  5115 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:09.391054  5145 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:09.392613  5846 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:09.418605  5189 consensus_peers.cc:597] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.424232  5041 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.432415  5115 consensus_peers.cc:597] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.433288  5189 consensus_peers.cc:597] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.439754  5115 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.439846  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.454440  5189 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.464435  5041 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.472339  5041 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.496088  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.539142  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.564628  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.565734  5041 consensus_peers.cc:597] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.576799  5041 consensus_peers.cc:597] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.654613  5189 consensus_peers.cc:597] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.655311  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:09.725309  5041 consensus_peers.cc:597] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:10.001374  1517 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 5022)
W20260709 12:23:10.096338  5041 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:23:10.495978  5844 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.496639  5844 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:10.499078  5844 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:10.500237  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:10.501082  5072 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:10.502332  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:10.503283  5844 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:23:10.503482  5115 leader_election.cc:336] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:10.503705  5844 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.504127  5844 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:10.515659  5844 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:10.518386  5844 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:10.518625  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:10.519383  5076 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:10.527606  5076 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
W20260709 12:23:10.528231  5115 leader_election.cc:336] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:23:10.532824  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:10.534145  5844 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:10.534775  5844 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:10.536280  5844 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:10.567888  4897 catalog_manager.cc:5697] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 3, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 3 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:10.963347  5076 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:10.964926  5844 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:10.996239  5041 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:10.998541  5115 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:11.002156  1517 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 5022)
W20260709 12:23:11.176705  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:11.245460  5115 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.292023  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:11.298198  5041 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:11.495613  5041 consensus_peers.cc:597] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:11.532446  5041 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:11.561100  5041 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.576196  5115 consensus_peers.cc:597] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.590018  5041 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.611203  5115 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:11.642745  5189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111) [suppressed 430 similar messages]
W20260709 12:23:11.649583  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.654951  5189 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.685182  5115 consensus_peers.cc:597] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:11.703622  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.704167  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.738363  5041 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.752710  5041 consensus_peers.cc:597] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.780846  5115 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:11.809451  5041 consensus_peers.cc:597] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.828832  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.834473  5115 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.862185  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.865868  5041 consensus_peers.cc:597] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.906220  5115 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.918143  5041 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:11.919628  5189 consensus_peers.cc:597] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.934667  5115 consensus_peers.cc:597] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:11.950736  5041 consensus_peers.cc:597] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:23:12.003127  1517 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 5022)
W20260709 12:23:12.042611  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.047619  5189 consensus_peers.cc:597] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.053912  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.058389  5041 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.061851  5189 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.069777  5115 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.110776  5041 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.111364  5189 consensus_peers.cc:597] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.141183  5115 consensus_peers.cc:597] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.145185  5041 consensus_peers.cc:597] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.200376  5041 consensus_peers.cc:597] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.225301  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.269510  5041 consensus_peers.cc:597] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.484480  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:12.828692  5041 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:23:12.958801  1517 tablet_server.cc:196] TabletServer@127.1.123.65:0 shutdown complete.
I20260709 12:23:13.080170  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:13.090873  5887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:13.091562  5888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:13.091583  5890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:13.092772  1517 server_base.cc:1061] running on GCE node
I20260709 12:23:13.094153  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:13.094461  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:13.094691  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599793094675 us; error 0 us; skew 500 ppm
I20260709 12:23:13.095274  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:13.101414  1517 webserver.cc:533] Webserver started at http://127.1.123.65:36141/ using document root <none> and password file <none>
I20260709 12:23:13.102166  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:13.102468  1517 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:23:13.108124  1517 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:23:13.120683  5896 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:13.121821  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:23:13.122215  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-0-root
uuid: "9847a7797ddc4df689a1ab0911af8785"
format_stamp: "Formatted at 2026-07-09 12:22:51 on dist-test-slave-08l4"
I20260709 12:23:13.122751  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:13.153189  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:13.155256  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:13.157574  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:13.163800  5903 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 12:23:13.433037  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:23:13.433418  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.271s	user 0.010s	sys 0.001s
I20260709 12:23:13.433840  1517 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:23:13.439587  5903 tablet_bootstrap.cc:492] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.460413  5903 tablet_bootstrap.cc:492] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.461627  5903 tablet_bootstrap.cc:492] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.462380  5903 ts_tablet_manager.cc:1403] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 12:23:13.465337  5903 raft_consensus.cc:359] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.466136  5903 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.467092  5903 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.471344  5903 ts_tablet_manager.cc:1434] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:23:13.473371  5903 tablet_bootstrap.cc:492] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.495515  5903 tablet_bootstrap.cc:492] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.496580  5903 tablet_bootstrap.cc:492] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.497337  5903 ts_tablet_manager.cc:1403] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.016s
I20260709 12:23:13.500191  5903 raft_consensus.cc:359] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.500964  5903 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.501729  5903 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.503671  5903 ts_tablet_manager.cc:1434] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:23:13.504477  5903 tablet_bootstrap.cc:492] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.521634  5903 tablet_bootstrap.cc:492] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.522651  5903 tablet_bootstrap.cc:492] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.523320  5903 ts_tablet_manager.cc:1403] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:23:13.525923  5903 raft_consensus.cc:359] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:13.526607  5903 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.527278  5903 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:13.529359  5903 ts_tablet_manager.cc:1434] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:23:13.530308  5903 tablet_bootstrap.cc:492] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.547475  5903 tablet_bootstrap.cc:492] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.548530  5903 tablet_bootstrap.cc:492] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.549244  5903 ts_tablet_manager.cc:1403] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 12:23:13.551476  5903 raft_consensus.cc:359] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:13.552167  5903 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.552788  5903 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:13.554348  5903 ts_tablet_manager.cc:1434] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:23:13.555009  5903 tablet_bootstrap.cc:492] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.571578  5903 tablet_bootstrap.cc:492] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.572650  5903 tablet_bootstrap.cc:492] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.573398  5903 ts_tablet_manager.cc:1403] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 12:23:13.576453  5903 raft_consensus.cc:359] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:13.577136  5903 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.577775  5903 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:13.579766  5903 ts_tablet_manager.cc:1434] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:13.580614  5903 tablet_bootstrap.cc:492] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
W20260709 12:23:13.585417  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:23:13.603629  5903 tablet_bootstrap.cc:492] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.604800  5903 tablet_bootstrap.cc:492] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.605562  5903 ts_tablet_manager.cc:1403] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.004s
I20260709 12:23:13.608386  5903 raft_consensus.cc:359] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:13.609140  5903 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.609913  5903 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:13.612818  5903 ts_tablet_manager.cc:1434] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
W20260709 12:23:13.613514  5115 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:23:13.613801  5903 tablet_bootstrap.cc:492] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
W20260709 12:23:13.637734  5041 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:23:13.648413  5903 tablet_bootstrap.cc:492] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.650853  5903 tablet_bootstrap.cc:492] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.651691  5903 ts_tablet_manager.cc:1403] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.012s
I20260709 12:23:13.654687  5903 raft_consensus.cc:359] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:13.655449  5903 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.656199  5903 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:13.658627  5903 ts_tablet_manager.cc:1434] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:13.659466  5903 tablet_bootstrap.cc:492] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.697985  5903 tablet_bootstrap.cc:492] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.699014  5903 tablet_bootstrap.cc:492] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.702366  5903 ts_tablet_manager.cc:1403] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.043s	user 0.026s	sys 0.012s
I20260709 12:23:13.705298  5903 raft_consensus.cc:359] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:13.706058  5903 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.706900  5903 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:13.711105  5903 ts_tablet_manager.cc:1434] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:13.712030  5903 tablet_bootstrap.cc:492] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.725653  5903 tablet_bootstrap.cc:492] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.726433  5903 tablet_bootstrap.cc:492] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.726976  5903 ts_tablet_manager.cc:1403] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.015s	user 0.005s	sys 0.007s
I20260709 12:23:13.728893  5903 raft_consensus.cc:359] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.729470  5903 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.730161  5903 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.732190  5903 ts_tablet_manager.cc:1434] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:23:13.733038  5903 tablet_bootstrap.cc:492] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.757197  5903 tablet_bootstrap.cc:492] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.758237  5903 tablet_bootstrap.cc:492] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.758874  5903 ts_tablet_manager.cc:1403] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.004s
I20260709 12:23:13.760811  5903 raft_consensus.cc:359] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.761340  5903 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.761965  5903 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.763563  5903 ts_tablet_manager.cc:1434] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:23:13.764333  5903 tablet_bootstrap.cc:492] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.769728  1517 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:23:13.770149  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.336s	user 0.274s	sys 0.010s
I20260709 12:23:13.780356  5903 tablet_bootstrap.cc:492] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.781195  5903 tablet_bootstrap.cc:492] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.781819  5903 ts_tablet_manager.cc:1403] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 12:23:13.784080  5903 raft_consensus.cc:359] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.784802  5903 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.785784  5903 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.788410  5903 ts_tablet_manager.cc:1434] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:13.789264  5903 tablet_bootstrap.cc:492] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
W20260709 12:23:13.817368  5041 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:23:13.821725  5903 tablet_bootstrap.cc:492] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.823151  5903 tablet_bootstrap.cc:492] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.824071  5903 ts_tablet_manager.cc:1403] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.000s
I20260709 12:23:13.827896  5903 raft_consensus.cc:359] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:13.828773  5903 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.830091  5903 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:13.832897  5903 ts_tablet_manager.cc:1434] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:13.834020  5903 tablet_bootstrap.cc:492] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.854184  5903 tablet_bootstrap.cc:492] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.855039  5903 tablet_bootstrap.cc:492] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.855985  5903 ts_tablet_manager.cc:1403] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 12:23:13.856237  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.65:37253
I20260709 12:23:13.856436  5973 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.65:37253 every 8 connection(s)
I20260709 12:23:13.858501  5903 raft_consensus.cc:359] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.859426  5903 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.860114  5903 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:13.862200  5903 ts_tablet_manager.cc:1434] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:13.863181  5903 tablet_bootstrap.cc:492] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.884423  5903 tablet_bootstrap.cc:492] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.885594  5903 tablet_bootstrap.cc:492] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.886302  5903 ts_tablet_manager.cc:1403] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 12:23:13.889322  5903 raft_consensus.cc:359] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:13.891103  5903 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.891868  5903 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:13.894824  5903 ts_tablet_manager.cc:1434] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:13.895788  5903 tablet_bootstrap.cc:492] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:13.916198  5974 heartbeater.cc:344] Connected to a master server at 127.1.123.126:38285
I20260709 12:23:13.916787  5974 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:13.917809  5974 heartbeater.cc:507] Master 127.1.123.126:38285 requested a full tablet report, sending...
W20260709 12:23:13.940858  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:23:13.943930  5115 consensus_peers.cc:597] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:23:13.949770  4899 ts_manager.cc:194] Re-registered known tserver with Master: 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:13.951550  5903 tablet_bootstrap.cc:492] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:13.952939  5903 tablet_bootstrap.cc:492] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:13.953958  5903 ts_tablet_manager.cc:1403] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.058s	user 0.029s	sys 0.016s
W20260709 12:23:13.958220  5041 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:13.959107  5903 raft_consensus.cc:359] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:13.963593  5115 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:13.968272  5903 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.977074  5903 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:13.980754  5903 ts_tablet_manager.cc:1434] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:23:13.981916  5903 tablet_bootstrap.cc:492] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.000613  5903 tablet_bootstrap.cc:492] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.001858  5903 tablet_bootstrap.cc:492] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.002602  5903 ts_tablet_manager.cc:1403] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 12:23:14.005707  5903 raft_consensus.cc:359] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.006505  5903 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.007650  5903 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.010156  5903 ts_tablet_manager.cc:1434] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:14.011135  5903 tablet_bootstrap.cc:492] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.014535  4899 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49944
I20260709 12:23:14.020418  5974 heartbeater.cc:499] Master 127.1.123.126:38285 was elected leader, sending a full tablet report...
W20260709 12:23:14.024125  5189 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:23:14.026773  5041 consensus_peers.cc:597] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:23:14.034503  5903 tablet_bootstrap.cc:492] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.036983  5903 tablet_bootstrap.cc:492] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
W20260709 12:23:14.038226  5115 consensus_peers.cc:597] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:14.037801  5903 ts_tablet_manager.cc:1403] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.009s
I20260709 12:23:14.044428  5903 raft_consensus.cc:359] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.045195  5903 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.047948  5903 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.050143  5903 ts_tablet_manager.cc:1434] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:23:14.050945  5903 tablet_bootstrap.cc:492] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.065867  5903 tablet_bootstrap.cc:492] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.067291  5903 tablet_bootstrap.cc:492] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.068037  5903 ts_tablet_manager.cc:1403] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 12:23:14.068312  5948 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:14.070856  5903 raft_consensus.cc:359] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:14.071712  5903 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.072482  5903 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:14.074766  5903 ts_tablet_manager.cc:1434] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:14.075596  5903 tablet_bootstrap.cc:492] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.100514  5903 tablet_bootstrap.cc:492] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.101677  5903 tablet_bootstrap.cc:492] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.104033  5903 ts_tablet_manager.cc:1403] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.000s
I20260709 12:23:14.106885  5903 raft_consensus.cc:359] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.108124  5903 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.108865  5903 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.111119  5903 ts_tablet_manager.cc:1434] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 12:23:14.111963  5903 tablet_bootstrap.cc:492] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.115950  5948 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:14.134526  5903 tablet_bootstrap.cc:492] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.135900  5903 tablet_bootstrap.cc:492] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.136732  5903 ts_tablet_manager.cc:1403] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.000s
I20260709 12:23:14.139612  5903 raft_consensus.cc:359] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:14.140421  5903 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.141316  5903 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:14.142807  5903 ts_tablet_manager.cc:1434] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:14.143698  5903 tablet_bootstrap.cc:492] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.164397  5903 tablet_bootstrap.cc:492] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.165519  5903 tablet_bootstrap.cc:492] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.166313  5903 ts_tablet_manager.cc:1403] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.000s
I20260709 12:23:14.171289  5903 raft_consensus.cc:359] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.171885  5903 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.172573  5903 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.175371  5903 ts_tablet_manager.cc:1434] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:14.176283  5903 tablet_bootstrap.cc:492] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.190464  5903 tablet_bootstrap.cc:492] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.191370  5903 tablet_bootstrap.cc:492] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.192054  5903 ts_tablet_manager.cc:1403] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.002s
I20260709 12:23:14.194623  5903 raft_consensus.cc:359] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.195173  5903 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.195796  5903 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.197696  5903 ts_tablet_manager.cc:1434] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.005s
I20260709 12:23:14.199007  5903 tablet_bootstrap.cc:492] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.201965  5948 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:14.214856  5946 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:14.216781  5903 tablet_bootstrap.cc:492] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.218833  5903 tablet_bootstrap.cc:492] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.219635  5903 ts_tablet_manager.cc:1403] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.011s
I20260709 12:23:14.222644  5903 raft_consensus.cc:359] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.223344  5903 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.224036  5903 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.233556  5903 ts_tablet_manager.cc:1434] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.004s
I20260709 12:23:14.234546  5903 tablet_bootstrap.cc:492] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
W20260709 12:23:14.253583  5041 consensus_peers.cc:597] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:14.262678  5903 tablet_bootstrap.cc:492] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.263896  5903 tablet_bootstrap.cc:492] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.265084  5903 ts_tablet_manager.cc:1403] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.008s
I20260709 12:23:14.268110  5903 raft_consensus.cc:359] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.268838  5903 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.269661  5903 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.272006  5903 ts_tablet_manager.cc:1434] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:14.272944  5903 tablet_bootstrap.cc:492] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
W20260709 12:23:14.281812  5041 consensus_peers.cc:597] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:14.298979  5903 tablet_bootstrap.cc:492] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.300226  5903 tablet_bootstrap.cc:492] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.301095  5903 ts_tablet_manager.cc:1403] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.012s
I20260709 12:23:14.301252  5948 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:14.303972  5903 raft_consensus.cc:359] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.304790  5903 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.305644  5903 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.308548  5903 ts_tablet_manager.cc:1434] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:14.309785  5903 tablet_bootstrap.cc:492] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
W20260709 12:23:14.332056  5189 consensus_peers.cc:597] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:14.334831  5903 tablet_bootstrap.cc:492] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.335759  5903 tablet_bootstrap.cc:492] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.336450  5903 ts_tablet_manager.cc:1403] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.007s
I20260709 12:23:14.338619  5903 raft_consensus.cc:359] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:14.339305  5903 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.340032  5903 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:14.342317  5903 ts_tablet_manager.cc:1434] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:23:14.343050  5903 tablet_bootstrap.cc:492] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.359522  5903 tablet_bootstrap.cc:492] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.360731  5903 tablet_bootstrap.cc:492] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.361413  5903 ts_tablet_manager.cc:1403] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.004s
I20260709 12:23:14.364085  5948 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:14.364239  5903 raft_consensus.cc:359] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.365109  5903 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.366106  5903 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.368461  5903 ts_tablet_manager.cc:1434] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:23:14.369392  5903 tablet_bootstrap.cc:492] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.375192  5946 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:14.388840  5041 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:14.391261  5903 tablet_bootstrap.cc:492] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.392436  5903 tablet_bootstrap.cc:492] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.393134  5903 ts_tablet_manager.cc:1403] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.000s
I20260709 12:23:14.395953  5903 raft_consensus.cc:359] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:14.396646  5903 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.397257  5903 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:14.399523  5903 ts_tablet_manager.cc:1434] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:23:14.400410  5903 tablet_bootstrap.cc:492] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.421052  5903 tablet_bootstrap.cc:492] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.422317  5903 tablet_bootstrap.cc:492] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.423282  5903 ts_tablet_manager.cc:1403] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.000s
I20260709 12:23:14.425806  5903 raft_consensus.cc:359] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.426343  5903 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.426905  5903 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.428853  5903 ts_tablet_manager.cc:1434] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:23:14.429786  5903 tablet_bootstrap.cc:492] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
W20260709 12:23:14.436478  5115 consensus_peers.cc:597] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:14.446902  5903 tablet_bootstrap.cc:492] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.448282  5903 tablet_bootstrap.cc:492] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.449084  5903 ts_tablet_manager.cc:1403] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:23:14.452452  5903 raft_consensus.cc:359] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.453230  5903 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.454130  5903 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.456275  5903 ts_tablet_manager.cc:1434] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:23:14.457320  5903 tablet_bootstrap.cc:492] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
W20260709 12:23:14.464363  5115 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:14.479044  5903 tablet_bootstrap.cc:492] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.480262  5903 tablet_bootstrap.cc:492] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.481074  5903 ts_tablet_manager.cc:1403] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.004s
I20260709 12:23:14.483748  5903 raft_consensus.cc:359] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:14.484531  5903 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.485297  5903 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:14.486491  5115 consensus_peers.cc:597] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:14.487798  5903 ts_tablet_manager.cc:1434] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:23:14.488757  5903 tablet_bootstrap.cc:492] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
W20260709 12:23:14.521467  5189 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:14.543561  5903 tablet_bootstrap.cc:492] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.544927  5903 tablet_bootstrap.cc:492] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.545796  5903 ts_tablet_manager.cc:1403] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.057s	user 0.028s	sys 0.024s
I20260709 12:23:14.548708  5903 raft_consensus.cc:359] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.549731  5903 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.550598  5903 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.553061  5903 ts_tablet_manager.cc:1434] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:14.554755  5903 tablet_bootstrap.cc:492] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
W20260709 12:23:14.562300  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 47160.9us
I20260709 12:23:14.600628  5903 tablet_bootstrap.cc:492] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.601869  5903 tablet_bootstrap.cc:492] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.602626  5903 ts_tablet_manager.cc:1403] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.048s	user 0.034s	sys 0.006s
I20260709 12:23:14.605598  5903 raft_consensus.cc:359] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:14.606612  5903 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.607712  5903 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:14.610999  5903 ts_tablet_manager.cc:1434] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.007s
I20260709 12:23:14.612385  5903 tablet_bootstrap.cc:492] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.637605  5946 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:14.655676  5189 consensus_peers.cc:597] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:23:14.675182  5903 tablet_bootstrap.cc:492] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.676715  5903 tablet_bootstrap.cc:492] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.677577  5903 ts_tablet_manager.cc:1403] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.066s	user 0.043s	sys 0.007s
I20260709 12:23:14.681924  5903 raft_consensus.cc:359] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.682709  5903 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.683867  5903 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.686343  5903 ts_tablet_manager.cc:1434] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:14.687286  5903 tablet_bootstrap.cc:492] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.710163  5903 tablet_bootstrap.cc:492] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.711076  5903 tablet_bootstrap.cc:492] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.711711  5903 ts_tablet_manager.cc:1403] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.005s
I20260709 12:23:14.714800  5903 raft_consensus.cc:359] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.715548  5903 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.716600  5903 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.720018  5903 ts_tablet_manager.cc:1434] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:14.721295  5903 tablet_bootstrap.cc:492] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.751569  5903 tablet_bootstrap.cc:492] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.752597  5903 tablet_bootstrap.cc:492] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.753439  5903 ts_tablet_manager.cc:1403] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.012s
I20260709 12:23:14.755785  5903 raft_consensus.cc:359] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.756536  5903 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.757207  5903 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.759593  5903 ts_tablet_manager.cc:1434] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:14.760426  5903 tablet_bootstrap.cc:492] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.775877  5903 tablet_bootstrap.cc:492] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.776870  5903 tablet_bootstrap.cc:492] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.777640  5903 ts_tablet_manager.cc:1403] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.017s	user 0.017s	sys 0.000s
I20260709 12:23:14.779672  5903 raft_consensus.cc:359] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.780222  5903 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.780777  5903 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.782691  5903 ts_tablet_manager.cc:1434] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:23:14.784020  5903 tablet_bootstrap.cc:492] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.807286  5903 tablet_bootstrap.cc:492] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.808240  5903 tablet_bootstrap.cc:492] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.808959  5903 ts_tablet_manager.cc:1403] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.003s
I20260709 12:23:14.811414  5903 raft_consensus.cc:359] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.812124  5903 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.812892  5903 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.814783  5903 ts_tablet_manager.cc:1434] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:23:14.815846  5903 tablet_bootstrap.cc:492] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.905573  5903 tablet_bootstrap.cc:492] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.908223  5903 tablet_bootstrap.cc:492] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.909090  5903 ts_tablet_manager.cc:1403] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.094s	user 0.044s	sys 0.043s
I20260709 12:23:14.911813  5903 raft_consensus.cc:359] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.912377  5903 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.913633  5903 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.916122  5903 ts_tablet_manager.cc:1434] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:14.917012  5903 tablet_bootstrap.cc:492] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.950344  5903 tablet_bootstrap.cc:492] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.951514  5903 tablet_bootstrap.cc:492] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.952330  5903 ts_tablet_manager.cc:1403] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.036s	user 0.030s	sys 0.004s
I20260709 12:23:14.955132  5903 raft_consensus.cc:359] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.955754  5903 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.956540  5903 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:14.958189  5903 ts_tablet_manager.cc:1434] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:23:14.959407  5903 tablet_bootstrap.cc:492] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:14.991520  5903 tablet_bootstrap.cc:492] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:14.992830  5903 tablet_bootstrap.cc:492] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:14.993690  5903 ts_tablet_manager.cc:1403] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.035s	user 0.019s	sys 0.000s
I20260709 12:23:14.996722  5903 raft_consensus.cc:359] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:14.997556  5903 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.998413  5903 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:15.000563  5903 ts_tablet_manager.cc:1434] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:15.001652  5903 tablet_bootstrap.cc:492] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:15.047876  5903 tablet_bootstrap.cc:492] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:15.048974  5903 tablet_bootstrap.cc:492] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:15.050226  5903 ts_tablet_manager.cc:1403] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.008s
I20260709 12:23:15.052687  5903 raft_consensus.cc:359] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:15.053566  5903 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.054301  5903 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:15.056180  5903 ts_tablet_manager.cc:1434] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:23:15.057196  5903 tablet_bootstrap.cc:492] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:15.081225  5903 tablet_bootstrap.cc:492] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:15.082397  5903 tablet_bootstrap.cc:492] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:15.083179  5903 ts_tablet_manager.cc:1403] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.004s
I20260709 12:23:15.085376  5903 raft_consensus.cc:359] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:15.086004  5903 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.086812  5903 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:15.090747  5903 ts_tablet_manager.cc:1434] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:15.092715  5903 tablet_bootstrap.cc:492] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:15.112149  5903 tablet_bootstrap.cc:492] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:15.113739  5903 tablet_bootstrap.cc:492] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:15.114403  5903 ts_tablet_manager.cc:1403] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 12:23:15.116941  5903 raft_consensus.cc:359] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:15.117671  5903 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.118427  5903 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:15.120543  5903 ts_tablet_manager.cc:1434] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 12:23:15.121387  5903 tablet_bootstrap.cc:492] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Bootstrap starting.
I20260709 12:23:15.138293  5903 tablet_bootstrap.cc:492] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:15.139190  5903 tablet_bootstrap.cc:492] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Bootstrap complete.
I20260709 12:23:15.139801  5903 ts_tablet_manager.cc:1403] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.004s
I20260709 12:23:15.142122  5903 raft_consensus.cc:359] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:15.142817  5903 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.143525  5903 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:15.145900  5903 ts_tablet_manager.cc:1434] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:23:15.146015  5946 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:15.370556  5947 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:15.481524  5947 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:15.557452  5947 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:23:19.327505  6044 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.329475  6044 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.334646  6044 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.396430  6053 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.397944  6053 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.400058  6054 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.615331  6054 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:19.617895  6053 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.598569  6057 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.412235  6044 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.622587  6044 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.626606  6044 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.629022  6044 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.630005  6044 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.634662  6054 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:19.635885  6054 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.636340  6056 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.636832  6054 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.637382  6056 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.639590  6054 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.639997  6056 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.619596  6053 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.635907  6055 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.644374  6053 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.644613  6055 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.647445  6053 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.648458  6055 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.643222  6056 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.642252  6054 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.653326  6053 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:19.656301  5190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.180s	user 0.003s	sys 0.010s
I20260709 12:23:19.656220  6053 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.659535  6053 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.652316  6054 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.664503  6047 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:19.662029  5190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.186s	user 0.004s	sys 0.010s
I20260709 12:23:19.665923  6047 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.659840  6055 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.670318  6055 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.672855  6055 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.650043  6056 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.675835  6056 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.663322  6053 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.678208  6057 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:19.678608  6053 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.681828  6053 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.683490  6054 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.669204  6047 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.681371  6057 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:19.689862  6058 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.690887  6056 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.695147  6056 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:19.698975  6056 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:19.700098  6058 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:19.713197  6054 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.691962  6055 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.713037  6056 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.717344  6053 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.717914  6056 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.718518  6053 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.721000  6056 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.715231  6054 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.724602  6059 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.735260  6059 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.738399  6057 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.739441  6057 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.742777  6057 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:19.746905  6059 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.731185  6055 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:19.731406  6049 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.723266  6058 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:19.723544  6054 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.721514  6053 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.756464  6049 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:19.758399  6055 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:19.780330  6067 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.781397  6067 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.783901  6061 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.785081  6061 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:19.788306  6061 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:19.784262  6067 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.818919  6062 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.819885  6062 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:19.759639  6049 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.855749  6062 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:19.861070  6055 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.862139  6055 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.863488  6053 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.864283  6053 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.865031  6054 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.865882  6054 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.866962  6053 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.864557  6055 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.868413  6054 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.868347  6060 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.869709  6059 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.869177  6056 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.883771  6059 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.890825  6059 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.891732  6056 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.893213  6060 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.872889  6053 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.901158  6053 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:19.903170  6060 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.900038  6056 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.904281  6053 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:19.910063  6067 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.911115  6067 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.913769  6049 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.916448  6049 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:19.927874  6067 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.925477  6049 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:19.930861  6071 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.931795  6071 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:19.934742  6067 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.935253  6049 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.935665  6067 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.936108  6049 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:19.938203  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 1.74783e+06us
I20260709 12:23:19.939412  6049 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.944057  6057 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.944195  6058 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.944913  6057 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:19.945022  6058 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:19.946250  6060 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.951993  6054 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.951512  6056 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.953738  6055 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.954027  6056 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:19.954612  6055 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:19.947125  6060 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:19.943810  6072 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.952324  6059 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.958858  6072 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:19.959102  6059 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:19.960572  6060 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.957453  6056 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:19.961313  6059 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:19.961501  6055 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:19.941732  6049 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:19.964646  6057 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:19.961300  6072 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:19.963519  6058 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:19.966163  6053 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:19.961385  6060 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.029100  6066 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:20.030608  6066 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:20.031243  6070 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.031987  6070 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.033718  6073 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:20.034457  6073 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.037055  6073 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:20.038395  6067 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:20.010664  6069 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.039660  6069 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:20.014187  6062 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.042215  6062 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:20.044646  6062 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.045802  6053 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.047677  6072 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:19.953778  6054 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.048576  6071 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:20.048420  6072 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.050769  6054 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:20.050930  6072 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.051461  6054 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.052295  6054 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.052783  6053 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.054750  6069 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:20.055421  6069 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.057803  6069 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:20.029091  6060 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.063109  6073 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:20.064175  6060 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.063977  6073 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.033298  6066 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.066403  6073 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:20.068361  6070 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:20.069648  6071 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.069775  6054 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.070665  6067 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:23:20.045248  5187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.570s	user 0.006s	sys 0.014s
I20260709 12:23:20.083382  6069 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:20.086076  5187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.611s	user 0.010s	sys 0.018s
I20260709 12:23:20.071568  6047 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.111160  6054 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.117756  6075 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.118070  6067 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:20.118319  6054 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.118803  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:20.120836  6053 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.107728  6070 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.134217  6070 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:20.135070  6067 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:20.084476  6060 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:20.136742  6070 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.118714  6075 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:20.125522  6061 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.133013  6053 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.110991  6049 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:20.128425  6069 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.139068  6056 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.139508  6055 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.153522  6054 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.153853  6063 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.139858  6047 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.154381  6054 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.154676  6063 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:20.155161  6054 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.156733  6047 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:20.139236  6059 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.157757  6059 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.153913  6055 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.151826  6078 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.162249  6054 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.163448  6078 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.164064  6073 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:20.164703  6073 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.139392  6072 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.166077  6072 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.166805  6073 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:20.167362  6073 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.160430  6059 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:20.137140  6071 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.146515  6067 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.175261  5942 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:20.162967  6069 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.181079  6044 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:20.481305  6077 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:20.546444  6077 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:20.178088  6060 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.548059  6069 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:20.544442  6083 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:20.237146  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:20.549305  6083 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:20.481277  6070 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.550486  6070 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.238535  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:20.553575  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.554338  5939 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:20.554574  6070 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:20.555776  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:20.556056  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.556530  5939 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:20.177901  6062 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.239660  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:20.558044  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.558513  5940 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:20.186028  6056 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.184304  6075 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:20.548249  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:20.232460  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:20.558854  5219 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:20.544665  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:20.561116  6056 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:20.560241  5945 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:20.557885  6062 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.560818  6083 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:20.185775  5844 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.561717  5946 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:20.549933  5947 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:20.179448  6066 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.304352  6073 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.481451  6047 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.332312  6063 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:20.181999  6071 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:20.179744  6061 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:20.900175  6066 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.902591  6061 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:20.903946  6047 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:20.906497  6047 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:20.907902  6066 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:20.912139  6073 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:20.915973  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.921737  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.922260  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.921216  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.923533  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.928788  5221 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:20.929301  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.930348  5217 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:20.187780  6078 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:20.409449  6082 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:20.932060  6082 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:20.935139  6082 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.207669  6072 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.186218  6067 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:20.937821  6067 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:20.551362  6049 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:20.565439  6077 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.569607  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.939010  5219 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:20.570495  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:20.549379  6060 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:20.574398  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:20.940873  5939 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:20.583071  5948 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:20.649814  6084 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.944346  6084 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:20.759259  5970 debug-util.cc:398] Leaking SignalData structure 0x7b08002b0e60 after lost signal to thread 1520
W20260709 12:23:20.949288  5970 debug-util.cc:398] Leaking SignalData structure 0x7b08001e7660 after lost signal to thread 4933
I20260709 12:23:20.947392  6084 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:20.761335  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.952286  5218 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:20.895270  5844 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.898443  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.929448  5222 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:20.955554  5844 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:20.563382  6085 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:20.983577  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.984511  5222 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:20.984582  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.985668  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.986835  5218 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:20.987010  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:20.987975  6085 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:20.991798  6085 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:21.035243  5116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.048s	sys 0.070s
I20260709 12:23:21.035552  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:21.035884  5116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.049s	sys 0.070s
I20260709 12:23:21.036728  6077 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.036831  5844 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.037429  6077 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:20.961221  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.038539  5945 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:21.037513  5844 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.039930  6077 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.041267  5844 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:21.041445  6077 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.042160  6077 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.042959  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.044162  6077 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:21.044836  6077 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.044952  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.045439  6077 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:21.045797  5219 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:21.048559  6077 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:21.055835  5970 debug-util.cc:398] Leaking SignalData structure 0x7b08001cf020 after lost signal to thread 5100
W20260709 12:23:21.058200  5189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.000s	sys 0.010s
W20260709 12:23:21.058589  5189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.000s	sys 0.010s
W20260709 12:23:21.058952  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 518033us
I20260709 12:23:21.064990  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.067322  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.067481  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.068034  5222 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:21.068234  5218 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:21.068418  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.069059  5219 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:21.070116  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.071296  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.071513  5218 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:21.070456  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.070531  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.070845  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.071915  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.072742  5222 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:21.076118  6067 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.071916  5219 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:21.076727  6067 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:21.077700  6063 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.078020  5189 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.078380  6063 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:21.079015  6084 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.079444  6084 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.079876  6084 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.080956  6063 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:21.081979  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.082937  6047 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.083091  5189 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.083377  6047 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.083824  6047 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.084143  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.085719  6071 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.086128  6071 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.086514  6071 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.089505  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.089828  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.090526  6084 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.091598  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.092998  6084 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:21.093713  6084 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:21.094923  6084 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.095348  6084 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:21.099926  6067 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:21.074003  6089 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.101105  6089 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.101478  6089 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.101599  6062 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.104725  6084 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.107282  6084 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.074993  5899 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.110883  5899 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.111742  6072 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.111915  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.112185  6072 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.112630  6072 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.112983  6060 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.075186  6087 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.113564  6060 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.114015  6060 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.113936  6087 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:21.116652  6087 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:21.120522  6053 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:21.121287  6053 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.124235  6053 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.124666  6053 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.125458  6060 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.125910  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.127831  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.128032  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.128655  5943 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.073894  6085 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.128731  5945 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.129676  6085 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.130159  6085 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.128033  6060 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.131651  6060 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:21.132002  6060 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.132385  6060 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:21.134718  6053 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.137063  6053 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.137235  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.138041  5945 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.138729  6089 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.140056  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.140790  5945 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
W20260709 12:23:21.141700  5115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.970s	user 0.026s	sys 0.061s
I20260709 12:23:21.142235  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.142936  5945 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
W20260709 12:23:21.143092  5115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.972s	user 0.026s	sys 0.061s
W20260709 12:23:21.143532  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 2.97202e+06us
I20260709 12:23:21.146893  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.147547  5945 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.151602  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:21.152253  5940 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.152289  6089 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.155479  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 116) took 1039 ms. Trace:
I20260709 12:23:21.162341  5940 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:21.162030  6047 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.188504  5218 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:21.162463  6071 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.189653  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.191076  6071 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.191493  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.194089  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.151219  6085 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.163865  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:21.156023  5944 rpcz_store.cc:276] 0709 12:23:20.115724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.128364 (+ 12640us) service_pool.cc:224] Handling call
0709 12:23:21.154389 (+1026025us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416e9c 00007fd8a5416a5b 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:21.154672 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:21.154885 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:21.154919 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:21.154970 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:21.155002 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.155066 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:21.155175 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5079936}
I20260709 12:23:21.194743  6047 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.198426  6085 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.205513  6060 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.222328  5939 raft_consensus.cc:2393] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:21.179378  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.470485  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.177947  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.487825  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.169616  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.499610  5219 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:21.178263  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.501545  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.504405  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.506147  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.176877  5221 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:21.177626  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.187326  5899 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.187628  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.166819  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 118) took 1050 ms. Trace:
I20260709 12:23:21.526280  6093 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.526808  5942 rpcz_store.cc:276] 0709 12:23:20.116687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.118290 (+  1603us) service_pool.cc:224] Handling call
0709 12:23:21.166580 (+1048290us) spinlock_profiling.cc:243] Waited 42.2 ms on lock 0x7b5400005854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:21.166751 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96981248}
I20260709 12:23:21.527105  6093 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:21.528654  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.177947  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.531558  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.532742  6080 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.181711  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.533499  6080 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.535410  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.536113  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.166118  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:21.538694  6080 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:21.539481  6080 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.540105  6080 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:21.187222  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:21.185912  5970 debug-util.cc:398] Leaking SignalData structure 0x7b08002ef9e0 after lost signal to thread 5174
I20260709 12:23:21.167362  6072 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.551659  6080 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:21.217195  6074 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.554112  6074 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:21.554970  6072 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.556798  6074 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:21.235846  6060 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:21.363267  5900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.1.123.67:46855 for RPC RequestConsensusVote timed out after 1.659s (ON_OUTBOUND_QUEUE) [suppressed 194 similar messages]
I20260709 12:23:21.398757  6044 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.560431  6044 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.533308  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.406637  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:21.417074  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 160888us
I20260709 12:23:21.434446  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.443564  6082 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.438488  6017 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.438375  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.446869  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.447086  6062 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.447762  5844 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.446943  6077 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.449576  6094 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.452831  6066 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.454078  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.456068  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.457801  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.458865  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.461280  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.464583  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.473469  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.472144  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.478048  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.597481  6066 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.489374  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.493165  6055 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:21.507331  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.598809  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.599598  5144 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.512763  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:21.513212  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 325947us
I20260709 12:23:21.603106  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.603299  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.603459  5899 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.603829  5149 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.604772  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.604933  5899 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.606741  5899 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.513901  6085 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.596522  6077 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:21.607545  6085 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.607980  6085 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.515954  6079 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.521184  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:21.529436  6092 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.562942  6044 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:21.402732  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.569106  6082 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:21.569200  6017 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:21.594130  5844 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:21.559798  6093 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:21.595892  6094 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.599917  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.599891  6066 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:21.597146  5970 debug-util.cc:398] Leaking SignalData structure 0x7b080037ea20 after lost signal to thread 5248
W20260709 12:23:21.622146  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 101054us
I20260709 12:23:21.626929  5149 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.595232  5939 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:21.624828  5143 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.625576  5150 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.609208  6077 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:21.625406  6055 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.626792  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.638234  5144 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.623083  6072 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.638792  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.639320  6072 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.639635  5142 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.639847  6072 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.641219  6082 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:21.641585  5970 debug-util.cc:398] Leaking SignalData structure 0x7b08002376c0 after lost signal to thread 5973
I20260709 12:23:21.642158  5899 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.643064  6090 raft_consensus.cc:2764] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:23:21.643234  5899 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.610075  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:21.625198  6092 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.595455  6062 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:21.609184  6079 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.626048  6060 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:21.588954  6058 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
W20260709 12:23:21.635134  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 272034us
I20260709 12:23:21.657285  6017 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:21.647593  6092 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:21.677846  6096 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:21.678515  6093 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.678941  6068 consensus_queue.cc:1059] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.090s
I20260709 12:23:21.679494  6093 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:21.682902  6090 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.683362  6093 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:21.686647  6094 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:21.647619  6062 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:21.636243  6055 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.683740  6090 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.651440  6060 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.703403  5844 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:21.699023  6055 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.703003  6090 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.705495  6060 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:21.701803  6080 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.721913  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.710759  6079 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:21.699333  6095 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:23:21.723627  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.731951  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.732542  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.732909  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.743790  5142 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:21.743813  5150 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:21.745169  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.745322  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:21.696261  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 53121.9us
I20260709 12:23:21.733057  6080 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:21.733990  6057 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:21.721948  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.739180  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.732012  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.743186  6095 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:21.784970  6057 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:21.784839  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.785781  6095 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:21.786190  5142 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:21.786840  6095 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.787483  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.763350  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.766121  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.767777  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.760084  5899 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:21.850744  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 104455us
I20260709 12:23:21.851519  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.776477  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:21.852962  6082 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:21.853130  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 76824.9us
I20260709 12:23:21.854095  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:21.855005  6109 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.788831  6095 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.856168  6106 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.856645  6106 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.857053  6106 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.761409  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.785250  5148 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:21.786849  6057 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.794795  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.793308  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.796821  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.867285  5149 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:21.867740  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.868186  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.869128  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.869372  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.870527  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.871529  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.872491  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.803954  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.872756  5146 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:21.873941  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.805114  6092 raft_consensus.cc:3055] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:21.806263  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.809834  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.811270  6080 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:21.727487  5190 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:21.829807  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 260359us
I20260709 12:23:21.884388  6055 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.875291  6072 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.886541  5940 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:21.887742  6072 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.828086  6085 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.892716  5941 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:21.833176  6044 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.833729  6107 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:21.838284  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.839321  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.864987  6109 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.839624  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.875151  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.840947  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.841481  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.842582  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.843880  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.845615  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.846750  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.896051  6107 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.904024  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.855301  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:21.906610  6107 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:21.857704  6082 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:21.905586  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.859414  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.896265  6044 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:21.909286  5068 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.906426  6095 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:21.861200  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.911124  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:21.911356  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 50259.5us
I20260709 12:23:21.911479  5188 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:21.912300  5899 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.912663  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.912930  6047 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.898530  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.913471  6047 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.914198  6047 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.914860  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.915974  6055 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:21.916759  6055 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.897486  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.918206  6106 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.918684  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:21.866299  6057 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.864822  5142 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:21.875133  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.922580  5144 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:21.876643  5143 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:21.876144  6092 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:21.877942  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.803064  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:21.907624  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 52403.6us
I20260709 12:23:21.746891  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.924549  6092 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:21.925259  5223 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:21.899772  6069 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.906895  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.926018  6069 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.926256  6092 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.926496  6069 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.927194  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:21.928755  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.930697  6106 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:21.894243  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 122) took 1663 ms (client timeout 1547 ms). Trace:
W20260709 12:23:21.931496  5941 rpcz_store.cc:269] 0709 12:23:20.230554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.477687 (+1247133us) service_pool.cc:224] Handling call
0709 12:23:21.894028 (+416341us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400007b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:21.894170 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":676698880}
I20260709 12:23:21.866134  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:21.911345  6044 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:21.898705  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:21.935277  6060 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.935631  5041 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.935921  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:21.916110  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.936544  5939 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:21.893680  6085 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:21.937055  6044 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:21.936883  5041 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:21.938799  6060 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:21.938951  6044 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.939496  6044 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:21.939534  6080 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.940141  6080 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:21.899564  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:21.936836  6095 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:21.940654  6080 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.941788  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.943257  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.944670  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.897861  6109 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.901438  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.936555  6107 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:21.946271  5069 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.938452  5115 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:21.930994  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.948132  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.902297  5067 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:21.966017  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.966286  5941 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:21.967519  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.968407  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.969884  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.973601  5900 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:21.951359  5188 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:21.983279  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.952091  6095 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:21.917164  6055 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:21.956379  5940 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:21.957620  5216 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:21.995047  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.957868  5943 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:21.996865  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.997042  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.908721  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.993655  6049 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.001151  6049 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:22.001645  6049 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.001771  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.958727  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.961935  6082 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:21.962152  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.991858  5221 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.006307  5844 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.006644  5944 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:22.006785  5844 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:21.995117  6055 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.963009  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:22.007460  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.963711  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.964450  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:22.008106  5844 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.967551  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.979399  6097 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:22.009451  5844 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.010138  6097 raft_consensus.cc:3055] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:22.010545  6097 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:21.980377  5946 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:22.012007  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.949122  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:22.012342  6097 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:21.913578  6072 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.015646  6072 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:22.016096  6072 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.019460  5075 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:22.023228  6060 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:21.981487  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:21.981889  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:21.981961  6080 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:21.990113  6095 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.025156  6095 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:21.993682  6090 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:22.027770  6090 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:22.027905  5076 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
W20260709 12:23:22.028419  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 54953.9us
I20260709 12:23:21.994143  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:21.994385  5945 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
W20260709 12:23:21.899072  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 171698us
I20260709 12:23:22.014804  6047 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:22.029352  5900 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:22.031103  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:22.040374  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.042238  5844 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.042740  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.034876  6097 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.045115  5900 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.046865  6058 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:22.048122  6058 raft_consensus.cc:3055] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.048560  6058 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:22.048667  6060 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:22.049842  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 65072.5us
I20260709 12:23:22.052114  6062 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.053640  5071 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.050163  6058 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:22.054101  5900 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:22.055048  6058 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.062165  6047 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.057433  5217 raft_consensus.cc:1240] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.056978  6080 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.065361  6047 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.065891  6047 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.066349  6047 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.050563  6069 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:22.039145  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 126936us
I20260709 12:23:22.069204  5216 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.070784  6069 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.073654  6095 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:22.074213  6049 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:22.076452  6095 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:22.040323  6092 consensus_queue.cc:1059] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:23:22.079859  6080 raft_consensus.cc:3055] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.080328  6080 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:22.080197  6072 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:22.081063  6044 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:22.081621  6080 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:22.050843  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:22.084625  5844 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.084828  5189 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:22.039873  6090 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.086017  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:22.086629  5844 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.089555  5844 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.088518  6113 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.091323  5189 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.083000  6080 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.094638  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.108036  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.112196  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.082633  6079 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:22.089380  6090 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.096292  6049 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.097643  6044 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.098201  6109 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:22.099686  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.100041  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.116156  6057 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.100425  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.100940  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.101852  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.102349  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.102933  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.103571  6072 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:22.119302  6090 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:22.140918  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 49698.1us
W20260709 12:23:22.108047  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 57184.8us
I20260709 12:23:22.106853  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.115669  6069 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.142544  6069 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.143025  6069 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.140818  5071 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:22.147969  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.148679  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:22.119458  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.149557  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:22.150434  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.113847  6082 consensus_queue.cc:1059] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:23:22.136719  6055 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:22.120247  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.151608  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.151938  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.121726  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.133682  6101 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.153223  6101 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:22.153520  6072 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.151702  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.154321  6101 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:22.155524  5939 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.155944  6101 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.127152  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.128712  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.129408  6105 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.129964  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.133188  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.151660  6082 raft_consensus.cc:3055] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.158735  6082 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:22.159672  6082 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:22.160432  6092 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:22.160846  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:22.114935  5148 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:22.161164  6092 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:22.161856  5945 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:22.150520  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:22.121033  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 90002.7us
I20260709 12:23:22.162400  6092 raft_consensus.cc:3055] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.162832  6092 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:22.163718  6092 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:22.120196  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.164520  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:22.164776  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:22.161001  6082 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.157038  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:22.177402  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.177006  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.127389  5844 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:22.094072  6113 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:23:22.152597  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 45833.5us
W20260709 12:23:22.166038  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 110773us
I20260709 12:23:22.150964  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:22.167724  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:22.186491  5946 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:22.168653  5189 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.171007  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.172118  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:22.172497  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.193146  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.193734  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:22.194399  5944 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.194572  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.196290  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.196411  6090 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:22.185462  6113 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.190508  6069 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:22.200142  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.172741  6049 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.173233  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.173940  6112 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.174330  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.175359  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.176162  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.197527  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.179957  5145 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.183758  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.209820  5223 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:22.184844  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.183796  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.184309  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.157512  6055 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:22.184149  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.220324  5939 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.220578  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.186764  5844 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:22.223374  6110 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:22.223939  6110 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.224364  6110 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.224623  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:22.225611  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 44474.8us
I20260709 12:23:22.226337  5945 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.228907  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:22.188103  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:22.189997  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 47820.6us
I20260709 12:23:22.190598  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.191583  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.230134  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.231428  5189 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.232105  6109 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:22.233052  6107 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.233574  6107 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:22.199275  6047 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:22.234042  6107 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.234949  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.166015  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.179606  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.184713  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.203038  6049 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.219048  5217 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:22.210207  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.216028  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:22.219637  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:22.233273  6105 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:22.210479  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.213506  6114 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:23:22.243940  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 79251.1us
I20260709 12:23:22.199977  6092 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:22.248893  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 44580.5us
W20260709 12:23:22.266490  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.243070  6047 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.253986  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.254349  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.254899  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:22.257051  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:22.257433  5219 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.251420  6114 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:22.258993  5939 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.259265  6105 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:22.260195  6062 consensus_queue.cc:1059] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:22.241357  6079 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.236953  6112 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:22.249441  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:22.214558  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.297104  6062 raft_consensus.cc:3055] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.268066  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.297683  6090 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.297839  5216 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:22.213860  6095 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:23:22.298362  6062 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:22.298893  6095 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.299268  6095 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:22.299381  6062 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:22.300153  6095 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:22.300943  6062 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.240671  6049 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.302758  5900 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:22.303864  5900 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:22.267709  6069 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:22.305792  5150 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
W20260709 12:23:22.263244  5097 debug-util.cc:398] Leaking SignalData structure 0x7b08002c3fa0 after lost signal to thread 1520
I20260709 12:23:22.307377  6055 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:22.307828  6055 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.308220  6055 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.308614  5899 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.242712  6044 consensus_queue.cc:1059] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:23:22.282028  6047 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.283805  5945 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:22.286860  6120 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.257490  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.311559  6047 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:22.312006  6047 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.315583  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:22.316201  5939 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.317023  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:22.279814  5189 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:22.320909  5844 consensus_queue.cc:1059] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:22.292855  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45308.9us
W20260709 12:23:22.295132  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 45775.8us
I20260709 12:23:22.323966  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:22.278952  6114 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.280525  6105 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:23:22.308920  5097 debug-util.cc:398] Leaking SignalData structure 0x7b08002c4680 after lost signal to thread 4933
I20260709 12:23:22.309720  6116 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.309772  6109 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:22.310498  6120 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:22.291720  6110 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:22.267439  6066 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:23:22.328286  6090 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.345165  6120 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.350271  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:22.344539  5097 debug-util.cc:398] Leaking SignalData structure 0x7b08001c7f80 after lost signal to thread 5100
I20260709 12:23:22.327608  6069 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.318463  6112 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.325759  5114 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:22.353199  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 78501.9us
I20260709 12:23:22.338865  6116 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:22.353734  5097 debug-util.cc:398] Leaking SignalData structure 0x7b0800141c00 after lost signal to thread 5101
I20260709 12:23:22.305469  5900 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.339370  6109 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.352985  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.353175  6069 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.359543  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.348140  6058 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:23:22.356103  6118 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.362393  6116 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.630303  6097 raft_consensus.cc:3055] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.630524  6058 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.630860  6097 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:22.631054  6058 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:22.631901  6097 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:22.631999  6058 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:22.493283  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 163699us
I20260709 12:23:22.378459  6069 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:22.619753  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 314352us
W20260709 12:23:22.517899  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 180750us
I20260709 12:23:22.646845  5189 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.648371  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:22.590960  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 265291us
I20260709 12:23:22.650890  6097 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.651906  5114 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.653995  6117 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:22.596653  5073 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:22.654480  6062 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:22.645428  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:22.656261  6115 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:22.596593  6047 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:22.598579  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 220324us
I20260709 12:23:22.593684  6113 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:22.658825  6047 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:22.659825  6113 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:22.654525  6117 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:22.660669  5221 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.661463  6117 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:22.666249  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 291542us
I20260709 12:23:22.380920  6090 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.602711  6114 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:22.436277  5097 debug-util.cc:398] Leaking SignalData structure 0x7b08002ae0a0 after lost signal to thread 5174
I20260709 12:23:22.403208  5844 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.604341  6110 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.374413  5141 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:22.612589  6112 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:22.614889  6095 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.616951  6055 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:22.374228  6109 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.618412  6107 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:22.374570  6105 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:22.479928  6066 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:22.626227  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.587775  6049 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:22.634084  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.635979  6068 raft_consensus.cc:3055] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.637620  5071 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:22.653738  6127 consensus_queue.cc:1059] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:22.382613  6124 raft_consensus.cc:3055] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.653211  5114 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:22.714833  5097 debug-util.cc:398] Leaking SignalData structure 0x7b08002951a0 after lost signal to thread 5248
I20260709 12:23:22.712639  5940 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:22.744094  6066 raft_consensus.cc:3055] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.744686  6066 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:23:22.745126  5097 debug-util.cc:398] Leaking SignalData structure 0x7b0800307740 after lost signal to thread 5973
I20260709 12:23:22.747279  6068 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:22.747515  6127 raft_consensus.cc:3055] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.747980  6127 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:22.717654  6057 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:23:22.748445  6068 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:22.748937  6127 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:22.694279  6069 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:22.695181  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:22.660162  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.695744  6118 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.696615  6116 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:22.599409  5144 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:22.699352  5188 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:23:22.757623  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 58347.1us
I20260709 12:23:22.736857  5223 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
W20260709 12:23:22.750337  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 124310us
I20260709 12:23:22.693111  6134 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.761698  6134 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.762138  6134 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:22.752722  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 99593us
I20260709 12:23:22.762923  6116 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.763444  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.764868  6069 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:22.754629  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 94567.2us
I20260709 12:23:22.765280  5216 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.766011  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:22.755262  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 60184.7us
I20260709 12:23:22.737854  5221 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.756311  6118 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.746555  6124 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:22.767167  5844 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:22.767473  6124 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:22.768251  6107 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:22.769186  6068 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.768970  6133 consensus_queue.cc:1059] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:23:22.770171  6133 raft_consensus.cc:3055] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:22.770501  6133 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:22.710577  6120 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:22.766418  6122 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:22.768970  6124 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.760021  5042 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.774758  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:22.777748  6127 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.778367  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:22.778417  6120 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:22.780184  6120 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.780767  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.782536  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.765329  5114 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:22.745733  6066 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:22.786499  5189 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.790729  6132 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:22.791128  6122 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:22.791316  6132 raft_consensus.cc:697] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:22.791497  5188 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:22.734787  6090 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:22.787575  6123 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.796031  5844 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.796838  6125 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:22.793231  6114 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:22.771381  6133 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:22.798570  6133 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.754413  6058 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.800737  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:22.753891  6082 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.800274  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.803417  5114 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.803660  5042 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.804339  6049 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.783603  6113 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.806629  5844 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:22.806854  6044 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.806209  6057 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.784216  6117 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:22.808676  5114 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.806262  6097 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:22.811388  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.811690  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.811875  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.812031  6112 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:22.812373  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.792366  6132 consensus_queue.cc:237] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:22.812711  6117 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:22.814867  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.819156  5114 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:22.819753  5141 raft_consensus.cc:2393] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:22.810040  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:22.820276  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:22.820464  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.810787  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.787694  6080 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.821866  5844 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.821703  6082 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:22.821714  5150 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 2: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:22.822654  6062 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:22.809376  5189 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.822996  6082 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:22.787541  5939 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:22.823402  6082 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:22.824331  6082 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:22.795435  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.815395  6117 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.818581  6055 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.819006  6097 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.826207  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.826854  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.821609  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.828905  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.829054  6136 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:22.832602  5844 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:22.832434  5114 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:22.832434  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.804425  6128 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:23:22.835191  5844 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.835381  6092 consensus_queue.cc:1059] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:23:22.838495  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:22.842460  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.838706  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.816474  6114 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.721704  6130 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.116s
I20260709 12:23:22.826829  6125 raft_consensus.cc:697] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:22.802919  6090 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:22.830072  6082 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.862310  6090 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:22.864051  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.831790  6105 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.852509  6047 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.857301  6130 raft_consensus.cc:3055] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.842519  6119 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.880517  5145 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
W20260709 12:23:22.900444  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.884884  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.912531  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:22.833673  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 49145.5us
I20260709 12:23:22.923174  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:22.868263  6080 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.883149  6101 raft_consensus.cc:3055] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.930343  5940 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:22.934587  5215 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.934953  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.935400  6057 raft_consensus.cc:3055] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:22.919483  6123 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.937315  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:22.855073  6109 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:22.943910  5042 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:22.871129  6130 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:22.955472  5223 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.947551  6044 consensus_queue.cc:1059] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:23:22.976136  6109 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.976712  5222 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.977649  5947 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:22.978691  6134 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:22.980206  5945 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.982344  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:22.983937  5141 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:22.986487  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:22.961392  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:22.988591  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:22.989971  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:22.990852  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:22.941774  6125 consensus_queue.cc:237] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:22.872381  6118 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:22.998275  5224 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.999508  6118 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:22.959312  6057 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:23.000860  6057 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:23.002254  6057 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.958220  6066 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:23.003580  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44485.2us
I20260709 12:23:22.944995  6101 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:22.972370  5071 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:22.967247  5221 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:23.009306  5142 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:23.006155  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:23.011248  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 67449.8us
I20260709 12:23:23.011279  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.014272  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:23.015007  6107 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.015933  6107 raft_consensus.cc:3055] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:23.016336  6107 raft_consensus.cc:740] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:23.016410  5947 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:23.014173  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:22.959097  5900 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:23.019315  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 73813us
I20260709 12:23:23.020258  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.021247  6136 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.020489  6101 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:23.022080  6136 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.022800  6101 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.023952  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.014117  6131 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.013561  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:23.033710  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 59650.6us
I20260709 12:23:23.033747  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.035701  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.037670  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:23.038329  6116 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.039134  6069 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.037910  6049 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.044234  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.048930  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:23.050477  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.051142  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:23.061285  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 47850.2us
I20260709 12:23:23.063134  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:23.064546  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.068703  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.069522  5042 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.070541  6138 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.071075  6133 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.071591  6122 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:23.072228  6123 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.072765  6129 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.074819  6134 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:23.055603  6044 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:23:23.075644  6068 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.075933  6047 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.036568  6136 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.055540  6092 consensus_queue.cc:1059] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:23:23.036322  6131 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:23:23.078287  6114 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.078763  6115 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.068150  6080 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:23:23.068040  6113 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.100256  6123 raft_consensus.cc:3055] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:23.100888  6123 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:23.083588  6079 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:23.101352  6114 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:23.102198  6079 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:23.102787  6079 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:23.102443  6114 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:23.017410  6107 consensus_queue.cc:260] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:23.093192  6131 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:23.093464  6133 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:23:23.098496  6068 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:23.100718  6113 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:23.109254  6068 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:23.089977  5147 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:23.109798  6068 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.110098  6113 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:23.110675  6058 raft_consensus.cc:3055] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:23.111088  6058 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:23.100980  6080 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:23.113209  6080 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:23.087230  6121 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:23:23.084183  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:23.107105  5189 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:23.103953  6079 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:23.116551  6121 raft_consensus.cc:3055] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:23.116962  6121 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:23.091149  6049 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.099575  6092 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:23.115636  6112 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.117178  6079 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:23.120651  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 50476.7us
I20260709 12:23:23.122454  6080 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:23.126992  5149 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:23.128118  5190 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:23.120224  6092 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:23.106591  6131 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:23.130287  6092 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:23.130187  6121 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:23.139346  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 69988.3us
I20260709 12:23:23.112025  6058 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:23.131003  5900 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.134639  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.130858  6069 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.134243  6131 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:23.143302  5900 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.144309  6134 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.145041  6134 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.145597  6134 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:23.105647  6130 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:23.140406  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.143460  6131 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.151252  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.152549  5114 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.154102  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.123653  6132 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.156168  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:23.157071  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.161018  5071 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
W20260709 12:23:23.160940  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 77283.4us
I20260709 12:23:23.161509  6128 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.158252  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.163170  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.164516  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.164826  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.158525  6069 raft_consensus.cc:697] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:23.166167  6069 consensus_queue.cc:237] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:23.168253  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.164301  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.146536  6134 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:23.163791  6082 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.136286  6123 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:23.170998  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 54258.7us
I20260709 12:23:23.177425  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.172199  5069 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.179064  6092 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.180008  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.177510  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.172655  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.178798  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.184479  5075 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.171617  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.185626  5071 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.174285  6121 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.158996  6127 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.183876  5076 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:23.163352  6080 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:23.189458  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.194409  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.180692  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.172992  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.196218  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.201021  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.175319  6123 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:23.174708  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.188007  6132 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:23.164866  6142 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:23.212870  5071 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.180686  5069 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:23.224663  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.229362  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.236181  5071 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:23.240338  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.241593  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.202804  6047 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.195417  5068 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.242537  5071 raft_consensus.cc:2376] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2. Re-sending same reply.
I20260709 12:23:23.249833  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.257325  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.251794  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.264380  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.268633  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.256410  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.248278  5070 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:23.269845  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.243283  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.270591  5076 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.261894  5150 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:23.258148  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.271791  5147 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:23.271128  5146 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:23.274663  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:23.269892  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.284441  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.285746  5190 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:23.252667  6119 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.291991  5072 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:23.292729  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.293401  6107 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.250387  6117 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:23.293409  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.304301  5068 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:23.303642  6117 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:23.305742  6116 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.314666  5073 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.305744  5219 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:23.268664  6132 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:23.264780  6118 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:23:23.321676  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.324743  5075 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.332790  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.338399  5071 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.334120  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.340898  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:23.320504  5898 leader_election.cc:336] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.518s (SENT)
I20260709 12:23:23.342905  5188 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.344054  6138 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:23:23.344558  5898 leader_election.cc:336] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.545s (SENT)
I20260709 12:23:23.283213  6068 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:23.348315  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.346290  5149 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:23.355337  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.330193  6113 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.366724  6058 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.373951  5148 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:23.376050  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.381058  6044 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:23.387472  6068 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:23.390933  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.390746  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.391517  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.427851  5142 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:23.407689  6130 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:23.408466  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 226127us
W20260709 12:23:23.409480  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 51012.6us
W20260709 12:23:23.366942  5898 leader_election.cc:336] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.554s (SENT)
I20260709 12:23:23.338163  6118 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:23.423480  6044 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:23.430775  6136 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.434197  6113 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:23.430347  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.424801  6115 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:23:23.462023  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.467356  5947 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:23.468284  6113 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:23.460602  6080 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:23:23.470579  6080 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:23.470782  6118 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:23.470925  6080 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:23.472018  6080 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:23.472221  6044 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:23.468899  5067 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:23.473464  6044 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.456887  5898 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.474785  6136 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:23.474988  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 108085us
W20260709 12:23:23.475785  5898 leader_election.cc:336] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.560s (SENT)
W20260709 12:23:23.456650  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 80711.9us
I20260709 12:23:23.476835  4898 catalog_manager.cc:5697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 2, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:23.478170  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.478964  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.479537  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.481942  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.482486  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.483055  5141 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.483184  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.482031  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.484675  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.484972  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.485329  5073 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.485690  5075 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.486351  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.486717  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.489928  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.484355  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.478989  6136 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.495479  6136 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.473258  6080 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:23.498134  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.500643  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:23.500993  5072 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.501461  6149 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.502014  6149 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:23.502279  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:23.502943  6149 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:23.507030  5219 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:23.483348  5067 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:23.518728  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:23.519536  5142 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:23.522149  5141 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:23.477380  5898 leader_election.cc:336] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.605s (SENT)
I20260709 12:23:23.525650  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.525952  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.533679  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:23.534771  5945 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:23.532915  5898 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.535127  6118 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:23.535826  5190 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:23.536334  6134 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.536929  6134 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:23.537452  5947 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:23.539517  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 62171.1us
I20260709 12:23:23.548070  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.545104  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:23.550580  5898 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.659s (SENT)
I20260709 12:23:23.539368  6136 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:23.551709  5898 leader_election.cc:336] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.664s (SENT)
I20260709 12:23:23.552263  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.539832  5074 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:23.553731  6136 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:23.540742  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.556684  6125 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:23.557341  6125 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:23.557881  6125 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:23.558459  6134 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:23.561488  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:23.560739  4899 catalog_manager.cc:5697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 1 to 2, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:23.553427  5898 leader_election.cc:336] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.696s (SENT)
I20260709 12:23:23.572671  5898 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.575649  5076 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.576664  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.578845  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:23.580080  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.579734  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.581715  5072 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:23.478787  4898 catalog_manager.cc:5697] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:23.584447  4898 catalog_manager.cc:5697] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:23.586807  4898 catalog_manager.cc:5697] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 2, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:23.588312  4898 catalog_manager.cc:5697] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 2, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:23.590263  5071 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:23.593672  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.594537  5070 raft_consensus.cc:2376] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3. Re-sending same reply.
I20260709 12:23:23.596163  5070 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:23.592260  6069 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:23:23.583194  6134 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:23.607302  5141 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:23.608141  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.608354  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.609113  5067 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:23.610179  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.611552  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.613425  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:23.615031  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.615031  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.618307  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.621019  5898 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.543738  5069 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.627681  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.638827  5071 raft_consensus.cc:2376] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2. Re-sending same reply.
I20260709 12:23:23.639182  5898 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.639816  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.640596  5071 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:23.641422  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.642151  5071 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:23.642887  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.643406  5071 raft_consensus.cc:2393] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:23.636549  5076 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.644021  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.644459  5076 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:23.618120  5148 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:23.646764  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.646224  5898 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.651324  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.647006  6109 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.653066  6109 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.653803  5072 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:23.655234  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:23.655824  5072 raft_consensus.cc:2376] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2. Re-sending same reply.
I20260709 12:23:23.656598  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.657155  5072 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.657948  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.659165  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.659767  5072 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.660928  5072 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:23.661955  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.662768  5148 raft_consensus.cc:2393] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:23.667176  6049 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:23.673092  5074 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:23.609386  6125 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:23.647440  6134 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.681979  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:23.682524  5068 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.638271  6112 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:23:23.683003  5076 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:23.685777  6125 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:23.688273  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:23.694660  5042 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:23.695801  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 49657.6us
I20260709 12:23:23.694077  6134 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:23.697362  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.703583  5149 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:23.703431  5075 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:23.705000  5190 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:23.676630  5067 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:23.707672  5114 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.712445  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:23.714669  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.719522  5149 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:23.722250  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.728965  5898 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.705753  6080 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.732419  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.736687  5114 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.739019  5114 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.705971  6149 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:23:23.728130  6120 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:23.717029  6136 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.730098  6109 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.744279  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:23.709705  6113 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:23:23.716794  6124 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:23.728662  6095 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.738646  6130 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:23.739856  6066 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.762730  6127 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.772651  6156 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.780573  6156 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:23.767198  6124 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:23.783871  6124 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.767074  6120 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:23.785219  6120 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:23.787768  6095 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:23.789181  5141 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:23.764494  6131 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:23:23.776239  6109 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.791466  6109 raft_consensus.cc:697] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:23.770150  6130 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:23.799499  6127 raft_consensus.cc:697] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:23.810590  6127 consensus_queue.cc:237] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:23.814888  5073 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:23.767666  6134 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:23.818746  5141 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:23.792733  6109 consensus_queue.cc:237] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:23.830646  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 98361us
I20260709 12:23:23.831664  5898 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.832645  6136 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:23.833241  5898 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:23.834780  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 81375.6us
I20260709 12:23:23.836746  5898 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.837719  5940 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:23.808120  6130 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.838176  5898 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.842638  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:23.850136  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52678.3us
I20260709 12:23:23.841514  5939 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:23.848485  5945 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:23.863571  6117 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:23.865747  6165 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.867043  6165 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:23.874459  6165 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:23.849859  5067 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:23.921298  6160 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 12:23:23.934139  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 96032.4us
I20260709 12:23:23.855371  6115 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 12:23:23.885139  6168 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:23.967819  5898 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.975183  5898 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:23.976979  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:23.984725  5070 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.987030  6168 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:23.987584  6168 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:23.988004  6168 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:23.931041  6167 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:23.991737  6167 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:23.991904  6134 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:23.992630  6134 raft_consensus.cc:697] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:23.993741  6134 consensus_queue.cc:237] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:23.999661  6168 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:24.028199  6136 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:23.974355  6124 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:24.061683  5899 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:24.062875  6136 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:24.072995  5219 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:24.082535  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:24.002413  6149 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
W20260709 12:23:24.095142  6178 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:23:24.098788  5945 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:24.030270  6168 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:24.044862  6115 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:23:24.101413  5939 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:24.102257  5147 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:24.084694  6113 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
W20260709 12:23:24.106503  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 129632us
I20260709 12:23:24.132578  6066 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.137841  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:24.143754  6066 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:24.144658  5898 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:24.146083  5898 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:24.146369  6066 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:24.146930  6172 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:24.147794  6172 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:24.148280  6172 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:24.148847  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:24.144161  5141 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:24.150997  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:24.149766  6167 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:24.155637  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:23:24.192575  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 130149us
I20260709 12:23:24.158663  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:24.135303  6109 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:23:24.196439  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:24.196861  5898 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:24.158886  6168 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:24.198776  5076 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:24.203168  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:24.159611  6181 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:24.211548  6181 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:24.212036  6181 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:24.212134  6124 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:24.158722  6069 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:23:24.204942  5898 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:24.208277  6173 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:23:24.227121  5142 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:24.196503  6167 raft_consensus.cc:697] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:24.149340  6172 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:24.199198  6136 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:24.203648  6156 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:24.230788  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:24.228706  6165 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:24.220413  6117 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:23:24.230401  6164 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:23:24.240484  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:24.280987  5114 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:24.286223  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:24.250875  6136 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:24.257164  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 48437.3us
I20260709 12:23:24.263311  6109 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:24.241708  6130 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:24.318315  5076 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:24.303243  6168 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:24.319832  5114 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:24.286952  6165 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:23:24.329160  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 74168.9us
I20260709 12:23:24.304183  6136 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:24.325857  5221 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:24.255118  6125 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
W20260709 12:23:24.335444  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 65768.6us
I20260709 12:23:24.330021  5070 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:24.318157  6066 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:24.332219  6082 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:24.340763  6165 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:24.340785  6190 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:24.343514  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:24.266812  6167 consensus_queue.cc:237] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:24.342761  6066 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.344918  6168 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:24.372045  5076 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:24.384337  6134 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:24.385361  6134 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:24.389129  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 158468us
I20260709 12:23:24.390673  5898 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:24.391986  5898 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:24.355042  6066 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:24.401908  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:24.405673  5898 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:24.456754  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 51640.4us
I20260709 12:23:24.393865  6082 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:24.394693  6130 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:24.396106  6195 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:24.415325  6172 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:24.482512  6115 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:24.484895  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 120722us
I20260709 12:23:24.514946  5076 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:24.485565  6134 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:24.523193  6164 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:24.484513  6195 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:24.489447  6115 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:24.532047  6164 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:24.552206  6201 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 12:23:24.567528  6193 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:23:24.528167  6082 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:24.591298  6134 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:24.598471  5076 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:24.610101  6164 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:24.630877  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:24.631281  6184 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.372s	user 0.001s	sys 0.004s
W20260709 12:23:24.650857  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 320572us
I20260709 12:23:24.630908  6123 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.682606  6164 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:24.641812  6190 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:24.658999  6113 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.687240  6202 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:24.642823  5146 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:24.689710  6164 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:24.693996  6131 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:24.701383  5071 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:24.698858  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:24.602154  6195 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:24.688154  6123 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:24.704649  6117 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.698097  6167 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:23:24.709893  6204 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:24.714953  6206 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:23:24.719071  6080 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.714285  6095 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:23:24.718240  6130 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.697592  6113 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:24.735522  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 427431us
I20260709 12:23:24.707652  6115 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:24.726107  6200 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:24.748610  6211 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.757108  5946 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:24.703840  6202 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:24.737504  6204 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:23:24.761960  6156 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.012s	sys 0.015s
I20260709 12:23:24.739912  6172 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:24.730908  6117 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:24.764001  6181 consensus_meta.cc:287] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.003s
I20260709 12:23:24.794071  6156 raft_consensus.cc:697] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:24.767834  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:24.743379  6080 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:24.761420  6130 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:24.820209  6058 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.821225  6058 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:24.827287  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:24.812577  6156 consensus_queue.cc:237] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:24.773180  6200 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:24.808122  6181 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:24.840569  6120 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.873746  5147 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:24.773690  6211 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:24.872057  6058 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:24.878527  6207 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.879112  6207 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:24.783324  6190 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:24.881536  6058 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.882156  6058 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:24.872288  6113 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:24.887952  6058 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:24.888337  6130 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:24.875458  6164 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:24.889724  6080 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:24.890568  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:24.888947  6058 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.891534  6058 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:24.891776  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:24.877292  6095 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:24.892503  6095 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:24.893486  5149 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
W20260709 12:23:24.893517  6109 consensus_meta.cc:287] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.005s
I20260709 12:23:24.904758  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:24.906163  5219 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:24.879136  6131 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.886890  6123 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:24.919839  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:24.920142  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:24.920490  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:24.920670  5224 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:24.921370  6131 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:24.923398  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:24.924136  5218 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:24.877113  6120 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:24.922467  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:24.937517  6221 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.938406  6221 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:24.874092  6172 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:24.901891  6109 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:24.915416  5221 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:24.954589  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:24.924103  6209 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.957711  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:24.959178  5074 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:24.958395  5219 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
W20260709 12:23:24.943769  5940 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.015s	sys 0.035s
I20260709 12:23:24.943454  6095 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:24.923799  6066 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:24.963033  6136 consensus_meta.cc:287] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.618s	user 0.013s	sys 0.003s
I20260709 12:23:24.997236  6222 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.957079  5941 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:24.948205  6113 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:23:24.880203  6204 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:24.988994  5940 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.016s	sys 0.035s
I20260709 12:23:24.961663  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:24.984426  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.003619  6218 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:24.995599  6210 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:23:24.996654  6202 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:25.009240  6220 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 12:23:25.034854  6217 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:23:25.034631  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:25.038362  5941 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:24.961555  6209 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:25.035372  6124 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:25.039414  6218 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:25.046772  6124 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:24.998065  6222 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:25.002262  6136 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:25.006016  6221 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:25.057776  5073 raft_consensus.cc:2393] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:23:25.073338  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 146582us
I20260709 12:23:25.067798  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:25.058167  6058 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:25.081065  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:25.082975  5218 raft_consensus.cc:2393] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:25.036826  5942 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:25.083927  6131 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:25.086970  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:25.087604  6227 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.054563  6209 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:25.088279  6120 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:25.088184  6227 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:25.089370  6117 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:25.087917  5222 raft_consensus.cc:2393] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:25.052966  6207 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:25.089527  6211 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:25.095894  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:25.096218  6068 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.084470  5042 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:25.097272  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:25.098099  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:25.139052  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 54698.7us
I20260709 12:23:25.139364  5946 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:25.100112  5224 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:25.077019  6069 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:23:25.048018  6124 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:25.082315  5146 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:25.114073  6066 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:25.078703  6200 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:25.085464  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.143738  6068 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:25.196161  5069 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:25.197489  6068 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:25.051548  6218 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:25.200040  6227 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:25.118770  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:25.182890  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.201849  5076 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 3: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:25.182838  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:25.096225  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:25.114843  6136 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:25.203258  5218 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:25.203892  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:25.214336  5076 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:25.201457  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.169682  6123 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:25.215817  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:23:25.217024  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 298666us
I20260709 12:23:25.224097  5041 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:25.224152  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:25.188926  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:25.224977  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:25.219565  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:25.243883  5947 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:25.230506  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:25.253233  5946 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:25.216111  6124 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:25.259770  5115 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:25.237251  5041 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:25.216889  6227 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:25.220409  5943 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:25.216363  6123 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.274363  5221 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:25.240886  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:25.275983  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:25.225683  6080 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.239928  6058 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:25.230145  5073 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:25.291584  6080 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:25.289929  6231 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:25.302451  6231 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.302877  6231 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:25.270738  6066 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:25.273625  6227 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.299453  6080 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:25.297040  6068 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:25.301254  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.293305  6095 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.309851  6095 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:25.312987  6095 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:25.301874  6202 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:25.314003  6202 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:25.259917  5076 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:25.217181  6222 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:25.273168  5115 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:25.294404  6058 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.316699  6068 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.317085  6058 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:25.316787  6202 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:25.286026  6123 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:25.327679  5946 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:25.322104  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.333590  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.317281  6068 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:25.389207  5073 raft_consensus.cc:2393] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 3: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:25.309026  6227 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:25.346690  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.393895  5069 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:25.350304  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:25.395087  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44921.1us
I20260709 12:23:25.352260  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:25.343106  6058 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:25.307456  6066 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.396194  6066 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:25.396481  6136 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:25.397122  6136 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:25.397643  6136 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:25.412681  6181 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:25.412873  6068 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:25.415612  6068 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:25.423506  4897 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43226 (request call id 126) took 1393 ms. Trace:
I20260709 12:23:25.424131  4897 rpcz_store.cc:276] 0709 12:23:24.030228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.030413 (+   185us) service_pool.cc:224] Handling call
0709 12:23:24.160477 (+130064us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b54006a5754. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a5090 00007fd8a8543f3a 00007fd8ae2f9e9c 00007fd8ae3f804e 00007fd8ac8b92a4 00007fd8ac8b9232 00007fd8ac8b9161 00007fd8ac8b90dc 00007fd8ac8b8392 00007fd8a687648c 00007fd8a68758c6
0709 12:23:25.423231 (+1262754us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":96713600}
I20260709 12:23:25.425694  6231 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:25.432071  5072 raft_consensus.cc:2368] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
W20260709 12:23:25.442878  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 116911us
I20260709 12:23:25.446182  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:25.447196  5224 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
W20260709 12:23:25.451025  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 178160us
W20260709 12:23:25.449167  6190 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.013s	sys 0.018s
I20260709 12:23:25.433485  6231 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:25.458986  5071 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.044s	sys 0.037s
W20260709 12:23:25.474619  5071 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.044s	sys 0.037s
W20260709 12:23:25.474593  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 424270us
W20260709 12:23:25.499676  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.448027  6080 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.459008  6202 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:25.513051  5147 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:25.469084  6190 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.013s	sys 0.018s
I20260709 12:23:25.517946  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:25.496421  6058 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:25.496263  6228 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:23:25.519887  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 248681us
I20260709 12:23:25.496731  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:25.525195  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 65846.8us
I20260709 12:23:25.534192  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:25.525846  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 49300.2us
I20260709 12:23:25.511897  6080 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:25.528290  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:25.530043  6232 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.538509  5115 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:25.534286  5188 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:25.508827  6164 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:25.546540  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:25.549602  5075 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.016s	sys 0.024s
W20260709 12:23:25.550185  5075 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.016s	sys 0.024s
W20260709 12:23:25.566448  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 346993us
I20260709 12:23:25.507511  6069 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:23:25.548795  5221 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:25.567239  5188 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:25.567862  6127 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:25.570389  5946 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:25.572579  5115 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:25.562850  6124 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:25.614924  6230 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:25.612402  6058 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:25.617817  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50674.2us
I20260709 12:23:25.571077  5141 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:25.617805  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 45309.4us
I20260709 12:23:25.563828  6222 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:25.536064  6227 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:25.630584  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:25.549299  6164 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:25.587317  6080 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:25.602444  6123 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:25.614315  6231 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.708663  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.630358  5224 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:25.684945  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.564833  6232 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:25.703735  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.624795  6169 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:25.624465  6082 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:25.673743  5149 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:25.710879  5072 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:25.623836  6115 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:25.666869  6235 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.135s
I20260709 12:23:25.668074  6204 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:25.705303  6236 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:25.624168  6124 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:25.733362  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:25.664660  6218 consensus_queue.cc:1059] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:23:25.680724  6222 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:25.728204  6181 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:23:25.683539  6190 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 12:23:25.629637  6058 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:25.726250  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.735401  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:25.742770  5149 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
W20260709 12:23:25.740657  6240 log.cc:927] Time spent T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:23:25.730316  6234 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.734411  5067 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:25.748147  6236 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.712205  6231 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:25.667939  6230 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:25.746739  6113 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.750999  5068 raft_consensus.cc:2368] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:25.729687  6169 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:25.750765  6207 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.755441  6095 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:25.728322  4897 catalog_manager.cc:5697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 2 to 3, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 3 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:25.749099  6204 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:25.817025  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:25.818615  6215 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:25.819461  6215 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:25.791779  4897 catalog_manager.cc:5697] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 2, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:25.821846  6233 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 12:23:25.821563  6136 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:25.765784  5070 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:25.768348  6236 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:25.793040  5069 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:23:25.833958  6156 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.014s	sys 0.061s
W20260709 12:23:25.861450  6156 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.014s	sys 0.061s
I20260709 12:23:25.781105  6113 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:25.868731  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:25.867326  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 418980us
I20260709 12:23:25.871035  6246 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.777894  6234 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:25.864241  6242 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:23:25.908965  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:25.928936  6231 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:25.929482  5149 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:25.930791  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 203) took 1692 ms. Trace:
I20260709 12:23:25.816458  6207 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:25.804096  6095 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:25.896870  6080 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:25.916997  6241 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
W20260709 12:23:25.917765  6238 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 12:23:25.920351  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:25.924582  6204 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:25.925652  6222 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:25.823050  4897 catalog_manager.cc:5697] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 2, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:25.936010  6164 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:25.939360  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48738.4us
I20260709 12:23:25.938333  5147 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
W20260709 12:23:25.959498  6245 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.274s	user 0.001s	sys 0.001s
I20260709 12:23:25.960557  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:25.952912  4897 catalog_manager.cc:5697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 2, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:25.989825  5946 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:25.928964  6246 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.003902  4897 catalog_manager.cc:5697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 2, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:25.955971  6230 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:25.939868  6080 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:25.956744  6222 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:25.958683  6235 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:26.025137  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:26.033149  5943 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:23:25.956290  6124 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:26.019079  6058 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.001178  6164 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
W20260709 12:23:26.061887  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 211166us
I20260709 12:23:26.031924  6230 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.076550  5071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 512) took 1115 ms. Trace:
I20260709 12:23:26.046437  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.078962  5071 rpcz_store.cc:276] 0709 12:23:24.945204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.946051 (+   847us) service_pool.cc:224] Handling call
0709 12:23:24.946522 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:24.947051 (+   529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:24.947099 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:24.956491 (+  9392us) log.cc:844] Serialized 26 byte log entry
0709 12:23:25.475444 (+518953us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:25.475507 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.475615 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:26.059959 (+584344us) raft_consensus.cc:1685] finished
0709 12:23:26.059986 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:26.060174 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7615,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":9122,"thread_start_us":518459,"threads_started":1,"wal-append.queue_time_us":594727}
I20260709 12:23:26.057334  6169 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:26.122762  5946 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:26.036156  6222 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.064881  5141 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:26.064180  6058 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.048691  6235 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.062930  6123 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:26.058985  6082 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:26.062706  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.078526  6204 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:26.094153  6066 consensus_meta.cc:287] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.697s	user 0.015s	sys 0.003s
I20260709 12:23:26.083551  6232 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:26.120102  6218 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.166730  5224 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:26.119650  6202 consensus_meta.cc:287] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.605s	user 0.014s	sys 0.000s
I20260709 12:23:26.171012  5224 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:26.090138  6230 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.174459  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:26.177523  5219 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:26.173563  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.180275  5219 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:26.177321  6202 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:26.184798  6261 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.162230  6259 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:26.197669  6247 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 12:23:26.190065  6260 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.187820  6258 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:25.931253  5940 rpcz_store.cc:276] 0709 12:23:24.238088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.240683 (+  2595us) service_pool.cc:224] Handling call
0709 12:23:24.241057 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:24.735582 (+494525us) spinlock_profiling.cc:243] Waited 9.98 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a54310ac 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8a8533060
0709 12:23:25.034113 (+298531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.034225 (+   112us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:25.034904 (+   679us) log.cc:844] Serialized 26 byte log entry
0709 12:23:25.471177 (+436273us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:25.471236 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.471332 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:25.930446 (+459114us) raft_consensus.cc:1685] finished
0709 12:23:25.930465 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.930698 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":133097,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"spinlock_wait_cycles":22964480,"thread_start_us":1190941,"threads_started":2,"wal-append.queue_time_us":484869}
I20260709 12:23:26.204938  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:26.206923  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:26.219174  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 152513us
I20260709 12:23:26.219892  5149 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.150338  6058 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.215353  5075 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 514) took 1265 ms. Trace:
I20260709 12:23:26.171463  6232 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.166280  6204 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.166996  6066 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.207937  6262 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.185424  6261 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.176409  6218 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.256721  5149 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
W20260709 12:23:26.258785  6250 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:23:26.222671  6215 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:26.211336  6259 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:26.223161  6202 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.230243  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:26.232224  6095 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.293715  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:26.296485  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:26.260123  6244 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.254s	user 0.001s	sys 0.001s
I20260709 12:23:26.299414  5141 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.306260  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 219) took 1087 ms. Trace:
I20260709 12:23:26.306934  5945 rpcz_store.cc:276] 0709 12:23:25.218560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.221400 (+  2840us) service_pool.cc:224] Handling call
0709 12:23:25.241278 (+ 19878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:25.241768 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.241809 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:25.242548 (+   739us) log.cc:844] Serialized 26 byte log entry
0709 12:23:25.683342 (+440794us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:25.713108 (+ 29766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.729367 (+ 16259us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:26.305937 (+576570us) raft_consensus.cc:1685] finished
0709 12:23:26.305969 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:26.306179 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":215,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"thread_start_us":383217,"threads_started":1,"wal-append.queue_time_us":626094}
I20260709 12:23:26.310748  6218 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.312084  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.312897  5941 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:26.302094  5149 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:26.231503  6232 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:26.216351  6260 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.264349  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:26.286988  5222 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:26.223600  6082 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.267448  6080 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:26.318397  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 113566us
I20260709 12:23:26.266995  6231 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.290238  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.313604  6058 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.233354  6258 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.288013  6202 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.338708  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.301626  6246 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:26.352373  6260 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.353796  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:26.359285  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 135011us
I20260709 12:23:26.326078  6272 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.364248  6272 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.366106  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.320194  6267 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.336849  6251 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.345171  6080 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:23:26.368736  6115 consensus_meta.cc:287] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.626s	user 0.013s	sys 0.001s
I20260709 12:23:26.346454  6222 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:26.367633  5144 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:26.367558  6058 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.385082  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.375078  5221 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:26.340085  5940 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:26.336288  6279 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.331274  6231 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.378459  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.374804  6235 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.376116  6278 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.382169  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.262777  6262 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.383570  6080 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.375974  6272 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.397872  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:26.398830  6254 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:23:26.417512  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.382292  6172 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.384281  6251 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:26.402446  6232 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:26.402788  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:26.404058  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.387846  6246 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.405928  6228 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.388916  6115 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:26.392804  6281 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:26.417972  6258 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.396195  6230 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.383180  6267 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.394768  6279 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.418618  6172 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.415323  6080 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.397162  6215 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:23:26.407226  6134 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:23:26.396099  6058 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.441213  5218 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:26.435876  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.445771  5939 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:26.446754  6279 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.447256  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:26.448020  5073 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:26.449568  6251 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:26.451309  6123 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.451841  6123 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.454473  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.453946  6123 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.455504  6123 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:26.455966  6123 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.456826  6264 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.457827  6196 raft_consensus.cc:3055] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:26.458264  6196 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:26.458550  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.458562  6123 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:26.459199  6196 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.457441  6264 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.458675  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.432564  6204 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.469741  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:26.436008  6269 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:26.463763  5188 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:26.474762  5188 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:26.446756  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.485347  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:26.459542  6123 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.494485  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.434759  6246 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.499384  5188 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:26.443827  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.500965  6273 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:26.502243  6265 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.444322  6261 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:26.472173  6215 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:26.510717  6284 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:26.511250  6284 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.511046  6215 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.511650  6284 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:26.510707  6191 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:23:26.513574  6215 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:26.416787  6278 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.482978  6269 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.493221  6115 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:26.521291  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:26.526115  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:26.495849  5943 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:26.499905  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.489395  6181 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:23:26.498646  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:26.519178  6127 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.903s	user 0.026s	sys 0.001s
I20260709 12:23:26.493338  6222 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:23:26.433856  6228 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.522745  6115 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.518874  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.519971  6269 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.538053  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:26.519596  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.539031  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:26.522660  6202 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.507225  6123 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.435566  6281 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.540637  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.547333  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.548149  6136 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:26.503062  6265 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.523391  6215 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:26.546001  6277 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:26.585691  6082 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:26.540011  6127 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:26.590154  5223 raft_consensus.cc:2368] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:26.541285  6115 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.533303  6259 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:26.542330  5948 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:26.532241  6258 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:23:26.640231  6275 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:23:26.588583  5115 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:26.591122  6231 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.611789  6134 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.638597  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:26.657158  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 187472us
W20260709 12:23:26.658222  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 69726us
W20260709 12:23:26.647019  6280 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:23:26.644397  6082 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.666245  6082 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.644555  6264 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:26.651717  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:26.670841  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:26.658013  6258 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.662101  6127 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:26.682402  5899 leader_election.cc:400] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:26.687402  5899 leader_election.cc:403] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:23:26.688809  5899 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:26.669595  6191 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.676271  6058 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:26.682597  5073 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:26.678514  5144 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:26.687278  6127 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:26.744609  6127 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.675053  6136 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 12:23:26.674682  5067 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:26.683301  6258 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:26.677466  6222 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:26.676113  6172 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:26.749794  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:26.686892  6263 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:26.758788  6278 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.691073  6181 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.668867  6134 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.674305  6215 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:23:26.720850  6264 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:26.745221  5141 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:26.642223  6277 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.720508  6259 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:26.689018  6285 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:23:26.769882  6283 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:23:26.763127  6246 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:26.788578  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 116792us
W20260709 12:23:26.780294  6287 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 12:23:26.773773  6263 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.767508  6262 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:26.768082  6172 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.747573  6191 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.779194  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.783187  6222 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.793934  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:26.788694  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.803726  6285 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.806041  6263 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.821367  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:26.833276  5143 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:26.816474  5042 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:26.801360  6246 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:26.806562  6264 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.837653  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 611) took 1058 ms. Trace:
I20260709 12:23:26.813150  6262 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:26.808935  5221 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:26.836274  6285 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.844738  6281 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:26.841200  5042 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:26.830914  6228 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:26.863118  6296 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:26.863337  6228 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:26.863546  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.864058  6228 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:26.865049  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.850160  6058 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:26.865845  5945 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:26.846715  6191 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:26.822772  6123 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.808822  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:26.840786  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:26.838260  5145 rpcz_store.cc:276] 0709 12:23:25.779524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.921806 (+142282us) service_pool.cc:224] Handling call
0709 12:23:26.199639 (+277833us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:26.218259 (+ 18620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:26.218296 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:26.226240 (+  7944us) log.cc:844] Serialized 26 byte log entry
0709 12:23:26.318734 (+ 92494us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:26.318780 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.320119 (+  1339us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:26.837298 (+517179us) raft_consensus.cc:1685] finished
0709 12:23:26.837332 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:26.837527 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29979,"raft.queue_time_us":21975,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":9352,"spinlock_wait_cycles":1902976,"thread_start_us":59228,"threads_started":1,"wal-append.queue_time_us":190448}
W20260709 12:23:26.842810  6290 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.198s	user 0.000s	sys 0.002s
I20260709 12:23:26.865921  6058 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.789954  6259 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.890182  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:26.810199  6172 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.819594  6222 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.880900  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:26.881316  6296 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:26.847702  6262 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:26.903564  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 151444us
I20260709 12:23:26.847450  6080 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.890436  6058 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.877435  6281 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:26.221145  5075 rpcz_store.cc:276] 0709 12:23:24.949819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.950094 (+   275us) service_pool.cc:224] Handling call
0709 12:23:24.950469 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:24.950755 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:23:25.569174 (+618419us) log.cc:844] Serialized 14 byte log entry
0709 12:23:26.214849 (+645675us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:26.214926 (+    77us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:26.214991 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.215143 (+   152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:26.215277 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":348347,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":38130,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":12883,"thread_start_us":1235246,"threads_started":3,"wal-append.queue_time_us":644451}
I20260709 12:23:26.848842  6246 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.893332  6082 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.848764  6264 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.914710  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:26.916255  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:26.873430  6265 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:26.883644  6191 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:26.896294  6236 consensus_meta.cc:287] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.038s	user 0.015s	sys 0.000s
I20260709 12:23:26.865101  6228 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:26.920713  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:26.891669  5067 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:26.918673  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:26.907986  6262 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:26.928514  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 132477us
W20260709 12:23:26.917225  6288 log.cc:927] Time spent T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: Append to log took a long time: real 0.263s	user 0.002s	sys 0.000s
I20260709 12:23:26.890611  6284 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:26.959232  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:26.929279  6299 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.929013  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 598) took 1244 ms. Trace:
I20260709 12:23:26.974587  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:26.926730  6265 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:26.977909  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:26.983124  5142 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.982987  6265 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.985303  6080 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.929776  6236 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:26.986558  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:26.987154  6172 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:26.987331  5942 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:26.988010  6236 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:26.990439  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:26.975721  6299 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:26.978772  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:26.993865  5075 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:26.941130  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:26.976424  5148 rpcz_store.cc:276] 0709 12:23:25.681724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.701907 (+ 20183us) service_pool.cc:224] Handling call
0709 12:23:25.702318 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.765031 (+ 62713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.768243 (+  3212us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:25.774762 (+  6519us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:25.777339 (+  2577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.916877 (+1139538us) spinlock_profiling.cc:243] Waited 52.9 ms on lock 0x7b540036ea54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:26.916934 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:26.922905 (+  5971us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121782656}
I20260709 12:23:26.921361  6281 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:26.998092  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:27.005465  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:26.971400  6277 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:26.945628  6191 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:27.009024  6202 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:27.025619  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.015291  5142 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:27.047688  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.053633  6080 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.055256  6080 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:27.000922  6265 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:26.994316  6134 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.027987  6202 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:27.015362  6264 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:27.026806  6285 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:27.063001  6278 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:27.089063  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.098096  6299 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:27.098971  6080 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:27.126467  5068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 396) took 1985 ms (client timeout 1615 ms). Trace:
I20260709 12:23:27.127272  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:27.127197  5068 rpcz_store.cc:269] 0709 12:23:25.140573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.168358 (+ 27785us) service_pool.cc:224] Handling call
0709 12:23:25.321899 (+153541us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b5800560198. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:27.126048 (+1804149us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b540058a554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:27.126305 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":826025088}
I20260709 12:23:27.134420  6246 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:27.136000  6082 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:27.140750  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:27.142004  6222 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.142905  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:27.176044  6285 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.148934  6113 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.153080  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 600) took 1444 ms. Trace:
I20260709 12:23:27.153432  6234 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.177937  5146 rpcz_store.cc:276] 0709 12:23:25.708199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.708756 (+   557us) service_pool.cc:224] Handling call
0709 12:23:25.729743 (+ 20987us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.867884 (+138141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.872078 (+  4194us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:25.898115 (+ 26037us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:25.902248 (+  4133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.152754 (+1250506us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400238f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:27.152809 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:27.152983 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287542656}
I20260709 12:23:27.155555  6278 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:27.156796  6299 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:27.125082  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:27.129137  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:27.100325  6191 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:27.189522  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.129227  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:27.189718  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:27.089450  6202 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:27.179276  6115 consensus_meta.cc:287] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.001s
I20260709 12:23:27.194247  5072 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:27.190024  6299 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:27.169790  6267 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:27.194386  5074 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49646
I20260709 12:23:27.203217  6082 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:27.193894  6222 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:27.101110  6259 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.215709  5075 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:27.203312  5074 raft_consensus.cc:606] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.220374  5074 raft_consensus.cc:614] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:27.198458  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.223129  5075 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:27.193966  6058 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:27.229583  5075 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:27.230386  5150 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.017s	sys 0.043s
I20260709 12:23:27.233817  5074 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:27.200469  6115 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:27.236804  5150 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.017s	sys 0.043s
W20260709 12:23:27.221977  5217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.024s	sys 0.065s
I20260709 12:23:27.197376  6127 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:27.221432  6082 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:27.222867  6222 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:27.237999  6284 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:27.243922  5217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.025s	sys 0.065s
W20260709 12:23:27.284134  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 365536us
I20260709 12:23:27.285286  5142 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:27.245049  6258 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:27.244504  6281 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:27.291231  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.288552  5041 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:27.294016  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 358625us
I20260709 12:23:27.294576  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.299801  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:27.285398  6269 consensus_meta.cc:287] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.737s	user 0.010s	sys 0.005s
W20260709 12:23:27.296001  6181 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.525s	user 0.012s	sys 0.003s
I20260709 12:23:27.314388  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.315855  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.316182  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:27.316551  5947 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:27.317759  6181 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:27.320396  6181 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:27.320983  6181 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:27.322858  6222 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:27.323177  6127 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.323407  6191 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:27.323966  6191 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:27.324506  6115 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:27.326035  6191 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:27.323243  6181 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:27.331101  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.290901  6080 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:27.307114  6281 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:27.345888  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.347030  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.311816  6123 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:27.354032  6082 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.354241  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.355767  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.313879  6301 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:23:27.357868  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.358618  5942 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:27.359587  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.360742  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.362737  6066 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.343961  6080 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:27.348774  6263 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:27.349448  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.314128  6236 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:27.336095  6246 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:27.384704  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:27.397035  5071 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:27.398442  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:27.399250  5071 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:27.400415  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:27.401095  5071 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:27.386219  5947 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:27.379953  6263 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.403816  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.313683  6269 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:27.406778  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 94913.2us
I20260709 12:23:27.405504  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.386425  5942 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 3: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:27.357174  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.411145  5188 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:27.383620  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:27.382767  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:27.380688  6058 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:27.431447  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.431780  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.373167  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.416096  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:27.379864  5222 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:27.436915  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:27.418303  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 65) took 1118 ms. Trace:
I20260709 12:23:27.365206  6278 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:27.444620  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 615) took 1224 ms. Trace:
I20260709 12:23:27.418777  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.368875  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:27.385892  6308 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:23:27.372045  6296 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:27.421967  5188 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.375186  6258 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:27.371659  6080 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:27.384138  5069 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:27.464486  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:27.465734  5067 raft_consensus.cc:2393] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:27.381579  5220 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:27.466919  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 59) took 1003 ms. Trace:
I20260709 12:23:27.467132  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:27.468079  5067 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:27.467458  5220 rpcz_store.cc:276] 0709 12:23:26.463038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.468646 (+  5608us) service_pool.cc:224] Handling call
0709 12:23:27.381011 (+912365us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416eee 000000000036eb39 00007fd8ad48a1e0 00007fd8ad5530c4 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:27.381268 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:27.466529 (+ 85261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.466838 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11093248}
I20260709 12:23:27.392279  6246 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:27.469337  6246 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:27.385151  6123 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:27.470585  6123 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:27.385449  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:27.384958  6281 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:27.473745  5073 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:27.474912  5073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49646
I20260709 12:23:27.474538  6281 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:27.475762  5073 raft_consensus.cc:606] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.476295  5073 raft_consensus.cc:614] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:27.386289  6299 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.377130  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:27.438124  5150 rpcz_store.cc:276] 0709 12:23:26.299879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.431789 (+131910us) service_pool.cc:224] Handling call
0709 12:23:26.432157 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:26.432596 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:26.432628 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:26.436845 (+  4217us) log.cc:844] Serialized 26 byte log entry
0709 12:23:27.239075 (+802230us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:27.239119 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.239184 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:27.417987 (+178803us) raft_consensus.cc:1685] finished
0709 12:23:27.418012 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:27.418215 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1846,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":3696,"thread_start_us":755132,"threads_started":1,"wal-append.queue_time_us":971480}
I20260709 12:23:27.478638  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.479501  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.480149  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.377244  6262 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:27.480649  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:27.478638  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 101617us
I20260709 12:23:27.482327  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.430704  6115 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:27.483361  6115 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:27.483815  6115 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:27.483868  6309 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:27.484390  6309 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:27.484766  6309 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:27.433838  5070 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:27.443547  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.471200  6293 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:27.482524  6262 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:27.465648  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:27.480654  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.510749  5069 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:27.511008  5150 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:27.446116  5149 rpcz_store.cc:276] 0709 12:23:26.220629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.316587 (+ 95958us) service_pool.cc:224] Handling call
0709 12:23:26.316953 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:26.378408 (+ 61455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:26.378441 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:26.378488 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:26.378519 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.444351 (+1065832us) spinlock_profiling.cc:243] Waited 61.2 ms on lock 0x7b5400185954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:27.444417 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:27.444561 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140762496}
I20260709 12:23:27.513149  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.514358  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.515933  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.448056  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.507727  5188 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.445225  6066 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:27.509609  5114 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.517413  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.525842  5114 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:27.522419  5149 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:27.530462  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.485492  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:27.488117  5070 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:27.541901  5114 leader_election.cc:400] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:27.549036  5114 leader_election.cc:403] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:27.544119  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:27.545387  5041 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.68:43449 timed out after 1.572s (SENT) [suppressed 59 similar messages]
I20260709 12:23:27.489262  6269 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:27.509518  6258 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:27.559047  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:27.559463  6258 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:27.560117  6259 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:27.560631  5114 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:27.561713  5150 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:27.562095  6259 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:27.562847  6259 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:27.547271  6082 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:27.563957  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.564770  6082 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:27.565121  6082 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:27.565474  6082 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:27.510047  6293 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:27.567210  6293 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:27.567353  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.493373  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.557915  6127 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:27.498337  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.505206  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.569690  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:27.570930  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.572856  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.574900  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.580703  5114 leader_election.cc:400] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:27.492961  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:27.577911  5071 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:27.581133  5114 leader_election.cc:403] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:27.582227  5069 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:27.565351  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:27.522650  6299 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 12:23:27.592495  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.603209  5946 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:27.588711  5071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4935fc8f35664324a253df9804c8074d"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49646
I20260709 12:23:27.589941  5114 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:27.595419  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.605406  5947 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:27.596809  5224 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:27.568648  6262 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:27.573939  6127 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:27.604655  5071 raft_consensus.cc:606] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.609674  5947 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:27.610749  5115 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:27.602125  5067 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:27.612365  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.613183  5067 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:27.614173  5215 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:27.614676  5067 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:27.616248  5067 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:27.616925  5945 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:27.620621  5073 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:27.612581  6299 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:27.623054  6299 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:27.624387  6299 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:27.633719  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:27.642940  6301 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:27.634099  6299 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:27.634502  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:27.634847  5075 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:27.635098  6127 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:27.636041  6207 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:27.635763  6136 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:27.637079  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.734547  5145 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:27.606244  5070 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:27.639462  6236 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:27.586400  6259 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:27.644754  5069 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:23:27.645464  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:27.650545  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 230230us
I20260709 12:23:27.587756  6309 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:27.651973  6115 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:27.590590  6267 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:27.608175  6066 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:27.643594  5900 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:27.731868  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 186532us
I20260709 12:23:27.607690  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:27.759153  6196 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:27.774798  5224 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:27.759362  6309 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:27.762025  5074 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49646
I20260709 12:23:27.786307  5074 raft_consensus.cc:606] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:27.786307  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 175679us
I20260709 12:23:27.786846  5074 raft_consensus.cc:614] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:23:27.788161  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:27.766947  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 159359us
I20260709 12:23:27.791345  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.771199  6301 consensus_queue.cc:1059] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:27.772624  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:27.773434  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.747500  6236 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:27.754025  6315 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.164s	user 0.001s	sys 0.002s
I20260709 12:23:27.802261  5190 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:27.806093  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:27.766994  6136 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:27.796495  6181 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:27.807888  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.809021  6301 raft_consensus.cc:3055] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:27.838173  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.809914  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:27.824806  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 530) took 1371 ms. Trace:
I20260709 12:23:27.774658  6259 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:27.827147  5145 raft_consensus.cc:2393] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:27.787067  6258 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:27.750406  6246 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:27.841651  5076 rpcz_store.cc:276] 0709 12:23:26.465370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.465563 (+   193us) service_pool.cc:224] Handling call
0709 12:23:26.468975 (+  3412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:26.642797 (+173822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:26.642833 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:26.642911 (+    78us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:26.642939 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.824515 (+1181576us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54006a2554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:27.824569 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:27.824741 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399172352}
I20260709 12:23:27.849439  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:27.818372  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:27.844195  6259 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:27.853363  6259 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:27.841527  6066 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:27.854003  5067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49646
I20260709 12:23:27.854271  6066 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:27.855240  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:27.856673  5145 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:27.824638  6136 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:27.839679  6267 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:27.833922  6181 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:27.827237  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 183755us
I20260709 12:23:27.842137  6301 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:27.856612  5216 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:27.887328  5075 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:27.856868  5222 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:27.889295  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:27.901216  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 99102.5us
I20260709 12:23:27.857755  6066 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:27.857832  6309 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:27.887109  6258 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:27.884081  5067 raft_consensus.cc:606] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.888357  6181 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:27.854859  6082 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:27.901111  6267 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:27.897948  6227 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:27.906432  6306 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:27.933027  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:27.938159  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 100085us
I20260709 12:23:27.915973  5067 raft_consensus.cc:614] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:27.919358  6191 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:27.912288  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:27.921168  6309 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:27.966645  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.970033  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:27.971415  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 600) took 1103 ms. Trace:
I20260709 12:23:27.971848  5142 rpcz_store.cc:276] 0709 12:23:26.868402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.076237 (+207835us) service_pool.cc:224] Handling call
0709 12:23:27.076606 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:27.286008 (+209402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.971151 (+685143us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54004fa954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:27.971364 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479198592}
I20260709 12:23:27.972607  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.947431  6080 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:27.954890  6306 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:27.957556  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.953526  6136 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:27.983254  6136 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:27.983717  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.985098  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.979254  5939 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:27.986295  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.987464  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:27.988219  5216 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:27.989189  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.989864  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:27.989746  6172 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:27.973798  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:27.990136  6321 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:23:27.991187  6309 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:27.970137  6322 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:23:27.991423  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:27.992231  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 80) took 1205 ms. Trace:
I20260709 12:23:27.994655  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:27.996138  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.997509  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.999830  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:28.000746  5216 raft_consensus.cc:2393] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:28.002580  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.005944  5941 raft_consensus.cc:2410] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:23:28.007912  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.963932  6191 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:28.012883  6191 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:28.009626  6259 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:28.004396  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:27.976147  6320 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.186s	user 0.001s	sys 0.001s
I20260709 12:23:28.013432  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 237) took 1235 ms. Trace:
I20260709 12:23:28.068869  5944 rpcz_store.cc:276] 0709 12:23:26.777992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.782015 (+  4023us) service_pool.cc:224] Handling call
0709 12:23:27.615334 (+833319us) spinlock_profiling.cc:243] Waited 7.2 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:28.012480 (+397146us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416e9c 00007fd8a5416a5b 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:28.012719 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.012951 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:28.012984 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:28.013120 (+   136us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:28.013161 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.013232 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:28.013364 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95538304}
I20260709 12:23:27.993373  6246 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:28.022245  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 64) took 1749 ms. Trace:
W20260709 12:23:28.067596  6310 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.604s	user 0.006s	sys 0.000s
I20260709 12:23:28.117445  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:28.089035  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 184409us
I20260709 12:23:28.120157  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:28.118120  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 72) took 1447 ms. Trace:
I20260709 12:23:28.121189  6136 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:28.121851  6136 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.122408  6136 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:28.120312  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:28.124112  6202 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.921s	user 0.008s	sys 0.007s
I20260709 12:23:28.124680  6202 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.126021  5147 raft_consensus.cc:2393] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:28.121523  5143 rpcz_store.cc:276] 0709 12:23:26.670535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.875873 (+205338us) service_pool.cc:224] Handling call
0709 12:23:26.881065 (+  5192us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:26.881570 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:26.881598 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:26.882279 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 12:23:27.276705 (+394426us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:27.276778 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.276907 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:27.900549 (+623642us) spinlock_profiling.cc:243] Waited 45.1 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:23:28.117641 (+217092us) raft_consensus.cc:1685] finished
0709 12:23:28.117676 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:28.118028 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":820,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":132,"spinlock_wait_cycles":103843712,"thread_start_us":393806,"threads_started":1,"wal-append.queue_time_us":553677}
I20260709 12:23:28.127193  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 83) took 1340 ms. Trace:
I20260709 12:23:28.127748  5069 rpcz_store.cc:276] 0709 12:23:26.787260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.601135 (+813875us) service_pool.cc:224] Handling call
0709 12:23:27.644482 (+ 43347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:27.746567 (+102085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.126936 (+380369us) spinlock_profiling.cc:243] Waited 42.8 ms on lock 0x7b54006effd0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:28.127121 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98444672}
I20260709 12:23:28.128155  6202 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:28.129575  6202 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.130244  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:28.131054  5220 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:28.071668  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:27.975991  6334 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:27.974938  6258 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:27.992678  5068 rpcz_store.cc:276] 0709 12:23:26.786399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.245213 (+458814us) service_pool.cc:224] Handling call
0709 12:23:27.309843 (+ 64630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:27.348284 (+ 38441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.373199 (+ 24915us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:27.421404 (+ 48205us) log.cc:844] Serialized 26 byte log entry
0709 12:23:27.748910 (+327506us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:27.749059 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.749419 (+   360us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:27.991952 (+242533us) raft_consensus.cc:1685] finished
0709 12:23:27.991992 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:27.992160 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3797,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":17799,"thread_start_us":279222,"threads_started":1,"wal-append.queue_time_us":405019}
W20260709 12:23:28.073241  5221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 505) took 1638 ms (client timeout 1823 ms). Trace:
W20260709 12:23:28.147746  5221 rpcz_store.cc:269] 0709 12:23:26.434655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.436113 (+  1458us) service_pool.cc:224] Handling call
0709 12:23:28.072932 (+1636819us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54004830d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:28.073120 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710655872}
I20260709 12:23:28.149412  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.149523  6066 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:28.150826  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:28.151729  5221 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:28.152063  6066 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:28.152643  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:28.015832  5216 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:28.154613  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:28.077943  6319 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.328s	user 0.002s	sys 0.001s
W20260709 12:23:28.079464  6316 log.cc:927] Time spent T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.593s	user 0.002s	sys 0.001s
I20260709 12:23:28.159436  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.160331  5218 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:28.169700  5148 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
W20260709 12:23:28.171190  6293 consensus_meta.cc:287] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.603s	user 0.002s	sys 0.012s
I20260709 12:23:28.171829  6293 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:28.173960  5146 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:28.080401  6328 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.176267  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 64) took 1427 ms. Trace:
I20260709 12:23:28.176862  5223 rpcz_store.cc:276] 0709 12:23:26.748603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.749731 (+  1128us) service_pool.cc:224] Handling call
0709 12:23:26.770295 (+ 20564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:27.066415 (+296120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.066445 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:27.076968 (+ 10523us) log.cc:844] Serialized 26 byte log entry
0709 12:23:27.419925 (+342957us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:27.419965 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.420035 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:28.078733 (+658698us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:23:28.175794 (+ 97061us) raft_consensus.cc:1685] finished
0709 12:23:28.175827 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:28.176160 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":50137,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":4682,"spinlock_wait_cycles":25163520,"thread_start_us":592202,"threads_started":2,"wal-append.queue_time_us":392076}
W20260709 12:23:28.080989  6327 log.cc:927] Time spent T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:23:28.154630  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.181854  6309 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:28.182525  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.182452  6309 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:28.184844  6309 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:28.184659  6127 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:28.185491  6309 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:28.186066  6309 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.187733  6127 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:28.188136  6309 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:28.188764  6309 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:28.188796  6306 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.189234  6309 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:28.195266  6309 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:28.195770  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:28.081912  6329 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:23:28.082320  5940 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
W20260709 12:23:28.198472  6299 consensus_meta.cc:287] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.003s
I20260709 12:23:28.198897  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 238) took 1420 ms. Trace:
I20260709 12:23:28.083742  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:28.199599  5944 raft_consensus.cc:2410] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:28.200290  5942 raft_consensus.cc:2410] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:23:28.199728  6301 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:28.085767  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.202050  5224 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:28.202739  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 61) took 1480 ms. Trace:
I20260709 12:23:28.203195  5219 rpcz_store.cc:276] 0709 12:23:26.722204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.722456 (+   252us) service_pool.cc:224] Handling call
0709 12:23:28.073662 (+1351206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.073946 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:28.073984 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:28.074048 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:28.074088 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.074158 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:28.202560 (+128402us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54004830d0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:28.202674 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3106214784}
W20260709 12:23:28.090777  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 113255us
I20260709 12:23:28.093801  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:28.204406  5188 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:23:28.205231  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 111829us
I20260709 12:23:28.205317  6294 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:28.205454  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:28.206000  6294 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:28.206136  5898 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:28.206619  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:28.207018  6294 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:28.098408  6262 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:28.015148  6259 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:28.018755  6082 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:28.113090  5144 rpcz_store.cc:276] 0709 12:23:26.255902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.416587 (+160685us) service_pool.cc:224] Handling call
0709 12:23:26.443627 (+ 27040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:26.740486 (+296859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.005276 (+1264790us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b5400582854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:28.005553 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449114368}
I20260709 12:23:28.134696  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:28.156896  6136 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:28.235740  6136 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:28.211793  6262 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:28.237298  5948 raft_consensus.cc:2368] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:28.178751  5070 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:28.201279  6301 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:28.207437  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:28.085837  6181 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.212901  5218 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:28.218312  6309 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.199333  5940 rpcz_store.cc:276] 0709 12:23:26.778394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.782111 (+  3717us) service_pool.cc:224] Handling call
0709 12:23:27.610463 (+828352us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:28.198792 (+588329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18130688}
I20260709 12:23:28.215035  6299 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:28.221190  6293 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:28.240199  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:28.231328  5939 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:28.249816  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:28.229936  6243 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:23:28.260679  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 44716.2us
I20260709 12:23:28.261300  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:28.264719  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:28.266868  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:28.197242  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:28.269263  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:28.271953  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:28.153477  5221 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:28.241616  6273 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:28.268448  6172 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:28.242416  6136 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:28.277256  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:28.164587  6236 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:28.272889  6309 raft_consensus.cc:2764] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:23:28.242172  6181 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:28.265350  6068 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:28.237514  5220 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:28.268185  6191 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.272157  6243 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:28.313138  6192 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:28.289870  5945 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:28.275611  6299 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:23:28.290258  6080 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:28.313478  6273 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:28.387270  6273 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:28.313653  6172 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:28.317945  6068 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:28.366276  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:28.367092  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:28.369897  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:28.370827  5067 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:28.371573  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:28.394938  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:28.396114  5071 raft_consensus.cc:2368] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:28.373904  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:28.369791  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 605) took 1496 ms. Trace:
I20260709 12:23:28.377159  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:28.306599  6082 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 12:23:28.307189  6261 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:23:28.271566  6190 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 12:23:28.318064  6281 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:28.318859  6127 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:28.385427  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:28.319928  6309 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:23:28.372689  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:28.358990  6136 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:28.409559  5074 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:28.435336  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 50042.8us
I20260709 12:23:28.436269  5042 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:28.364938  6243 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:28.389204  6192 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:28.390218  6080 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:28.441737  5042 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:28.411233  6222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.030s	sys 0.041s
I20260709 12:23:28.438814  6246 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:28.434253  6127 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.438185  5041 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:28.411163  5073 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 5
W20260709 12:23:28.502991  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 508370us
W20260709 12:23:28.447712  6222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.030s	sys 0.041s
W20260709 12:23:28.490396  6307 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.000s	user 0.014s	sys 0.029s
I20260709 12:23:28.509766  5218 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:28.434506  6281 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:28.443259  6123 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.505184  5074 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:28.514078  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:28.491246  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 219329us
I20260709 12:23:28.409138  5148 rpcz_store.cc:276] 0709 12:23:26.873643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.005071 (+131428us) service_pool.cc:224] Handling call
0709 12:23:28.369572 (+1364501us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000ebd54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:28.369695 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2675138944}
I20260709 12:23:28.527731  6181 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:28.534267  6222 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:28.535511  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.536460  5216 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:28.537011  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:28.537741  5215 raft_consensus.cc:2376] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3. Re-sending same reply.
I20260709 12:23:28.538619  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:28.538815  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.540637  5217 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:28.442576  6136 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.520759  6236 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:28.543869  6136 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:28.522053  6307 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.015s	sys 0.030s
I20260709 12:23:28.546329  6259 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:28.540095  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:28.548787  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:28.550148  6337 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:23:28.550660  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:28.551350  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.552655  6328 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:28.553103  6328 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:28.553448  6328 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:28.454174  6172 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:28.557898  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.559486  6191 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:28.560137  6191 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:28.560609  6259 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:28.561129  6259 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:28.561301  6333 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:23:28.561517  6259 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:28.562649  6191 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:28.563189  6191 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:28.559554  6336 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:28.563647  6191 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.567042  6068 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:28.568993  6191 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:28.565328  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.565791  6299 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:23:28.570240  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:28.566594  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:28.570544  6192 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:28.557092  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:28.572273  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:28.575204  6209 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.575937  6209 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.576154  6080 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:28.578830  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:28.583530  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 141942us
I20260709 12:23:28.585076  5042 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:28.585495  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.599468  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.600660  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.602108  6306 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:28.602458  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.602703  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.603118  5145 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:28.603543  5148 raft_consensus.cc:2368] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:28.605111  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.603916  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:28.604056  6306 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:28.614027  6306 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:28.589573  6123 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:28.614781  6068 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:28.615013  6350 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:28.615448  6068 raft_consensus.cc:697] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:28.615686  6350 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.616384  6115 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:28.596282  6222 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:28.617964  6222 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:28.618664  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:28.620090  6222 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:28.615039  6306 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.621816  6127 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:28.585011  6246 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:23:28.622387  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.622769  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:28.623174  5941 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:28.604542  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:28.624893  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:28.625692  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:28.622133  6306 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:28.652209  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:28.626538  5149 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:28.653227  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.623023  6281 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:28.662989  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.663347  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.663872  5149 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:28.653905  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.667120  5150 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:28.593861  6191 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:28.667934  6191 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.668263  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:28.632287  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:28.669028  5069 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:28.654240  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.670210  5147 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:28.670787  6191 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:28.654556  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:28.673403  5143 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:28.656317  5142 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:28.664646  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:28.597929  5189 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:28.640252  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:28.644588  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:28.680847  5076 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:28.624696  5145 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:23:28.647282  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.647574  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:28.616458  6068 consensus_queue.cc:237] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:28.683631  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 85806.3us
I20260709 12:23:28.686050  5150 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:28.689900  5946 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:28.711546  5947 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:28.713488  5069 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:28.664625  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.734639  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:28.723875  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.651772  5073 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:28.739491  5223 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:28.656047  5148 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:28.715221  6222 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:28.741413  6222 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:28.744340  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.725255  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.654246  5221 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:28.746001  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:28.746166  6301 consensus_meta.cc:287] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.507s	user 0.003s	sys 0.009s
I20260709 12:23:28.653592  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:28.687394  6127 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:28.607077  5900 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:28.785154  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 48651us
I20260709 12:23:28.704988  6209 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.808257  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:28.773377  6236 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.772482  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:28.704984  6309 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.104s
I20260709 12:23:28.661921  6262 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:28.735383  6336 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:23:28.773149  5142 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:28.829202  5149 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
W20260709 12:23:28.831660  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 272785us
I20260709 12:23:28.825665  5188 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:28.818441  5144 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:28.843024  6311 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:28.844220  6311 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:28.843742  6236 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:23:28.842597  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 404480us
I20260709 12:23:28.817905  6301 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:28.854101  5142 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:28.801363  6328 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.827782  6116 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.807008  6127 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:28.835062  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 233546us
W20260709 12:23:28.861248  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 224782us
I20260709 12:23:28.868336  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:28.872051  5041 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:28.862651  6236 raft_consensus.cc:697] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:28.876739  5076 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:28.871398  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:28.898058  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.898501  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.900013  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.872591  6080 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.875765  5073 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:28.909603  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:28.911536  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 54391.9us
I20260709 12:23:28.924940  5188 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:28.925510  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.928162  5076 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:28.918027  6336 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.931974  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:28.931414  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 51218.5us
I20260709 12:23:28.933176  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:28.876129  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:28.872714  6353 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 12:23:28.941437  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:28.937677  5042 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:28.902700  6243 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:23:28.943028  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:28.929214  6116 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:28.947028  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:28.904969  6259 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.950659  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:28.931212  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:28.957520  5222 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:28.957911  6202 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:28.958712  6202 consensus_queue.cc:1243] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:23:28.960242  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:28.927793  6361 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:28.952746  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:28.936810  6354 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:28.988982  5145 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:28.933996  6336 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:28.936193  6080 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:23:28.963148  6326 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:28.935931  6273 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 12:23:28.982381  6351 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:23:29.050333  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:29.028405  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 158351us
I20260709 12:23:28.983399  6354 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:29.051573  6172 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:23:28.988412  6352 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:23:28.938329  6236 consensus_queue.cc:237] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:29.086759  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 149185us
I20260709 12:23:28.991026  6116 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:28.971858  6234 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 12:23:29.086086  6328 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:29.107002  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:29.050973  6068 raft_consensus.cc:2764] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:23:29.107161  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:29.078879  5148 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:29.027526  6309 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 12:23:29.087617  6262 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:29.087791  6202 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:29.127790  6202 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:29.130333  6112 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:29.130904  6112 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:29.131655  6336 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:29.131855  6112 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:29.127832  5215 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:29.130332  6202 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:29.115123  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:29.115933  5042 leader_election.cc:400] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:29.137739  5042 leader_election.cc:403] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:29.088115  6354 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.103649  6127 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:29.126037  6359 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.163s	user 0.001s	sys 0.001s
I20260709 12:23:29.122109  6373 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:23:29.098094  6172 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:29.108830  6190 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:23:29.110821  6369 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:29.174778  6068 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:29.175694  6246 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
W20260709 12:23:29.225775  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 112258us
I20260709 12:23:29.150167  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:29.177655  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:29.179908  6127 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:29.183319  6259 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:29.197238  6357 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 12:23:29.166950  6136 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:29.199556  5943 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:29.165927  6368 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:29.200603  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 85574.7us
I20260709 12:23:29.304733  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:29.160804  6202 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.209849  6191 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:29.176790  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 254) took 1301 ms. Trace:
I20260709 12:23:29.212527  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:29.173488  6172 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.311630  5944 rpcz_store.cc:276] 0709 12:23:27.875097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.154228 (+279131us) service_pool.cc:224] Handling call
0709 12:23:29.176453 (+1022225us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b54001a0254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:29.176696 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100442368}
I20260709 12:23:29.316533  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:29.317216  5941 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:29.394380  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.402789  5074 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:29.404178  5940 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:23:29.405236  5940 pending_rounds.cc:85] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 1
I20260709 12:23:29.301699  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.215406  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:29.413949  5939 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:29.415199  5072 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:29.183410  6350 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:29.218439  6068 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:29.416705  6068 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:29.416805  6362 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.317s	user 0.003s	sys 0.001s
I20260709 12:23:29.416951  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.418452  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.198035  6123 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:29.309281  6191 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:29.312687  6202 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:29.297024  6136 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:29.294426  6082 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:29.333128  6309 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:29.210276  6116 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:29.312162  6190 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.298967  6267 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:29.472088  6267 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:29.473147  5067 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:29.339583  6368 consensus_queue.cc:1059] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:29.474445  5073 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:29.475270  6368 raft_consensus.cc:3055] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:29.475732  6368 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:29.236158  5220 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:29.380476  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.453217  6191 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:29.377146  6259 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.385561  6181 raft_consensus.cc:3055] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:29.387161  6349 raft_consensus.cc:3055] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:29.393924  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.394025  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.493955  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.494688  5069 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
W20260709 12:23:29.495288  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:29.395699  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 521299us
I20260709 12:23:29.396461  6361 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:23:29.399695  6301 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.001s
I20260709 12:23:29.405916  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:29.474470  6267 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:29.395779  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.314309  5900 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:29.507010  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 192747us
I20260709 12:23:29.426232  5216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:29.431483  6209 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:29.432801  5142 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:29.433303  6354 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:29.443152  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:29.445613  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 196493us
I20260709 12:23:29.466084  4897 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43234 (request call id 123) took 3869 ms. Trace:
I20260709 12:23:29.465564  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:29.445119  6336 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:29.445147  6123 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:29.384239  6311 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:29.447448  5940 pending_rounds.cc:107] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:23:29.507392  6301 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:29.487056  5898 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:29.394548  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.497804  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.466719  6309 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:29.453497  6306 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:23:29.295648  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:29.476884  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:29.499958  5070 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:29.515791  6369 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:29.516206  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:29.491561  6338 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:29.490460  6181 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:29.469458  6082 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:29.492518  6349 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:29.479854  6190 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:29.532931  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:29.539122  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 52185.7us
I20260709 12:23:29.540292  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:29.525692  5073 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:29.538420  5075 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:29.517978  6358 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:29.492199  6259 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:29.532361  6336 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:23:29.558943  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 84989.5us
I20260709 12:23:29.564268  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.609958  5074 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:29.521118  5068 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:29.548671  6309 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:29.545825  6350 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:29.554720  6338 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:29.607569  5142 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:29.611225  5898 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:29.622686  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 82508.2us
I20260709 12:23:29.624440  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:29.584241  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:29.560148  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:29.604562  5141 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:29.541934  5940 raft_consensus.cc:2889] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:29.537268  6172 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:29.627280  5147 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:23:29.627552  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 151572us
I20260709 12:23:29.645298  6127 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:29.646975  5114 leader_election.cc:400] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:29.754554  5114 leader_election.cc:403] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:23:29.751757  5188 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:29.624414  6350 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:29.672065  6367 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.531543  6136 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:29.743791  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 132690us
I20260709 12:23:29.750458  5219 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
W20260709 12:23:29.741706  6384 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
W20260709 12:23:29.665732  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 56695.7us
I20260709 12:23:29.759531  6082 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:29.759414  6068 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:29.760850  6370 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:29.760587  6338 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:29.629561  6267 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:29.753137  6307 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:29.773011  6370 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:29.770627  6367 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:29.777907  6383 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 12:23:29.777920  6338 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:29.779577  6367 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:29.765612  6350 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.785666  6191 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:29.768476  6388 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:29.769521  6202 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:29.765905  6349 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:29.778168  6259 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:29.798209  6373 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:23:29.800740  5941 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:29.762925  6181 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:29.801585  6301 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:29.802148  6181 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:29.802309  6301 consensus_queue.cc:1243] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:23:29.764518  6336 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:29.794124  6267 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:29.805913  6336 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:29.810523  6190 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:29.789502  6358 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.865613  6358 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:29.891294  6350 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:29.895002  6350 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:29.524669  4897 rpcz_store.cc:276] 0709 12:23:25.589204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.589381 (+   177us) service_pool.cc:224] Handling call
0709 12:23:29.458675 (+3869294us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:26.330793 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:26.368572 (+ 37779us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:26.375221 (+  6649us) write_op.cc:435] Acquired schema lock
0709 12:23:26.375241 (+    20us) tablet.cc:662] Decoding operations
0709 12:23:26.449753 (+ 74512us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:26.449825 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:26.449855 (+    30us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:23:26.450416 (+   561us) tablet.cc:701] Row locks acquired
0709 12:23:26.450431 (+    15us) write_op.cc:260] PREPARE: finished
0709 12:23:26.450598 (+   167us) write_op.cc:270] Start()
0709 12:23:26.450729 (+   131us) write_op.cc:276] Timestamp: P: 1783599806450552 usec, L: 0
0709 12:23:26.450747 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:23:26.451497 (+   750us) log.cc:844] Serialized 2074 byte log entry
0709 12:23:27.341940 (+890443us) op_driver.cc:464] REPLICATION: finished
0709 12:23:27.852834 (+510894us) write_op.cc:301] APPLY: starting
0709 12:23:27.852957 (+   123us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:27.853144 (+   187us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:27.853276 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:27.854130 (+   854us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:27.854343 (+   213us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:23:27.854389 (+    46us) write_op.cc:312] APPLY: finished
0709 12:23:27.882538 (+ 28149us) log.cc:844] Serialized 46 byte log entry
0709 12:23:27.897268 (+ 14730us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:27.905030 (+  7762us) write_op.cc:454] Released schema lock
0709 12:23:27.906696 (+  1666us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:27.948901 (+ 42205us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:27.953426 (+  4525us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1131,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":496256,"apply.run_cpu_time_us":3271,"apply.run_wall_time_us":76948,"num_ops":5,"prepare.queue_time_us":142768,"prepare.run_cpu_time_us":19441,"prepare.run_wall_time_us":184755,"raft.queue_time_us":403990,"raft.run_cpu_time_us":24631,"raft.run_wall_time_us":244303,"replication_time_us":890994,"thread_start_us":745058,"threads_started":4,"wal-append.queue_time_us":135056,"wal-append.run_cpu_time_us":34706,"wal-append.run_wall_time_us":2350346}]]}
W20260709 12:23:29.626947  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 129918us
I20260709 12:23:29.743755  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:29.961576  5075 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:29.974417  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:29.976322  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 329386us
W20260709 12:23:29.994228  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:29.977905  5071 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:29.978595  5141 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:29.980177  5142 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:23:29.981922  5148 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:29.984154  5144 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:29.997174  5142 pending_rounds.cc:85] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 1
I20260709 12:23:29.993963  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:29.790032  5943 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:29.769093  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 521) took 1182 ms. Trace:
I20260709 12:23:30.036713  6392 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:30.003054  5142 pending_rounds.cc:107] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 2.2
W20260709 12:23:30.044319  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 280768us
W20260709 12:23:30.047487  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.047670  5898 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:30.048732  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 638) took 1392 ms. Trace:
I20260709 12:23:30.049464  6350 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:30.049345  5143 rpcz_store.cc:276] 0709 12:23:28.656074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.704343 (+ 48269us) service_pool.cc:224] Handling call
0709 12:23:28.761727 (+ 57384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.865383 (+103656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:28.865419 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:28.865466 (+    47us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:28.865496 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.048114 (+1182618us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54003cda54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:30.048219 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.048487 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237843200}
W20260709 12:23:30.050984  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.052244  6191 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:30.057806  6398 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:30.059090  6398 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:30.040442  5142 raft_consensus.cc:2889] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:30.062006  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 65899.9us
I20260709 12:23:30.068316  5073 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:30.068596  6399 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.069489  6399 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.070950  5073 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:30.014837  6309 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:30.012204  5217 rpcz_store.cc:276] 0709 12:23:28.583524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.584655 (+  1131us) service_pool.cc:224] Handling call
0709 12:23:29.201676 (+617021us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:29.201965 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.202303 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:23:29.202762 (+   459us) log.cc:844] Serialized 14 byte log entry
0709 12:23:29.618223 (+415461us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:29.631088 (+ 12865us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:29.742223 (+111135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.745826 (+  3603us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:29.766095 (+ 20269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11129,"prepare.queue_time_us":11536,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":9240,"raft.queue_time_us":31548,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2782,"spinlock_wait_cycles":14303616,"thread_start_us":375427,"threads_started":2,"wal-append.queue_time_us":268651}
I20260709 12:23:30.073419  5073 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:23:30.103021  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 55756.6us
I20260709 12:23:30.105435  6273 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:30.105839  6115 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:30.113185  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.113723  5898 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:30.103248  6112 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:30.116751  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:23:30.097988  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:30.099124  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 105572us
I20260709 12:23:30.083199  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 639) took 1426 ms. Trace:
I20260709 12:23:30.096733  6398 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:30.094175  5041 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:30.101349  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 244) took 1158 ms. Trace:
I20260709 12:23:30.079144  6307 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:30.196986  5149 rpcz_store.cc:276] 0709 12:23:28.656319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.773967 (+117648us) service_pool.cc:224] Handling call
0709 12:23:28.828983 (+ 55016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.829769 (+   786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.082574 (+1252805us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b54000e2250. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:30.082910 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38562176}
I20260709 12:23:30.097252  6172 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:30.076121  6309 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:30.099086  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 520) took 1520 ms. Trace:
W20260709 12:23:30.214392  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 120417us
I20260709 12:23:30.218379  6127 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:30.219323  6127 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:30.203459  6181 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:30.094527  6400 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:23:30.226848  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 110401us
I20260709 12:23:30.227818  6358 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:30.230257  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 636) took 1574 ms. Trace:
I20260709 12:23:30.230841  5145 rpcz_store.cc:276] 0709 12:23:28.655599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.704190 (+ 48591us) service_pool.cc:224] Handling call
0709 12:23:28.739277 (+ 35087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.989500 (+250223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.229503 (+1240003us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400185954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:30.229986 (+   483us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":573702400}
I20260709 12:23:30.233644  6222 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:30.234319  6127 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:30.234655  6281 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:30.235316  6281 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:30.238238  5074 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:30.242354  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:30.243659  5221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35624 (request call id 713) took 1346 ms (client timeout 1674 ms). Trace:
W20260709 12:23:30.244091  5221 rpcz_store.cc:269] 0709 12:23:28.897337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.901220 (+  3883us) service_pool.cc:224] Handling call
0709 12:23:30.241845 (+1340625us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:30.243548 (+  1703us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8915328}
I20260709 12:23:30.236967  6281 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:30.248919  6261 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:30.249927  6261 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:30.252911  6261 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:30.257185  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.257522  5948 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:30.221904  6191 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:30.283329  6400 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:30.297744  6123 consensus_meta.cc:287] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.752s	user 0.012s	sys 0.013s
I20260709 12:23:30.298427  6123 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:30.299755  6408 log.cc:927] Time spent T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.053s	user 0.002s	sys 0.000s
I20260709 12:23:30.285349  6400 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:30.301091  6236 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:23:30.304407  6400 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:30.203186  5944 rpcz_store.cc:276] 0709 12:23:28.942622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.316351 (+373729us) service_pool.cc:224] Handling call
0709 12:23:29.316693 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.756023 (+439330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.762526 (+  6503us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:29.767451 (+  4925us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:29.770836 (+  3385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.100709 (+329873us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b5400003f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:30.100783 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.101077 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10156,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":6036,"spinlock_wait_cycles":709755136}
I20260709 12:23:30.216403  6398 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:30.284277  6397 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:23:30.294624  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 273) took 1372 ms. Trace:
W20260709 12:23:30.268108  5072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.010s	sys 0.007s
W20260709 12:23:30.322252  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 62225.7us
I20260709 12:23:30.265014  6191 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:30.211640  5220 rpcz_store.cc:276] 0709 12:23:28.578903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.581107 (+  2204us) service_pool.cc:224] Handling call
0709 12:23:30.098493 (+1517386us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400164c54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:30.098765 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422124160}
I20260709 12:23:30.334178  6136 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:30.334544  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:30.335698  5070 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:30.338280  5188 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:30.339582  6381 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:30.340083  6381 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:30.340489  6381 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:30.343286  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:30.344075  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:30.349162  5072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.011s	sys 0.008s
I20260709 12:23:30.276558  6368 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:30.326670  5948 rpcz_store.cc:276] 0709 12:23:28.896070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.896244 (+   174us) service_pool.cc:224] Handling call
0709 12:23:30.255863 (+1359619us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:30.256533 (+   670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.259988 (+  3455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.268216 (+  8228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13739776}
I20260709 12:23:30.326336  6123 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:30.347851  6191 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:30.350449  6068 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:30.374303  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 457770us
I20260709 12:23:30.379454  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:30.376850  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 453250us
I20260709 12:23:30.385687  6267 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.386528  6267 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:30.387135  6336 consensus_meta.cc:287] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.581s	user 0.018s	sys 0.003s
I20260709 12:23:30.388497  5900 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:30.369580  6398 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.382598  6172 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.395568  6172 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:30.397929  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.399096  5948 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:30.384593  6400 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:23:30.403359  6309 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:30.404621  6376 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:30.405695  6349 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:30.406908  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:30.407982  6376 raft_consensus.cc:3055] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:30.408456  6376 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:30.409348  6202 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.389611  6301 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.410054  6202 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.410473  6301 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.411114  6376 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.412714  6376 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:30.413003  6301 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:30.416132  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.417235  5941 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:30.418900  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.419698  6209 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:30.420730  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.421947  6389 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:30.422420  6389 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:30.423544  6399 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:30.423691  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:30.423916  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:30.389505  6267 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:30.426359  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.426884  6354 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:30.395607  6348 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:30.437006  6348 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.412717  6202 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:30.434593  6368 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:30.467483  6411 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:23:30.469641  5216 raft_consensus.cc:606] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:30.476410  6262 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:30.477077  5070 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:30.477283  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:30.478065  6389 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.479817  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:30.480818  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.480697  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:30.481355  6294 raft_consensus.cc:3055] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:30.482321  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 113) took 1069 ms. Trace:
I20260709 12:23:30.483124  6263 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:30.486044  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:30.490365  6407 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:23:30.490882  6192 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.491124  6234 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.495870  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 276) took 1521 ms. Trace:
I20260709 12:23:30.496364  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.498039  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.500095  4898 catalog_manager.cc:5697] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:30.501504  6311 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:30.542068  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.544155  5221 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:30.503890  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.514237  5216 raft_consensus.cc:614] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:30.548895  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57004 (request call id 0) took 1144 ms. Trace:
I20260709 12:23:30.514377  6207 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:23:30.516388  5947 raft_consensus.cc:2410] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:23:30.512938  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.516649  5075 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:30.558017  6373 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.517865  6348 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:30.559032  6373 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:30.518280  6270 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:23:30.518024  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.519984  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:30.564934  6299 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:30.565332  6301 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.520386  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.565735  6299 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:30.565999  6301 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:30.566520  5223 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:30.520414  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:30.568328  5069 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:30.571072  5942 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:30.571455  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.572111  5947 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:30.572528  5942 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:30.568394  6301 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:30.523147  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:30.574254  6373 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:30.574605  6392 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:30.525257  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:30.575233  6392 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:30.575455  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.575721  6392 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:30.526320  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:30.527170  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:30.532907  6192 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:30.528030  5948 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
W20260709 12:23:30.582906  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 56021.3us
I20260709 12:23:30.519177  5071 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:23:30.574699  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 51880.6us
I20260709 12:23:30.570943  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:30.534051  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 152883us
I20260709 12:23:30.576648  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:30.463958  5041 connection.cc:256] RPC call timeout handler was delayed by -1.3081327660001989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 12:23:30.467569  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 79314.5us
I20260709 12:23:30.558581  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:30.535820  4898 catalog_manager.cc:5697] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:30.528909  5072 rpcz_store.cc:276] 0709 12:23:29.412596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.526346 (+113750us) service_pool.cc:224] Handling call
0709 12:23:29.526799 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:29.527175 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.527222 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:29.527887 (+   665us) log.cc:844] Serialized 26 byte log entry
0709 12:23:30.383912 (+856025us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:30.394547 (+ 10635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.394618 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.482005 (+ 87387us) raft_consensus.cc:1685] finished
0709 12:23:30.482026 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.482241 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":808205,"threads_started":1,"wal-append.queue_time_us":814417}
I20260709 12:23:30.612010  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.612674  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.506356  6172 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:30.619719  5041 connection.cc:256] RPC call timeout handler was delayed by -1.0417291059998206s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33290000000000003s for a response
I20260709 12:23:30.503710  6398 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:30.528280  6294 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:30.626233  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.519280  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.622879  6267 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:30.631273  4898 catalog_manager.cc:5697] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:30.635759  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:30.639572  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:30.514663  6262 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:30.625824  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.515791  6269 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.116s
I20260709 12:23:30.532967  6234 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:30.647850  5900 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:30.530898  6243 consensus_queue.cc:1175] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9847a7797ddc4df689a1ab0911af8785" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 2 } }
I20260709 12:23:30.651696  6378 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:30.684473  6378 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:30.685676  6378 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.681921  5067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 109) took 1270 ms. Trace:
I20260709 12:23:30.686673  5067 rpcz_store.cc:276] 0709 12:23:29.411138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.474281 (+ 63143us) service_pool.cc:224] Handling call
0709 12:23:29.474681 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:29.475154 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.475198 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:30.519952 (+1044754us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b2c001a9d48. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:23:30.520612 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 12:23:30.632113 (+111501us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:30.635864 (+  3751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.670117 (+ 34253us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.681182 (+ 11065us) raft_consensus.cc:1685] finished
0709 12:23:30.681209 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.681654 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1446,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":11725,"spinlock_wait_cycles":51548416,"thread_start_us":99347,"threads_started":1,"wal-append.queue_time_us":153017}
I20260709 12:23:30.688711  5217 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:30.689216  6389 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:30.692370  6191 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:30.637006  6192 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:30.656097  6262 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:30.659770  5041 leader_election.cc:400] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:30.661042  5948 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
W20260709 12:23:30.661576  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.664551  5939 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:23:30.722288  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.664381  6261 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:30.665992  5942 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:30.666368  6263 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:30.640071  6299 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:30.673151  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 44952.3us
I20260709 12:23:30.674643  6398 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:30.549499  5216 rpcz_store.cc:276] 0709 12:23:29.404977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.426058 (+ 21081us) service_pool.cc:224] Handling call
0709 12:23:30.548511 (+1122453us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b540046ffd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a8535853 00007fd8ad50360a 00007fd8a6a897d4 00007fd8a6a89762 00007fd8a6a89691 00007fd8a6a8960c 00007fd8a6a888c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:30.548732 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2168673024}
I20260709 12:23:30.676724  6236 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:30.677441  6380 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:30.678516  6306 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:30.679013  5941 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:30.679667  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:30.689709  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 47515.7us
I20260709 12:23:30.652534  6294 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.693063  6112 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.739198  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.739375  6380 raft_consensus.cc:3055] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:30.739454  6112 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:30.740087  6380 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:30.740187  6370 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:30.653007  6267 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:30.533113  5943 rpcz_store.cc:276] 0709 12:23:28.974614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.326272 (+351658us) service_pool.cc:224] Handling call
0709 12:23:29.381303 (+ 55031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.012089 (+630786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.495338 (+483249us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b540000b754. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:30.495649 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":898191744}
I20260709 12:23:30.700812  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:30.701469  5945 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:30.702690  6136 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:30.703833  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 56096.2us
I20260709 12:23:30.704137  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:30.680820  6381 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:30.721633  5948 pending_rounds.cc:85] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 2
I20260709 12:23:30.760298  5948 pending_rounds.cc:107] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:23:30.760769  5948 raft_consensus.cc:2889] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:30.727939  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 68428.9us
I20260709 12:23:30.731377  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:30.731956  6068 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.724781  5939 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:30.763513  5221 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:30.746455  5115 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:30.741515  6370 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:30.741084  6380 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.742797  5224 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:30.742446  6267 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.744910  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:30.744861  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.738056  6234 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:30.765512  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 192935us
I20260709 12:23:30.769445  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:30.810456  5946 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:30.744537  6398 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:30.815429  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:30.828531  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.830030  5940 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
W20260709 12:23:30.813832  6116 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.062s	user 0.033s	sys 0.067s
W20260709 12:23:30.837882  6116 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.277s	user 0.034s	sys 0.068s
I20260709 12:23:30.840451  5900 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:30.764169  6294 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:30.795846  6202 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:30.763067  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:30.852062  5041 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:23:30.857055  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 119249us
I20260709 12:23:30.848356  6136 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:30.864423  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.865743  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.848690  6202 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.765861  6068 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:30.886760  5948 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:30.886943  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:30.890985  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.892170  5221 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:30.895154  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.862007  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.887135  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:30.866261  6172 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:30.900174  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:30.876276  6202 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:30.858840  6123 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:23:30.907804  5218 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:30.908156  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:30.908910  5145 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:30.853307  6368 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:23:30.910310  6368 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:30.910688  6368 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:30.911811  6368 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:30.913654  6234 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:30.886843  6306 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.914994  6306 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:30.915450  6368 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:30.897738  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:30.889796  6380 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:30.906247  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:30.859483  6299 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:23:30.921967  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.905520  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:30.897817  5216 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:30.927610  5149 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:30.927927  5215 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:30.928475  5074 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:30.901814  6172 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.929184  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:23:30.938329  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:30.916288  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:30.930979  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:30.948083  6399 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:30.860293  5939 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:30.948673  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:30.949194  5939 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:30.949594  5146 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:30.931310  6354 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:23:30.955792  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 142433us
I20260709 12:23:30.960306  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:30.962524  6427 consensus_queue.cc:1059] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:30.963711  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 117581us
I20260709 12:23:30.943140  6172 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:30.978272  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:30.979018  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.949195  5141 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:23:30.979166  5114 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:30.935753  6392 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:30.957358  6354 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.007393  6281 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:30.965196  6348 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:30.923328  6398 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:31.013172  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 110103us
I20260709 12:23:31.010973  5041 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:31.023504  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:31.024542  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:31.026134  5219 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:31.000479  6349 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:31.029435  5072 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:31.013104  6354 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:31.009298  6209 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.010188  6381 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:31.011216  6269 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:31.038327  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.044373  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:31.046690  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:31.045590  5898 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:31.054764  5188 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:31.030053  6399 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:31.063930  5219 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:31.026288  6148 consensus_queue.cc:1059] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
W20260709 12:23:31.075103  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 110375us
I20260709 12:23:31.029253  6348 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.078038  5114 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:31.102532  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.034122  6281 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.043643  6389 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:23:31.065382  6263 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:31.057581  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:31.107933  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.070065  6191 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:31.110394  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 55768.5us
I20260709 12:23:31.056877  6349 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:31.060771  6427 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:23:31.074641  5220 raft_consensus.cc:1217] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Deduplicated request from leader. Original: 1.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:23:31.081519  6112 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:31.076906  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:31.056550  6209 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:31.061761  6269 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.108160  6311 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.110256  6263 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:31.114173  6270 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:31.118956  6068 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:31.139928  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:31.147497  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.143625  5898 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:31.149111  6192 consensus_queue.cc:1059] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:31.138795  6263 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:31.126853  6369 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:31.130988  6191 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:31.135565  5220 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:31.144608  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:31.146530  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:31.126365  5216 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:31.152483  6068 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:23:31.156689  6425 log.cc:927] Time spent T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:23:31.159875  6192 raft_consensus.cc:3055] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:31.178639  6192 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:31.179261  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 87) took 2256 ms. Trace:
I20260709 12:23:31.179639  5148 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:31.179759  6192 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:31.180902  5141 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:31.179692  5218 rpcz_store.cc:276] 0709 12:23:28.923218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.930291 (+  7073us) service_pool.cc:224] Handling call
0709 12:23:28.930639 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.908406 (+1977767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.178968 (+270562us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b5400410354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:31.179182 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4400417152}
I20260709 12:23:31.183861  6392 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:31.142755  6436 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:31.176949  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:31.185557  5069 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:31.186700  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:31.188086  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:31.181238  6192 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:31.139043  6311 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.170569  6369 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:31.195539  6311 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:31.195719  6369 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:31.199013  6437 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:31.199856  6437 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:31.162580  6429 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:23:31.210666  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.166280  6433 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:31.152189  6306 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:31.243155  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 98679.4us
I20260709 12:23:31.248677  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:31.163581  6191 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:23:31.258395  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 320149us
I20260709 12:23:31.257926  6191 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:31.152875  6399 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.243188  6434 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:31.288033  5069 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:31.283568  5149 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:31.276499  6436 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:31.261181  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.301632  5067 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:31.290582  6436 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:31.302878  5224 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35624 (request call id 705) took 2773 ms (client timeout 1985 ms). Trace:
I20260709 12:23:31.275647  6433 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:31.303397  5224 rpcz_store.cc:269] 0709 12:23:28.529650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.529801 (+   151us) service_pool.cc:224] Handling call
0709 12:23:29.050162 (+520361us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:31.302636 (+2252474us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54003caad4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:31.302784 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3835729920}
I20260709 12:23:31.200336  6437 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:31.305856  5146 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:31.307368  6437 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:31.292088  6440 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:23:31.292486  6430 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.084s	user 0.001s	sys 0.001s
I20260709 12:23:31.293025  6349 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:31.294947  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 46418.3us
W20260709 12:23:31.312747  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 173554us
W20260709 12:23:31.298002  6424 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.191s	user 0.000s	sys 0.002s
W20260709 12:23:31.298462  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:31.314539  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:31.315739  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:31.315900  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 238009us
W20260709 12:23:30.951911  6301 consensus_queue.cc:1175] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" responder_term: 3 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: [2.2-2.2]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:31.300875  6136 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:31.302995  6436 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:31.310791  5223 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
W20260709 12:23:31.286715  6431 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:23:31.299839  6269 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:31.308956  6172 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:31.293470  6432 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:23:31.287835  6434 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:30.831085  6120 consensus_queue.cc:1175] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9847a7797ddc4df689a1ab0911af8785" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:31.326406  6202 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:31.339970  5898 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:31.340695  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:31.333015  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:31.312680  6440 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:31.350580  6433 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:31.337750  6281 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:31.369138  5114 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:31.381114  6136 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.386648  6136 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.375847  6262 raft_consensus.cc:3055] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:31.387709  6202 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:31.387874  6262 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:31.388597  6202 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:31.389102  6262 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:31.378713  6261 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.389029  6202 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:31.328753  6448 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:31.390744  6202 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:31.390532  6261 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:31.393024  6261 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:31.338430  6209 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:31.400682  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 60817.1us
I20260709 12:23:31.401795  5898 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:31.399112  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 44506.9us
I20260709 12:23:31.388932  6136 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:31.352705  6434 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:31.367461  6378 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:31.368160  6433 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:23:31.403666  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 394892us
I20260709 12:23:31.363943  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.357052  6440 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:31.403033  6370 consensus_meta.cc:287] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.607s	user 0.013s	sys 0.001s
I20260709 12:23:31.415616  6273 raft_consensus.cc:3055] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:31.416468  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:31.417313  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:31.367830  6369 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.420897  6136 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.493901  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:31.471206  6267 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.466811  6299 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:31.473835  5070 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:31.475020  6281 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.475535  6435 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:23:31.476127  6456 log.cc:927] Time spent T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:23:31.369510  6263 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:31.477222  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 108204us
W20260709 12:23:31.478223  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 58849.2us
I20260709 12:23:31.478976  6446 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:31.476151  6191 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.498463  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:31.494293  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:31.501055  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:31.409137  6243 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:23:31.492228  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.510238  5947 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
W20260709 12:23:31.515079  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:31.517445  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 176839us
I20260709 12:23:31.534166  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:31.535229  6461 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:31.537230  6399 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:31.421002  6349 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:31.470003  6273 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:23:31.522653  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.493897  6445 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.509771  6267 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:31.406581  6354 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.569056  6354 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:31.519174  6448 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.513386  6369 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.545303  6461 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:31.516891  6433 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:31.519335  6435 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:31.514549  5217 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:31.545909  6281 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.601792  6433 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.604462  6433 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:31.604570  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 135289us
I20260709 12:23:31.566181  5147 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:31.605525  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:31.606163  6263 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:31.607409  6191 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:31.607230  6273 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.609771  6267 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:31.581287  6445 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:31.571029  6294 consensus_meta.cc:287] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.721s	user 0.013s	sys 0.000s
I20260709 12:23:31.571296  6123 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:31.613130  6202 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:31.613791  6445 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:31.573994  6262 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:31.615525  6202 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:31.615847  6369 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:31.583911  6436 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.542912  5945 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:31.529615  5946 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:31.532200  6243 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:31.619004  6436 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:31.620404  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.513742  6136 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:31.593909  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.585041  6448 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:31.624046  6448 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:31.601254  6435 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:31.624408  6136 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:31.620347  6243 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.605113  6281 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:31.635180  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:31.649410  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 113225us
I20260709 12:23:31.656491  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:31.658368  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:31.654253  6123 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.630455  6243 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:31.669427  5900 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:31.670068  5898 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:31.626453  6461 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:31.627250  6459 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:31.666520  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:31.677006  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:31.667750  6123 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:31.679797  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 74364.3us
I20260709 12:23:31.678450  5222 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:31.680682  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:31.685160  6349 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:31.685702  6349 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:31.685590  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:31.686240  6349 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:31.689203  6390 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:31.689777  6390 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:31.689868  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:31.690793  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:31.691639  5215 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:31.692411  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.693842  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:31.694494  5221 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:31.695380  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.696283  5069 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:31.700060  5188 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:31.685179  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:31.696702  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:31.690253  6390 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:31.662547  6369 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:31.695713  5945 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:31.710803  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.706421  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:31.715049  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:31.712993  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 43649us
I20260709 12:23:31.716878  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:31.697381  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.706954  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:23:31.676489  6450 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.189s	user 0.000s	sys 0.004s
I20260709 12:23:31.719980  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:31.720778  5069 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:31.721614  5074 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:31.721022  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:31.690016  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:31.675269  6445 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:31.719539  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:31.725739  5071 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:31.677490  6172 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:31.733975  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 53371.4us
I20260709 12:23:31.735436  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:31.725984  5223 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:31.742556  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:31.710212  6392 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:31.697512  6434 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.718076  6234 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:31.737771  6236 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:31.750078  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:31.751416  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:31.752666  6172 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:31.746408  6202 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:31.755440  6465 consensus_queue.cc:1059] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:31.755990  6202 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.756450  6202 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:31.756842  6465 raft_consensus.cc:3055] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:31.736480  6440 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.757704  6080 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 12:23:31.759776  6440 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:31.760180  6464 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:23:31.760756  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.761605  6448 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:31.762073  6448 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.762439  6448 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:31.762603  6434 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:31.765254  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:31.757282  6465 raft_consensus.cc:740] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:31.766985  5900 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:31.764636  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:31.683758  6459 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.737689  5068 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:31.746706  6301 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:31.785115  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:31.749794  6467 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:23:31.771288  5940 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:31.788851  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.791121  5069 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:31.791990  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:31.792804  5073 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:31.768764  5148 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:31.794922  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:31.799943  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:31.764004  6236 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.809718  6468 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:31.796685  6446 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:23:31.810722  6466 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:23:31.785477  6392 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:31.811261  6112 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:23:31.812156  6445 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:31.812844  6445 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:23:31.813372  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.813928  6445 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.783599  6306 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:31.814565  6236 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:31.814993  6306 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.815405  6306 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:31.817812  6434 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:31.818250  6434 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:31.818544  6440 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.818681  6434 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:31.818569  5898 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:31.819111  6440 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.820236  6443 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:31.820252  5898 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:31.820824  6443 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.821213  6443 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:31.821580  6368 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:31.822431  6209 raft_consensus.cc:3055] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:31.822842  6209 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:31.823799  6209 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:31.825631  6468 raft_consensus.cc:3055] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:31.826081  6468 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:31.826929  6468 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:31.828608  6468 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:31.797032  6191 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:31.841872  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 93822.8us
I20260709 12:23:31.783267  6433 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:31.829576  6191 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.783334  6234 raft_consensus.cc:697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:31.846635  5898 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:31.847560  6299 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:31.848021  6299 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.848362  6299 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:31.790813  6172 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:23:31.852262  6192 consensus_meta.cc:287] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.006s
I20260709 12:23:31.841338  6267 raft_consensus.cc:2764] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:23:31.844641  6433 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.846096  6123 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:31.853199  6433 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:31.847143  6234 consensus_queue.cc:237] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:31.843640  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 43817.1us
I20260709 12:23:31.854111  5041 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:31.843364  5073 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:31.860621  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:31.853663  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.869118  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.874740  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.876166  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.879312  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:31.857717  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 75151.4us
I20260709 12:23:31.880505  5073 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:31.881254  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:31.883553  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:31.891534  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:31.861598  6392 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:31.858662  6123 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:31.898674  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:31.901448  6440 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:31.902984  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48998us
I20260709 12:23:31.882063  6267 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:31.895936  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:31.902884  5142 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:31.897675  5145 raft_consensus.cc:2376] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3. Re-sending same reply.
I20260709 12:23:31.907496  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:31.910241  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.912137  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.912577  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:31.914525  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:31.916162  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.918838  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:31.920066  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:31.898752  6378 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:31.904270  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:31.922005  5069 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:31.911815  5070 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:31.905104  6267 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.932777  6191 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:31.951164  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:31.967568  5076 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:31.970010  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:31.970759  5142 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:31.937824  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.939623  5141 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:31.916304  6369 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:31.943352  6476 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.905848  6123 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:31.938114  6136 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:31.943871  6435 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:31.945313  6378 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.986462  6299 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.947178  6349 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:31.987730  6435 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:31.989473  6349 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:31.987756  6378 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:31.974954  5071 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:31.992635  5074 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:31.992303  6448 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.992108  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:31.977445  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:31.994678  6474 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:23:31.995270  6369 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:31.979593  5073 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:31.996934  6123 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:31.998476  5189 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:31.999168  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:32.000622  6123 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:32.001010  6192 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:32.001444  6192 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.001829  6192 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.004369  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:32.004760  6438 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:32.005429  6123 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:32.005470  6438 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:32.007171  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:32.007504  6299 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:32.008086  6438 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:32.008028  6123 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:31.921911  6459 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:31.994745  6448 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:32.010408  6123 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:31.980345  6476 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:31.893770  6465 consensus_queue.cc:260] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:31.981112  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:32.026188  6465 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:31.976425  6392 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:31.983202  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:31.948508  6267 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.031035  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:32.031711  5221 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.014822  5041 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:32.024029  5074 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:32.034040  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:32.029613  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:32.024149  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:32.030201  5075 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:23:32.034994  5222 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:32.024638  5899 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:32.040170  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:32.040936  5071 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:32.026727  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:32.040789  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:32.046900  5215 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:32.048177  6438 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:32.028934  6392 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:32.054032  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:32.059545  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:32.050514  5041 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:32.053743  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:32.060331  6392 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.059584  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:32.025311  6306 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:32.045336  6207 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:32.050876  5141 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:32.038487  5068 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:32.073865  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:32.074703  5221 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:32.074721  5899 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:32.057090  6299 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:32.048866  6476 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:32.050561  6080 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:32.081187  5899 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:32.032680  6236 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:32.082676  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:32.085813  6438 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:32.065969  6273 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:32.066300  6123 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:32.090206  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:32.070670  6207 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.080718  6299 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.101663  5147 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:32.102873  5946 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:32.092594  6306 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:32.093052  6373 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:32.056346  6202 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:32.132972  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:32.134526  6202 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:32.135552  5147 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:23:32.137751  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:32.096367  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:32.090209  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:32.099901  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:32.100292  6477 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:23:32.093307  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:32.098511  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:32.159126  5948 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:32.091264  6438 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.106434  6348 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:32.107168  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:32.095291  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:32.109179  6236 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:32.109723  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:32.118172  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:32.118939  6192 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:32.121091  5141 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:32.121663  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:32.159396  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:32.123456  5900 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:32.125568  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:32.181468  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:32.127089  5114 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:32.127180  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:32.097921  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:32.129428  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.128189  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:32.156737  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:32.160312  5946 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:23:32.128391  6299 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:32.185758  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 87931.9us
I20260709 12:23:32.186399  5942 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 3: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:32.187213  6444 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:32.187557  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:32.187822  6444 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:32.188325  5942 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:32.163853  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:32.190496  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:32.190783  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:32.191383  5072 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.191828  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:32.192442  5070 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:23:32.166193  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:32.194247  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:32.195097  5940 raft_consensus.cc:2368] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:32.196151  5940 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:32.170064  6434 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:32.185899  6348 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:32.198629  6348 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:32.199345  6434 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:32.200209  5145 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:32.198628  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:32.103896  6207 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.098558  6461 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:32.122267  5071 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:32.163331  6438 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.175873  5074 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.236677  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:32.171242  5943 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:32.178489  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:32.238399  6443 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:32.239403  6446 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:23:32.182199  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 58821.7us
I20260709 12:23:32.240082  5947 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:32.175300  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:32.241207  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:32.185269  6448 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:32.243655  6439 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:23:32.185940  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 58962.7us
I20260709 12:23:32.244251  6439 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:32.244696  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:32.188838  6444 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:32.245095  6349 consensus_queue.cc:1059] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:32.245239  6439 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:32.246174  5940 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:32.246436  6373 raft_consensus.cc:3055] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:23:32.246965  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.248452  5114 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:32.246870  6373 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:32.250146  6465 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:32.251020  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:32.250981  6373 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:32.251608  6209 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:32.251911  6465 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:32.238483  5948 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:32.252224  6373 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.242040  5944 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:32.241174  6443 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:32.204746  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48108.3us
I20260709 12:23:32.205528  6459 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:23:32.208020  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 76833.2us
I20260709 12:23:32.208686  6390 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:32.209386  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:32.281672  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:32.216619  6482 log.cc:927] Time spent T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:23:32.218027  6398 raft_consensus.cc:3055] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:32.224279  5221 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:32.283409  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:32.225584  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.284039  5223 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:32.285369  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:32.233497  6348 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:32.265836  5074 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:32.252483  6448 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:32.273031  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:32.140959  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:32.117393  6433 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:32.279649  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:32.289603  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 200225us
I20260709 12:23:32.271123  5115 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:32.295317  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:32.301471  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:32.302398  5141 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:32.302619  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:32.304153  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:32.305361  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:32.306915  6444 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:23:32.312150  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 62428.6us
I20260709 12:23:32.313416  5074 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:23:32.314675  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 173803us
I20260709 12:23:32.311448  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:32.321221  5221 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
W20260709 12:23:32.344072  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.377522  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:32.379719  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:32.440320  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 60914.4us
W20260709 12:23:32.441403  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 61485.4us
I20260709 12:23:32.281116  6191 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:32.380409  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 60526.7us
I20260709 12:23:32.384821  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:32.444579  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 65418.3us
I20260709 12:23:32.450776  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:32.451687  5221 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:32.452734  5150 raft_consensus.cc:2368] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4.
I20260709 12:23:32.291527  6398 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:32.451277  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:32.456655  5224 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
W20260709 12:23:32.440070  5041 leader_election.cc:336] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Timed out: RequestConsensusVote RPC to 127.1.123.68:43449 timed out after 1.985s (SENT)
I20260709 12:23:32.379772  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:32.292505  6465 consensus_queue.cc:1059] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.161s
I20260709 12:23:32.372807  6301 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:32.267838  6392 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:32.373106  6380 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:32.310333  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:32.310693  6468 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:32.479360  5114 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:32.374524  6306 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:23:32.457790  6482 log.cc:927] Time spent T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:23:32.451783  6435 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:32.500204  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:32.462925  6440 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:32.502207  5142 raft_consensus.cc:2393] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:32.449715  6444 raft_consensus.cc:697] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:32.509071  5071 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:32.481015  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:32.315377  6399 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.177s
I20260709 12:23:32.507328  5073 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:32.482401  6236 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:32.457554  6369 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:23:32.486821  6380 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.493770  6468 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:32.517838  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 77846.8us
I20260709 12:23:32.456854  6123 consensus_queue.cc:1059] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:23:32.457142  6449 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:23:32.482667  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:32.462421  6191 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:32.551342  6486 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:23:32.498572  5068 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:32.552559  6468 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.553133  6191 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:32.511200  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:32.555164  6123 raft_consensus.cc:3055] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:32.555624  6123 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:32.556596  6123 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:32.557473  6433 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:32.498903  6434 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:32.558142  6434 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.558208  6433 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:32.558627  6434 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.558784  6433 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:32.560927  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:32.561795  5142 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:32.562784  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:32.563290  5142 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:32.564072  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:32.566215  6123 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:32.502898  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 205661us
I20260709 12:23:32.505299  6112 consensus_queue.cc:1059] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:23:32.541097  6273 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.583694  5223 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:32.550711  6461 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:32.569195  5898 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:32.582545  5072 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:32.550160  6369 raft_consensus.cc:3055] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:32.531608  6380 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:32.600775  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.587538  6399 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:32.592319  6369 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:32.611819  6433 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:32.611749  6369 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:32.612737  6433 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:32.525841  6440 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:32.613202  6369 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:32.676775  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 297054us
I20260709 12:23:32.685647  5115 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:32.664830  5143 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
W20260709 12:23:32.652771  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.585044  6236 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:32.689460  6236 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:32.696352  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:32.594251  6348 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:32.595605  6234 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.702276  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:32.724810  5221 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:32.625154  6390 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:32.711073  5145 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:32.718508  5900 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:32.717048  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:32.736130  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:32.644431  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:32.628517  5070 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:32.723634  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:32.746029  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:32.749140  5145 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
W20260709 12:23:32.750036  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 305763us
I20260709 12:23:32.692651  5141 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:32.594694  6273 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:32.677352  6438 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:32.613436  6433 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:32.753397  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 250619us
I20260709 12:23:32.755213  5900 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:32.756748  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:32.762197  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:32.763098  5041 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:32.621133  6444 consensus_queue.cc:237] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:32.771188  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:32.778349  5041 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:32.756239  5944 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:32.757647  6494 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:32.768069  6172 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:32.742388  6234 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:32.770247  6348 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:32.783000  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:32.773828  6493 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:32.772971  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:32.824115  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 62007.9us
I20260709 12:23:32.772500  6461 consensus_queue.cc:1059] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:23:32.774567  6465 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:23:32.790253  5215 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:32.779922  6492 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:32.789212  6294 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:23:32.807400  6267 consensus_meta.cc:287] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.775s	user 0.007s	sys 0.007s
I20260709 12:23:32.814801  6497 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:32.817832  5943 raft_consensus.cc:2368] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:32.842907  5216 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.846755  5222 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:32.846861  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:32.808059  6348 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:32.815698  6392 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:32.842384  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 64177.4us
I20260709 12:23:32.808229  6494 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:32.855257  6492 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.875485  6273 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:32.876927  6234 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:32.858409  6294 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.853143  6389 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:23:32.867213  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:32.904452  5146 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:32.904888  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:32.905426  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:32.899308  6299 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.003s
I20260709 12:23:32.866976  6348 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.859477  6498 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:32.841900  6495 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:32.909041  5146 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:32.911332  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 315) took 1414 ms. Trace:
I20260709 12:23:32.913527  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 669) took 1031 ms. Trace:
I20260709 12:23:32.915189  6494 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:32.868005  6499 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:23:32.917250  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.863075  6267 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:32.927342  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:32.917068  6234 raft_consensus.cc:3055] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:32.901242  6440 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:32.906834  6294 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:32.961295  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 93827.5us
I20260709 12:23:32.895417  6492 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.928757  5143 rpcz_store.cc:276] 0709 12:23:31.882105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.908994 (+ 26889us) service_pool.cc:224] Handling call
0709 12:23:32.913327 (+1004333us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400751cd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:32.913448 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775359872}
I20260709 12:23:32.922683  6301 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:32.985281  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 58058.6us
I20260709 12:23:32.923833  5944 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.992395  5041 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:32.990921  4898 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43226 (request call id 127) took 6539 ms. Trace:
I20260709 12:23:32.996376  5149 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:32.998018  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:32.988173  6243 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:32.984879  6234 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:32.910075  6504 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:32.927948  6299 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:33.007100  5190 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.621s (SENT) [suppressed 33 similar messages]
W20260709 12:23:33.003321  6209 consensus_meta.cc:287] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.751s	user 0.014s	sys 0.002s
I20260709 12:23:32.997346  6438 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:32.908849  6080 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 12:23:32.963526  6506 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:33.003751  6433 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:33.007787  6243 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:33.005837  6497 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:33.026603  5041 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:33.028926  5941 raft_consensus.cc:2368] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:32.928522  5939 rpcz_store.cc:276] 0709 12:23:31.497156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.513617 (+ 16461us) service_pool.cc:224] Handling call
0709 12:23:32.108023 (+594406us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:32.904712 (+796689us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416e9c 00007fd8a5416a5b 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:32.911230 (+  6518us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33059584}
I20260709 12:23:33.010828  6504 raft_consensus.cc:697] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:33.031857  5041 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:33.029835  6262 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.023846  6509 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:33.042354  6267 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:33.046070  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 629) took 1074 ms. Trace:
I20260709 12:23:33.046756  5073 rpcz_store.cc:276] 0709 12:23:31.971789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.997636 (+ 25847us) service_pool.cc:224] Handling call
0709 12:23:31.998045 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:32.519928 (+521883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.045663 (+525735us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b5400048654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:33.045970 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1170622336}
I20260709 12:23:33.022027  6506 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:32.993889  4898 rpcz_store.cc:276] 0709 12:23:26.389290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.389435 (+   145us) service_pool.cc:224] Handling call
0709 12:23:32.928988 (+6539553us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:31.049486 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:31.240776 (+191290us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:31.256398 (+ 15622us) write_op.cc:435] Acquired schema lock
0709 12:23:31.263775 (+  7377us) tablet.cc:662] Decoding operations
0709 12:23:31.349255 (+ 85480us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:31.349363 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:31.349393 (+    30us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:23:31.349859 (+   466us) tablet.cc:701] Row locks acquired
0709 12:23:31.349914 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:23:31.350107 (+   193us) write_op.cc:270] Start()
0709 12:23:31.350266 (+   159us) write_op.cc:276] Timestamp: P: 1783599811350038 usec, L: 0
0709 12:23:31.350305 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:23:31.351131 (+   826us) log.cc:844] Serialized 1307 byte log entry
0709 12:23:31.849032 (+497901us) op_driver.cc:464] REPLICATION: finished
0709 12:23:31.929095 (+ 80063us) write_op.cc:301] APPLY: starting
0709 12:23:31.929231 (+   136us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:31.929424 (+   193us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:31.929632 (+   208us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:31.931401 (+  1769us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:31.931704 (+   303us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:23:31.931765 (+    61us) write_op.cc:312] APPLY: finished
0709 12:23:31.932025 (+   260us) log.cc:844] Serialized 34 byte log entry
0709 12:23:31.932259 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:31.932379 (+   120us) write_op.cc:454] Released schema lock
0709 12:23:31.932453 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:31.932994 (+   541us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:31.933031 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":79636,"apply.run_cpu_time_us":3866,"apply.run_wall_time_us":3866,"num_ops":3,"prepare.queue_time_us":310125,"prepare.run_cpu_time_us":7790,"prepare.run_wall_time_us":488469,"raft.queue_time_us":181258,"raft.run_cpu_time_us":19682,"raft.run_wall_time_us":108489,"replication_time_us":483527,"thread_start_us":427555,"threads_started":4,"wal-append.queue_time_us":61882,"wal-append.run_cpu_time_us":33805,"wal-append.run_wall_time_us":1520926}]]}
W20260709 12:23:33.146561  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 139472us
I20260709 12:23:33.033578  6507 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:33.051224  6375 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.052135  6399 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:33.050925  6262 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.052395  6506 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:33.216353  5900 leader_election.cc:336] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.820s (SENT)
I20260709 12:23:33.224895  5900 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:33.033402  6234 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:33.194130  6448 consensus_meta.cc:287] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.897s	user 0.014s	sys 0.000s
I20260709 12:23:33.190972  6349 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.191325  6438 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:33.224164  6207 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.938s	user 0.009s	sys 0.005s
I20260709 12:23:33.197732  6273 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.208942  6449 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:33.254210  6504 consensus_queue.cc:237] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:33.194064  6507 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:33.241326  6468 consensus_meta.cc:287] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.688s	user 0.009s	sys 0.006s
W20260709 12:23:33.240844  6373 consensus_meta.cc:287] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.971s	user 0.012s	sys 0.003s
I20260709 12:23:33.251041  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:33.250471  6299 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:33.226528  6243 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:33.351513  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 259) took 1266 ms. Trace:
W20260709 12:23:33.302640  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.223884  6375 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:33.272667  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 56396.9us
I20260709 12:23:33.251173  6448 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:33.270968  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 239199us
I20260709 12:23:33.268651  6301 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:33.272552  6449 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:33.353924  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:33.278847  6234 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:33.352296  5947 rpcz_store.cc:276] 0709 12:23:32.084585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.086636 (+  2051us) service_pool.cc:224] Handling call
0709 12:23:33.351235 (+1264599us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b5400007154. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:33.351427 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230996480}
I20260709 12:23:33.262485  6349 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.264588  6438 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:33.368604  6349 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:33.364408  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 517592us
I20260709 12:23:33.318284  6507 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:33.273015  6273 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:33.332283  6509 raft_consensus.cc:697] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:33.360731  5217 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:33.366362  5939 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:33.366707  6375 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.369282  6438 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:33.361260  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:33.374673  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:33.226572  6497 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.264508  6207 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:33.376956  5946 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:23:33.360248  6449 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:33.345942  6468 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.388448  5041 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:33.390172  5114 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:33.397383  5041 leader_election.cc:400] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:33.399180  5041 leader_election.cc:403] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:33.400975  5114 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:33.407063  5114 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:33.410763  6262 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.437037  5147 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:23:33.407922  6380 consensus_meta.cc:287] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.751s	user 0.016s	sys 0.003s
I20260709 12:23:33.441735  5041 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:33.442960  5114 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:33.393428  6516 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:33.436831  6399 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:33.408420  6512 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:33.406414  6438 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.405941  6448 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:23:33.442478  6123 consensus_meta.cc:287] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.876s	user 0.008s	sys 0.011s
I20260709 12:23:33.436412  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 256) took 1131 ms. Trace:
I20260709 12:23:33.438102  6080 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:33.395669  6517 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:33.405875  6515 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:23:33.460554  6434 consensus_meta.cc:287] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.901s	user 0.014s	sys 0.003s
I20260709 12:23:33.461771  6434 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:33.445084  6508 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:23:33.441994  6498 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:23:33.464814  6434 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:33.465195  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35624 (request call id 723) took 1380 ms. Trace:
I20260709 12:23:33.445389  6433 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:23:33.465739  5217 rpcz_store.cc:276] 0709 12:23:32.084681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.087620 (+  2939us) service_pool.cc:224] Handling call
0709 12:23:33.464978 (+1377358us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b5400446e54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:33.465119 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917980416}
I20260709 12:23:33.466303  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:33.467089  5940 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:33.453366  5220 raft_consensus.cc:2368] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:33.460105  6509 consensus_queue.cc:237] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:33.451050  6465 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:33.445677  6267 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:23:33.457958  6392 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:33.446516  6380 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:33.465272  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 328) took 1217 ms. Trace:
W20260709 12:23:33.480930  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 44467.9us
I20260709 12:23:33.456566  6438 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:33.487425  6438 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:33.491743  6207 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:33.455227  5939 rpcz_store.cc:276] 0709 12:23:32.305448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.365796 (+1060348us) service_pool.cc:224] Handling call
0709 12:23:33.366155 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.390644 (+ 24489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.424787 (+ 34143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:33.496378  5939 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:33.498482  6498 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.499279  6498 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.482470  5945 rpcz_store.cc:276] 0709 12:23:32.248208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.259082 (+ 10874us) service_pool.cc:224] Handling call
0709 12:23:33.464343 (+1205261us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:33.464629 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.464861 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:33.464891 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:33.464953 (+    62us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:33.464989 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.465064 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:33.465204 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34372352}
I20260709 12:23:33.501969  6498 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.522006  6273 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:33.523802  6440 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:33.530230  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 88743.6us
I20260709 12:23:33.497411  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 258) took 1120 ms. Trace:
I20260709 12:23:33.524457  6440 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.533656  5939 rpcz_store.cc:276] 0709 12:23:32.377136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.495235 (+1118099us) service_pool.cc:224] Handling call
0709 12:23:33.495808 (+   573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.496963 (+  1155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.497285 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:33.540122  6294 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.547s	user 0.011s	sys 0.003s
I20260709 12:23:33.562422  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.546662  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.565155  5946 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:33.574012  5939 raft_consensus.cc:2393] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:23:33.576272  5216 consensus_meta.cc:287] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.689s	user 0.013s	sys 0.000s
I20260709 12:23:33.577330  5216 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:33.582573  6521 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.583314  6380 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:33.583459  6521 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:33.580343  5945 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:33.593277  6349 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:33.593644  6376 raft_consensus.cc:3055] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:33.593959  6349 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.594233  6376 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:33.571470  6506 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.595381  6376 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.596280  6349 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:33.596477  6376 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:33.597663  6509 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.594259  6440 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.586980  6438 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.605821  6438 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:33.608232  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 332) took 1341 ms. Trace:
I20260709 12:23:33.608462  6438 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:33.610422  5944 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.610s	user 0.013s	sys 0.000s
I20260709 12:23:33.611058  5944 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:33.611141  6192 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.568004  6294 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.604408  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35624 (request call id 724) took 1516 ms. Trace:
W20260709 12:23:33.606257  5150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 109) took 1742 ms (client timeout 1820 ms). Trace:
I20260709 12:23:33.588702  6468 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:33.612457  5149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 677) took 1716 ms (client timeout 1621 ms). Trace:
W20260709 12:23:33.620999  5149 rpcz_store.cc:269] 0709 12:23:31.896140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.903509 (+  7369us) service_pool.cc:224] Handling call
0709 12:23:33.612231 (+1708722us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540036f454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:33.612384 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2486107904}
I20260709 12:23:33.614497  5220 rpcz_store.cc:276] 0709 12:23:32.087717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.093735 (+  6018us) service_pool.cc:224] Handling call
0709 12:23:33.604146 (+1510411us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400553a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:33.604320 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2891253120}
I20260709 12:23:33.622527  6468 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.623221  6468 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:33.628829  6492 consensus_meta.cc:287] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.000s
I20260709 12:23:33.629832  6492 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:33.612576  5150 rpcz_store.cc:269] 0709 12:23:31.863717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.863861 (+   144us) service_pool.cc:224] Handling call
0709 12:23:33.605918 (+1742057us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b5400238f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:33.606156 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141431168}
I20260709 12:23:33.618023  6515 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.618800  6294 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.618669  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:33.619589  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:33.620007  6369 consensus_meta.cc:287] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.926s	user 0.015s	sys 0.000s
W20260709 12:23:33.616617  6348 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.655s	user 0.006s	sys 0.006s
I20260709 12:23:33.633605  5072 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:33.633304  5898 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:33.650729  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:33.647923  6498 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.654081  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:33.654342  6498 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:33.654976  5074 raft_consensus.cc:2393] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:33.655227  6517 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:33.648227  6506 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:33.608664  5947 rpcz_store.cc:276] 0709 12:23:32.266775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.365266 (+1098491us) service_pool.cc:224] Handling call
0709 12:23:33.365716 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.370822 (+  5106us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:23:33.379683 (+  8861us) log.cc:844] Serialized 14 byte log entry
0709 12:23:33.458883 (+ 79200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:33.458942 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:33.458976 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.607606 (+148630us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b18000e5420. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a541c2a6 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419
0709 12:23:33.607941 (+   335us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b5400005854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:33.607969 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:33.608147 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9395,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":66,"raft.queue_time_us":2688,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"spinlock_wait_cycles":30672000,"thread_start_us":76159,"threads_started":1,"wal-append.queue_time_us":113612}
I20260709 12:23:33.652293  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:33.658638  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:33.598277  6509 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:33.653731  6301 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:33.663201  5948 raft_consensus.cc:2368] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4.
I20260709 12:23:33.664795  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:33.638509  6392 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:33.648729  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:33.677315  6392 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:33.681636  6521 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.683017  6499 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:33.683544  6499 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:33.685370  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:33.685740  5939 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:23:33.686240  5075 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:33.687390  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.667829  6498 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:33.647833  6515 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:33.651528  5067 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:33.668735  6301 raft_consensus.cc:697] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:33.699815  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:33.700325  6301 consensus_queue.cc:237] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.701929  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:33.702543  5069 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:23:33.670616  5071 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:33.703135  6499 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.704432  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:33.705706  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:33.707182  5071 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:33.707484  6515 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.707710  6499 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.709779  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.710484  5947 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:33.711215  6507 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.651279  6468 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.714462  5072 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:23:33.717903  6491 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.718580  6491 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.719043  6492 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:33.660641  6373 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:33.728498  5221 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:33.730353  5220 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:33.731635  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:33.732085  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:33.732602  5216 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:33.732925  5947 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:33.667935  5073 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:33.662853  6369 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:33.646683  6349 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:23:33.664216  6299 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:23:33.691807  6311 consensus_queue.cc:1059] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:33.739437  6493 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.023s	sys 0.041s
I20260709 12:23:33.749913  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:33.743337  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.761281  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:33.779501  5114 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:33.692857  5076 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:33.693020  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 257) took 1316 ms. Trace:
I20260709 12:23:33.694494  5940 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:33.689692  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:33.698421  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.700830  6236 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:23:33.721266  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.792896  5071 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:23:33.793367  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:33.722968  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:33.688865  6449 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.740499  6438 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.726625  4895 catalog_manager.cc:5697] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 3, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:23:33.801151  5218 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:33.739424  6492 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.802865  4895 catalog_manager.cc:5697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 4, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 4 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:33.651145  6348 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.794955  5223 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:23:33.764748  6493 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.023s	sys 0.046s
I20260709 12:23:33.730371  6491 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:33.797729  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.799425  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:33.742985  6435 raft_consensus.cc:3055] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:33.786314  5075 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:33.788609  5069 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:33.808255  5221 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:33.789393  6306 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.812500  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:33.809366  6510 raft_consensus.cc:3055] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:33.813709  6449 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:33.797248  6236 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:33.814451  6438 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:33.815289  5948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 330) took 1549 ms (client timeout 1978 ms). Trace:
I20260709 12:23:33.783032  6504 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:33.739488  6369 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.792122  6468 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:33.802042  6521 consensus_queue.cc:1059] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 12:23:33.830842  6493 raft_consensus.cc:697] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:33.790405  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.819729  5067 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
W20260709 12:23:33.858011  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.858608  5074 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: [2.2-3.3]
I20260709 12:23:33.836431  6435 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:33.838591  6515 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:33.785707  5943 rpcz_store.cc:276] 0709 12:23:32.376678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.529562 (+152884us) service_pool.cc:224] Handling call
0709 12:23:33.692784 (+1163222us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b5400005d54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:33.692951 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73599360}
I20260709 12:23:33.742899  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.823377  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:33.840536  6510 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:33.842747  6236 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:33.826017  5073 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:33.867074  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.824388  4895 catalog_manager.cc:5697] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:33.870575  4895 catalog_manager.cc:5697] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 3, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 3 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:33.798115  5940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 333) took 1527 ms (client timeout 1905 ms). Trace:
I20260709 12:23:33.825779  6446 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.830037  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.821501  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:33.795904  6509 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.833423  5041 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:33.835602  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.838877  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:33.876675  5222 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
W20260709 12:23:33.876919  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 43623.5us
I20260709 12:23:33.877938  5041 leader_election.cc:400] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:33.878244  5041 leader_election.cc:403] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:23:33.873265  5940 rpcz_store.cc:269] 0709 12:23:32.270950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.465945 (+1194995us) service_pool.cc:224] Handling call
0709 12:23:33.796265 (+330320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:33.880049  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.880659  6438 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:33.881924  5068 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:33.882364  5141 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:23:33.862583  6515 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.882912  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:33.883076  6515 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:33.884222  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:33.884469  6392 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.885464  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:33.886045  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:33.886224  5145 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
W20260709 12:23:33.881201  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 261) took 1796 ms (client timeout 2070 ms). Trace:
I20260709 12:23:33.887462  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:33.887677  5941 rpcz_store.cc:269] 0709 12:23:32.085157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.086918 (+  1761us) service_pool.cc:224] Handling call
0709 12:23:33.880989 (+1794071us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b5400003a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:33.881137 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2026447616}
I20260709 12:23:33.888952  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:33.889183  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:33.889834  6468 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:33.889662  6446 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.890504  6468 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:33.840971  6306 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.891614  6468 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.892501  6446 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:33.892782  6306 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:33.893708  5149 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:33.894402  6435 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.895443  6435 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:33.895691  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:33.896291  6369 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:33.841423  5948 rpcz_store.cc:269] 0709 12:23:32.266135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.550312 (+1284177us) service_pool.cc:224] Handling call
0709 12:23:33.815037 (+264725us) spinlock_profiling.cc:243] Waited 96.7 ms on lock 0x7b54001536d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:33.815213 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222364928}
I20260709 12:23:33.897524  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.898195  5948 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:33.825572  6516 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.847111  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.898931  6516 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.818172  6492 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.880725  5940 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:33.904934  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.842545  6528 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:33.905354  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.905712  6492 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:33.905881  6528 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:33.906203  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:33.906828  5224 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:23:33.819641  6398 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.906911  6492 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.907788  6398 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:33.851904  6504 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.908643  6510 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:33.909776  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:33.910506  5147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 642) took 1606 ms (client timeout 1700 ms). Trace:
I20260709 12:23:33.860625  6498 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:33.804749  6080 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:23:33.823378  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:33.862540  5071 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:33.825037  5219 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:23:33.912798  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 89532.7us
I20260709 12:23:33.837767  5942 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:33.865579  6493 consensus_queue.cc:237] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:33.872740  6348 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:33.867874  5189 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:33.920722  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 52944.8us
W20260709 12:23:33.922772  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 94093us
I20260709 12:23:33.872448  4895 catalog_manager.cc:5697] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 3, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:23:33.896350  5939 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:33.925107  5898 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:33.926397  6506 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.926574  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:33.926872  6446 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:33.927351  6497 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:33.927752  6446 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:33.928050  6440 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:33.928303  6517 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:33.927924  6497 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.929276  6517 raft_consensus.cc:3055] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:33.929822  6517 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:33.929129  6446 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.930792  6517 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:33.931497  6497 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:33.932075  6497 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:33.932245  6517 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:33.932585  6497 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.844836  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.904791  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.907006  6528 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:33.859704  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:33.935190  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:33.845790  6507 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:33.919364  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:33.910951  5147 rpcz_store.cc:269] 0709 12:23:32.304075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.305093 (+  1018us) service_pool.cc:224] Handling call
0709 12:23:33.910277 (+1605184us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54004fa954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:33.910436 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2597080448}
I20260709 12:23:33.919240  5947 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:33.935417  5075 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:33.941118  5115 leader_election.cc:400] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:33.925055  4895 catalog_manager.cc:5697] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 3, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:23:33.889007  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.948297  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:33.942297  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:33.918931  6392 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:33.946554  4895 catalog_manager.cc:5697] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:23:33.889854  6509 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:33.910952  6504 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:33.952239  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.951038  5147 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:33.952433  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:33.954363  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:33.955050  5946 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:33.955134  5218 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:33.920141  6459 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:33.958511  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:33.959113  5145 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:33.959137  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:33.958957  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 80044.3us
I20260709 12:23:33.896688  6515 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:33.963232  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:33.939770  6311 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:33.964504  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.962628  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:33.966696  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.967379  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:33.967432  5041 leader_election.cc:400] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:33.968336  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:33.973951  5146 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:33.965343  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:33.975648  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.976287  5147 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:33.976372  5947 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:33.977777  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.979434  5947 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:33.971982  5218 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:33.973445  5041 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:33.977756  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:33.963109  5940 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:33.926975  6506 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:33.986646  5147 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:33.953088  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.995771  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:33.957716  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:33.956672  6440 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:33.948836  5068 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:34.016100  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.017098  5144 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:33.979099  6510 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:34.026536  6503 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:34.027163  6503 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:34.028105  6503 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:34.029249  6503 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:33.944516  6508 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:23:34.000152  5070 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:34.000481  5067 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:34.001865  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:34.007969  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:33.989511  6311 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:23:34.009867  5942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 331) took 1743 ms (client timeout 2177 ms). Trace:
I20260709 12:23:34.011026  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.051653  6533 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:33.975906  6392 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:33.968995  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:34.015290  6506 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:34.015167  6516 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:34.030503  6509 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:34.030994  5041 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:33.992882  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.031862  6440 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:34.041245  5948 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:33.961313  5941 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:34.046903  5071 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:34.050325  5224 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:34.065338  5075 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:34.060319  5898 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:34.067539  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:34.062395  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:34.069765  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.065366  5216 raft_consensus.cc:2393] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 2: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:34.073282  5947 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:34.083082  5948 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:34.103252  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:34.104166  5221 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:34.070499  5150 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:34.056455  5141 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
W20260709 12:23:34.133095  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 102171us
I20260709 12:23:34.067548  6202 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:34.101226  5940 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:34.155735  6202 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.156597  6515 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:34.158249  6202 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:34.053711  5942 rpcz_store.cc:269] 0709 12:23:32.266474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.621296 (+354822us) service_pool.cc:224] Handling call
0709 12:23:33.698236 (+1076940us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:34.009657 (+311421us) spinlock_profiling.cc:243] Waited 42.2 ms on lock 0x7b5400009e54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:34.009796 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167349376}
I20260709 12:23:34.162249  6236 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:34.162992  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.163863  5223 raft_consensus.cc:2368] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3.
I20260709 12:23:34.168360  6535 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:34.169059  6535 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:34.170408  6535 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.171873  6535 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:34.137226  6446 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:34.066565  6459 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.107s
I20260709 12:23:34.173569  6236 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:34.173681  6446 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:34.174799  6459 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:34.175062  6446 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:34.175420  6459 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:34.066797  6440 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:34.177856  5150 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:34.179081  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:34.180449  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:34.180948  6533 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:34.181008  5150 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:34.194355  6236 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:34.196367  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.197104  5147 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:34.199877  6509 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:34.201202  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:34.201062  6509 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.202029  5147 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:34.203236  5147 raft_consensus.cc:1240] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:34.204413  5147 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
W20260709 12:23:34.205504  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 322844us
I20260709 12:23:34.217206  5145 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
W20260709 12:23:34.219772  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:34.220402  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 59690.4us
I20260709 12:23:34.221501  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:34.214111  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:34.223174  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:34.225875  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.229390  5149 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:34.220423  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:34.176649  6459 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.058733  6500 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 12:23:34.073293  6348 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:34.237246  5146 raft_consensus.cc:2376] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2. Re-sending same reply.
W20260709 12:23:34.249151  5076 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 642) took 1230 ms (client timeout 1611 ms). Trace:
I20260709 12:23:34.251066  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:34.219180  5943 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:23:34.257745  5076 rpcz_store.cc:269] 0709 12:23:32.988799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.988958 (+   159us) service_pool.cc:224] Handling call
0709 12:23:34.218954 (+1229996us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b540047d454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:34.219079 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495253760}
I20260709 12:23:34.262247  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:34.263024  6528 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:34.194252  6449 raft_consensus.cc:3055] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:34.265453  5147 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:34.276986  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:34.320950  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 44109.6us
I20260709 12:23:34.322129  5041 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:34.263293  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:34.099630  6311 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:34.280388  6500 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.280968  6534 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:34.281984  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:34.284426  6392 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:34.286957  6192 consensus_queue.cc:1059] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:34.231863  6349 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 12:23:34.290906  5190 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:34.291729  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:34.296041  6530 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.387s	user 0.014s	sys 0.000s
I20260709 12:23:34.260048  6517 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:34.301800  6497 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:34.260730  6533 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.277626  6299 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:23:34.260399  6369 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:23:34.326639  6534 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:23:34.343142  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52323us
I20260709 12:23:34.232784  6398 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:23:34.326660  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 64530.9us
I20260709 12:23:34.319562  6435 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:34.327787  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 64641.2us
I20260709 12:23:34.328456  6202 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:34.261937  6348 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.258836  6409 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:34.336468  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 71463.5us
I20260709 12:23:34.279000  6528 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:34.319487  6449 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:34.452924  6498 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:34.446781  6497 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:34.249706  6521 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.082s
I20260709 12:23:34.326061  6507 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:34.328953  6491 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:23:34.368978  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:34.369582  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 241025us
I20260709 12:23:34.444056  5142 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:34.329769  6500 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:34.337581  5939 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:34.339715  5216 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:34.342666  6533 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.341588  6301 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 12:23:34.446094  5190 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:34.338301  6517 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:34.373567  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.450558  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:34.510727  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 60250.3us
W20260709 12:23:34.510738  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 64776.2us
I20260709 12:23:34.511535  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:34.460343  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.514194  5146 raft_consensus.cc:2393] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:34.461314  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.515362  5149 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:34.448393  6306 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:34.516372  6306 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.516901  6306 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:34.477104  6507 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:34.502707  6547 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:34.518239  6507 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:34.518415  6547 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:34.518931  6236 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:34.504765  6497 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:34.519431  6547 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:34.520156  5221 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:34.520924  6547 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:34.503456  6499 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:34.465065  5188 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:34.465232  6409 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:34.521554  6499 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:34.326162  6515 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:34.522505  6515 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:34.522799  6269 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:34.522912  6515 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:34.523566  6269 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:34.523844  6409 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:34.524159  6499 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:34.506857  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.524627  6269 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:34.525198  5148 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:34.456754  6348 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:34.497761  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.453684  6435 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.527274  6446 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.467420  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:34.480731  5041 leader_election.cc:400] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3.
I20260709 12:23:34.531592  5041 leader_election.cc:403] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:23:34.532692  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 52137.9us
I20260709 12:23:34.533849  5900 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:34.463725  6528 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:34.521950  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 57002.6us
I20260709 12:23:34.545519  5188 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:34.451371  6438 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.338027  5150 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:34.549722  5072 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:34.530526  6517 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:34.519441  6236 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.552268  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:34.541453  5900 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:34.527339  5141 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:34.563814  5146 raft_consensus.cc:1275] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:34.518265  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:34.569182  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.571010  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:34.570767  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:34.573339  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:34.534101  6398 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:34.503676  6534 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:34.558845  6438 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.581857  6476 raft_consensus.cc:3055] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:34.582038  6534 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:34.582432  6476 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:34.583310  6438 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:34.540836  6533 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:34.583443  6476 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:34.554829  6535 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:34.585691  6535 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:34.588145  6535 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:34.566803  6449 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.570967  6492 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:34.544575  6521 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:23:34.581055  6398 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:34.535105  6390 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:34.585080  6476 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:34.542492  6491 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:34.595161  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:34.603271  5149 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:34.578542  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:34.560276  6236 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:34.592654  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:34.537567  6446 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:34.558605  6499 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:34.599087  5145 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:34.604586  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.565150  6409 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.613206  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:34.616772  5946 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:34.617508  5146 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:34.616819  5149 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:34.558328  6509 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:23:34.625677  5042 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:34.602121  6398 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:34.602517  6390 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.568609  6440 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:34.629468  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 118014us
I20260709 12:23:34.614894  6491 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:34.635687  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:34.625152  6409 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:34.632359  6446 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.637553  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:34.630504  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.637410  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:34.639518  6192 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:34.640691  5042 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:34.641654  6446 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.644770  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:34.646647  6446 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:34.647179  6510 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:34.647212  6440 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:34.648226  5144 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:34.648627  6192 consensus_queue.cc:1059] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:34.650554  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:34.650766  6192 raft_consensus.cc:3055] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:34.651259  6192 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:34.636277  6390 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:34.655459  6544 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:23:34.658195  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:34.660472  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.660696  6440 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:34.661141  6440 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.661523  6440 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:34.662376  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.664456  6435 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:34.665717  6541 log.cc:927] Time spent T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 12:23:34.668462  6539 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:23:34.669865  6192 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:34.670833  6440 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:34.671450  6192 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:34.673220  6440 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:34.674998  5940 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:34.676924  6468 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:34.677114  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.629853  6499 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:34.632699  6299 consensus_queue.cc:1048] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:23:34.653774  6500 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:34.679010  6521 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3.
I20260709 12:23:34.640549  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.710053  5224 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:34.638796  6491 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:34.722422  6499 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:34.657198  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.780082  5141 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:34.781347  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.735088  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:34.783689  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 48723.4us
I20260709 12:23:34.652354  6510 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 12:23:34.786021  6544 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:34.786990  6544 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:34.787308  6544 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:34.732249  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:34.788246  6544 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:34.789974  6544 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:34.737030  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:34.792781  5148 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:34.721133  6468 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:34.797585  6468 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:34.794894  5148 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:23:34.798486  5148 pending_rounds.cc:85] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 2
I20260709 12:23:34.798792  5148 pending_rounds.cc:107] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:23:34.801030  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:34.801608  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:34.742686  5216 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:23:34.761300  6500 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:34.762001  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.763912  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.855054  5222 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:23:34.765344  6409 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:34.770097  6555 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:34.785526  6440 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:34.759002  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.817653  5076 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
W20260709 12:23:34.847864  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 46371.7us
I20260709 12:23:34.865232  5041 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:34.848461  5190 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:34.732336  6554 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:34.835573  5148 raft_consensus.cc:2889] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:34.835253  6500 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.847062  6301 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:34.846746  6468 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:34.853632  6550 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.846285  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:34.861014  6521 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:34.879978  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.886236  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.895937  5947 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:34.897652  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:34.899566  6561 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:23:34.888151  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 268543us
I20260709 12:23:34.900640  6565 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:34.901095  6565 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:34.857182  6348 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.905263  5223 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:23:34.885437  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 244827us
I20260709 12:23:34.861829  6349 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:23:34.879833  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.906756  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:34.907209  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 645) took 1029 ms. Trace:
I20260709 12:23:34.881840  6554 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:34.854022  6492 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:34.861460  6446 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:23:34.884295  6500 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:34.888386  6510 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.883061  6555 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:34.907727  5142 rpcz_store.cc:276] 0709 12:23:33.877473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.883536 (+  6063us) service_pool.cc:224] Handling call
0709 12:23:33.883861 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:34.495848 (+611987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.906928 (+411080us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54000ec754. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:34.907138 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211109120}
I20260709 12:23:34.916533  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.918694  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.889851  6516 raft_consensus.cc:3055] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:34.928233  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:34.904174  6507 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:34.929440  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:34.914716  6554 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:34.904659  6565 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:34.903383  6521 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:34.950917  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:34.974344  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.974694  5948 raft_consensus.cc:2376] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3. Re-sending same reply.
I20260709 12:23:34.952922  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:34.953792  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:34.944082  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:35.020356  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:35.046140  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 87529.1us
I20260709 12:23:34.889063  6301 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:34.922511  6535 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:34.942159  6567 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:35.050259  6567 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.050824  6567 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:35.052069  6301 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:35.053887  6535 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:35.054608  6535 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.055338  6301 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:35.032682  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 116134us
W20260709 12:23:34.974838  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.898783  6550 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.058219  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:35.060738  6550 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:34.927119  5945 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:34.956647  5071 raft_consensus.cc:1275] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:35.043361  6544 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.941645  6561 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:23:35.068054  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 170544us
W20260709 12:23:35.068171  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 219663us
I20260709 12:23:34.953344  6521 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:34.954144  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:34.943413  6516 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:34.951655  6507 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.042852  6435 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.040087  6468 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.940322  6510 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.071280  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:35.074611  6544 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.062528  6493 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:35.093056  5146 raft_consensus.cc:2393] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:35.093179  5899 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:34.942646  6517 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.063678  6080 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:23:35.079165  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:35.100695  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.109513  5071 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:35.090179  5148 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:23:35.099874  6306 consensus_meta.cc:287] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.001s
I20260709 12:23:35.045463  6554 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:35.106678  6515 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.581s	user 0.005s	sys 0.009s
W20260709 12:23:35.104342  6534 consensus_meta.cc:287] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.001s
I20260709 12:23:35.092039  5067 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:35.128360  5143 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:35.020398  6492 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:35.134197  6568 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:23:35.098097  6552 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.121469  5940 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.124473  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.100620  6564 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.140200  5146 raft_consensus.cc:2410] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:23:35.136694  6554 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:35.094740  6435 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.145198  5898 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:35.147696  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 87841.5us
W20260709 12:23:35.136876  6571 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:23:35.158519  6547 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.637s	user 0.015s	sys 0.000s
I20260709 12:23:35.159389  6392 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.160051  6392 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:35.161156  6560 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:23:35.137130  6575 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.149807  6564 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.100339  6468 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.121590  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:35.141520  6553 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:23:35.151701  5067 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:35.155040  5221 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:35.162160  6497 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:35.163547  6570 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:23:35.161502  6390 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.000s
I20260709 12:23:35.166114  6575 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:35.168048  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:35.188297  6236 consensus_meta.cc:287] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
I20260709 12:23:35.170384  6565 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.171926  6491 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:35.173112  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:35.172981  6476 consensus_meta.cc:287] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.557s	user 0.001s	sys 0.009s
I20260709 12:23:35.175833  6392 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:35.176143  6510 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:35.176148  6567 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:35.196040  6398 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.004s
I20260709 12:23:35.141089  5899 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:35.259830  6507 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.105317  6493 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:35.200103  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 55035.4us
I20260709 12:23:35.184145  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.170266  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:35.184526  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:35.141743  6438 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:35.210417  5068 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:35.288523  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 147495us
I20260709 12:23:35.137913  6516 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.226765  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.227090  6508 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.233325  6299 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:35.234757  6554 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.235066  6581 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.168147  6564 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:35.170513  6202 consensus_meta.cc:287] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.713s	user 0.008s	sys 0.010s
I20260709 12:23:35.175663  5940 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:35.165951  6449 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.240710  6550 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:35.241477  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.241930  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.242277  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.243695  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:35.244333  6468 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.245204  6390 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.126116  6306 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.373628  6306 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.175191  6544 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:35.262079  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.373974  6390 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.129356  6515 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.267858  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 693) took 1310 ms. Trace:
I20260709 12:23:35.374859  5143 rpcz_store.cc:276] 0709 12:23:33.962764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.964286 (+  1522us) service_pool.cc:224] Handling call
0709 12:23:34.662210 (+697924us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:35.267548 (+605338us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54000e2254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:35.267755 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071946880}
I20260709 12:23:35.304984  6574 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:35.106292  6555 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.124835  6348 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.170295  6521 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.158528  6535 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.289728  5899 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:35.313462  6510 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:23:35.210522  6577 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:35.273511  6497 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.212148  6558 consensus_queue.cc:1048] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:23:35.142663  6552 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.283664  6499 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.274070  6506 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.176048  6580 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.103s
W20260709 12:23:35.311615  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 237256us
I20260709 12:23:35.317255  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.325703  6409 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.289153  6438 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.271009  6236 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.331033  6299 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.293546  5943 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:35.329640  5067 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
W20260709 12:23:35.332504  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50797.6us
I20260709 12:23:35.293123  6593 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:23:35.327960  6579 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:35.338178  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 47555.7us
I20260709 12:23:35.285847  6557 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.290973  6492 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:35.141549  6561 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.311964  6392 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:35.361631  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 287468us
I20260709 12:23:35.286320  6398 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.364001  5217 raft_consensus.cc:2368] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:35.366266  6491 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:35.461094  5948 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:23:35.366472  6517 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.367580  6435 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:35.461365  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.289898  6507 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:35.370682  6588 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:23:35.313583  6534 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:23:35.443365  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.376389  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.347441  6301 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.317512  6556 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
W20260709 12:23:35.380400  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 90825.6us
I20260709 12:23:35.454281  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:35.391417  6515 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.392019  5070 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
W20260709 12:23:35.392931  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 130955us
I20260709 12:23:35.356220  6550 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:35.393764  6497 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.335445  6508 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.379791  6574 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.388942  6564 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:35.429144  5218 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:35.430109  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:35.419988  6409 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.423869  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:35.434386  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:35.402570  6506 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.350358  6581 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.505095  5144 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:35.453020  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:35.424577  6528 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.849s	user 0.012s	sys 0.000s
I20260709 12:23:35.457792  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:35.458405  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.354351  6202 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:35.402094  6438 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.421692  6236 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.463109  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:35.419785  6299 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:35.471997  6598 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:35.247718  6123 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.538300  6498 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:35.538944  6498 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:35.540949  6575 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.426290  6582 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.476490  6594 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:35.434155  6446 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:23:35.478142  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:35.461987  6392 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.479907  6569 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
W20260709 12:23:35.480742  6587 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:23:35.435144  6579 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.484270  6534 consensus_queue.cc:1243] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Peer ceec344dd00f40c18ef58414ce147016 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:23:35.451292  6557 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.428195  5068 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:35.556993  6594 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:35.490648  6565 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.491205  6459 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:35.400662  6499 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.495709  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:35.566841  5145 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.497937  5899 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:35.344961  6554 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.499280  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:35.571455  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 73596.3us
I20260709 12:23:35.572476  6564 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:35.504560  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.565675  6573 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:23:35.573185  5069 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
W20260709 12:23:35.573725  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 69281.7us
I20260709 12:23:35.572288  5899 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:35.575204  5114 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.504436  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:35.473609  6435 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:35.494413  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:35.581699  6579 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.583180  5217 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:35.583473  6438 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.584966  5939 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 4. Ops: []
I20260709 12:23:35.585079  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.585877  5220 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:35.586278  6558 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.488323  6493 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:35.588320  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.590843  5189 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:35.512174  5073 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:35.443475  5067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49646
W20260709 12:23:35.513180  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 83217.8us
I20260709 12:23:35.452392  5142 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:23:35.503109  6269 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:35.516711  5143 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:35.516880  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:35.521380  6500 consensus_meta.cc:287] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.000s
I20260709 12:23:35.522655  6507 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.525753  6521 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.526067  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.602958  5215 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:35.603178  6438 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.517757  6528 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.516216  6556 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:35.530004  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:35.501356  6301 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.532200  6409 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.533456  6581 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.538501  6123 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.434819  6306 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.613708  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:35.589885  6192 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:23:35.614065  6306 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.614542  5947 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.546537  5074 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:35.614787  6602 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:35.614678  6123 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.511389  6574 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.572669  6554 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.572266  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:35.484354  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:35.577289  5114 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.629323  6301 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:35.630061  5944 raft_consensus.cc:2368] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
W20260709 12:23:35.630366  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 53173.5us
I20260709 12:23:35.586834  6558 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.579205  6435 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.579723  6544 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:23:35.597044  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 80232us
I20260709 12:23:35.598459  5073 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:35.599020  6585 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:35.599768  6586 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:35.601579  5899 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:35.604271  6500 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.606040  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.604125  6269 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.594237  6564 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.609743  5067 raft_consensus.cc:606] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.636472  6602 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:35.546310  6582 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.615800  5223 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:35.619779  5145 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:35.604206  6565 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.643458  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:35.660624  5068 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.660457  6565 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.660863  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:35.661554  5145 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.558143  6594 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.626536  6192 raft_consensus.cc:3055] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:35.578603  6491 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:35.632668  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:23:35.691922  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 59392.9us
I20260709 12:23:35.642695  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:35.640232  5947 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:35.645252  6301 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:35.643028  5189 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:35.697753  6435 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:35.654066  6493 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:35.698560  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 55669.7us
I20260709 12:23:35.698755  6493 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.699407  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:35.700402  5147 raft_consensus.cc:2410] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:23:35.692914  6581 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.645606  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.702329  6572 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:35.702121  6581 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.703078  6572 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:35.703234  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:35.645342  6554 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.703979  6440 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.613830  6449 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:35.704326  6581 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.704505  6440 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.705035  6390 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:35.706272  6390 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:35.706646  6390 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:35.707511  6390 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.623646  6123 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.637140  6556 raft_consensus.cc:697] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:35.611908  6557 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.662055  5075 raft_consensus.cc:2368] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:35.639240  5148 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:35.659624  6564 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.668591  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:35.658566  5215 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:35.659412  6306 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:35.694305  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 51674.6us
I20260709 12:23:35.714352  6409 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:35.701680  6438 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:35.702996  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.638986  5941 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:35.704604  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.739717  5142 raft_consensus.cc:2410] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:35.701231  6493 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.714856  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.723266  6581 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.658264  5067 raft_consensus.cc:614] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:35.719877  6123 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.728387  5899 leader_election.cc:400] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:35.694773  6192 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:35.696084  6392 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:35.716280  6459 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.714031  6558 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:35.715739  5947 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:35.745368  5145 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:35.704399  6554 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.745090  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:35.753175  5899 leader_election.cc:403] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:35.761250  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:35.730554  6409 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.728173  6390 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.771158  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:35.771970  5142 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:35.762427  5145 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:35.763151  6459 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.774063  6582 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:35.764991  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:35.777976  6565 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.779543  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:35.780714  5215 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:35.781689  6554 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:35.782034  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 651) took 1334 ms. Trace:
I20260709 12:23:35.782437  5147 rpcz_store.cc:276] 0709 12:23:34.447589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.453097 (+  5508us) service_pool.cc:224] Handling call
0709 12:23:35.781790 (+1328693us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54003ae3d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:35.781973 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2687229696}
I20260709 12:23:35.772562  6409 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:35.783659  6564 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:35.770149  6493 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:35.784766  6493 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.785116  6493 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.785797  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.768085  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:35.769196  6590 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:35.769750  6558 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:23:35.783751  5149 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:23:35.798621  5149 pending_rounds.cc:85] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 2
I20260709 12:23:35.798913  5149 pending_rounds.cc:107] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:23:35.799269  5149 raft_consensus.cc:2889] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:35.766408  6123 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:35.755954  6581 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.787988  5221 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:35.787631  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 574) took 1053 ms. Trace:
I20260709 12:23:35.801347  5218 rpcz_store.cc:276] 0709 12:23:34.734331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.734527 (+   196us) service_pool.cc:224] Handling call
0709 12:23:35.170073 (+435546us) spinlock_profiling.cc:243] Waited 41.1 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:35.787432 (+617359us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400129dd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:35.787575 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":420540544}
I20260709 12:23:35.753860  6202 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:35.774077  6559 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.002s	sys 0.025s
I20260709 12:23:35.806820  6581 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.786963  6573 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.819435  6573 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.807781  6574 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.826515  5072 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 3: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:35.796973  5944 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:35.827713  5142 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:35.798240  6528 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.829010  6202 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.831677  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.804957  6500 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.809950  6557 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:35.837370  5142 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:35.799865  6498 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:35.834592  6559 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.003s	sys 0.027s
I20260709 12:23:35.827654  5068 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:35.845901  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.846612  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.857136  6516 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.861888  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.849133  5145 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:35.835006  6192 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.836097  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.846628  6581 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.857614  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.869869  5068 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:35.857209  5148 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:35.872397  6202 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:35.763682  6556 consensus_queue.cc:237] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.875665  5941 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:35.854966  6574 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.879886  5067 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:35.881215  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.881719  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.882649  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.883545  5073 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:35.884522  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.860945  6557 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:35.813215  6555 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:23:35.861708  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:35.887707  6535 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.888981  5142 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:35.891140  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.892464  5071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49646
I20260709 12:23:35.893067  5071 raft_consensus.cc:606] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.894546  5944 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:35.895087  5114 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:35.896284  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:35.896595  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:35.897039  5898 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:35.898454  6301 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:35.899015  6301 raft_consensus.cc:697] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:35.900002  6301 consensus_queue.cc:237] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.901258  6497 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:35.901974  6497 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:35.902768  6497 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:35.903194  6497 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:35.903630  6497 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:35.865247  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.863214  6573 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.867352  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.910094  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.910670  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:35.910869  5073 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:35.866078  6564 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.868640  6498 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:35.869091  6192 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.870048  6528 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:35.869884  6581 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.872215  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:35.873638  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:35.858235  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:35.882654  5067 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:35.906665  5145 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:35.859715  6299 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:35.918083  6564 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.923038  6557 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:35.907791  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.924916  6564 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:35.893441  5071 raft_consensus.cc:614] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:35.926025  5114 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:35.926623  5071 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:35.927278  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:35.908564  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.928809  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.929268  6544 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.929464  5147 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:35.930644  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:35.931805  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.932598  5147 raft_consensus.cc:2410] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:23:35.928238  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:35.911681  6535 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.912392  5073 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:35.875722  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:35.915331  6609 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:35.942549  5071 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:35.934376  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:35.934408  5073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:49646
I20260709 12:23:35.944330  5073 raft_consensus.cc:606] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:23:35.944859  5073 raft_consensus.cc:614] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:35.947865  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:35.923095  6614 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:23:35.907030  6409 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:35.956784  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:35.957702  5145 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.924937  6299 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:35.937283  5148 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.961222  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:35.938594  6594 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:35.946542  6435 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:35.967693  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.942715  6602 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:35.948117  6202 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:35.952407  6528 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:35.981201  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:35.951311  5945 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 1.
I20260709 12:23:35.963413  6609 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:35.947218  6535 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:35.986389  6602 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.006246  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:35.990873  6528 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.986227  6202 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:35.997026  6594 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:36.007658  6602 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:36.014147  6202 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:36.017199  6612 log.cc:927] Time spent T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:23:36.018006  6506 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:36.022584  6409 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:36.022804  6581 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:36.012588  6528 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:36.033735  5145 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:36.038453  6493 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:36.016065  6609 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.025698  6202 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:36.040637  5150 raft_consensus.cc:2368] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:36.025967  6558 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:36.016309  6594 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:36.041270  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:36.046066  5145 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:36.047535  6594 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:36.029042  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 67882us
I20260709 12:23:36.060869  5147 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:36.063386  6508 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:36.065496  6493 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:36.066344  6435 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.067025  6435 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:36.071008  6535 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:36.077919  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.079089  5217 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:36.079852  5076 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:23:36.080336  5076 pending_rounds.cc:85] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 1
I20260709 12:23:36.080977  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:36.080570  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.080585  5076 pending_rounds.cc:107] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:23:36.083815  5076 raft_consensus.cc:2889] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:36.081822  5042 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:36.086830  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.087420  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 659) took 1138 ms. Trace:
I20260709 12:23:36.087832  5945 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:36.088972  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 164) took 1135 ms. Trace:
I20260709 12:23:36.087844  5149 rpcz_store.cc:276] 0709 12:23:34.949088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.163739 (+214651us) service_pool.cc:224] Handling call
0709 12:23:35.783367 (+619628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.800094 (+ 16727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.087236 (+287142us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54003ae3d0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:36.087360 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1424165760}
I20260709 12:23:36.091512  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:36.089380  5074 rpcz_store.cc:276] 0709 12:23:34.953240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.953555 (+   315us) service_pool.cc:224] Handling call
0709 12:23:36.088801 (+1135246us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b540041e6d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:36.088911 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1154279168}
I20260709 12:23:36.092341  5149 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:36.092825  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:36.092900  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:36.065496  6202 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:36.096271  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:36.098532  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:36.099238  5221 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:36.100693  5073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49646
I20260709 12:23:36.101752  5042 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:36.102574  5073 raft_consensus.cc:606] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:36.107507  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.098660  6476 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:36.110355  5074 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:36.110848  6476 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:36.111934  6476 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:36.113176  6476 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:36.116456  6521 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:36.117071  6521 raft_consensus.cc:697] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:36.117601  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 476) took 1538 ms. Trace:
I20260709 12:23:36.117978  6521 consensus_queue.cc:237] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:36.118208  5070 rpcz_store.cc:276] 0709 12:23:34.579571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.583882 (+  4311us) service_pool.cc:224] Handling call
0709 12:23:34.584256 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.503729 (+919473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.117241 (+613512us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b5400327654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:36.117500 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1857138944}
I20260709 12:23:36.088999  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.129074  5899 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:23:36.128170  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.131640  5221 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:36.101673  6581 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:36.134182  5188 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:36.144035  6558 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:36.144510  6558 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:36.144909  6558 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:36.145402  6390 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:36.155552  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 285) took 1297 ms. Trace:
I20260709 12:23:36.156133  5941 rpcz_store.cc:276] 0709 12:23:34.858535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.860659 (+  2124us) service_pool.cc:224] Handling call
0709 12:23:35.443869 (+583210us) spinlock_profiling.cc:243] Waited 5.51 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:36.155302 (+711433us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54001a0254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:36.155471 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307880576}
I20260709 12:23:36.157476  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:36.158000  5948 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:36.158793  5944 raft_consensus.cc:2368] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:36.160015  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.161782  6581 raft_consensus.cc:697] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:36.163350  6581 consensus_queue.cc:237] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:36.138870  6573 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.166335  6573 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:36.045189  6609 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:36.169000  6573 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:36.106674  6535 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:36.118042  6459 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:36.174350  6535 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:36.246446  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 145310us
I20260709 12:23:36.247953  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:36.249311  5142 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:36.256852  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:36.247968  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 106464us
W20260709 12:23:36.195425  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.259660  5041 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:36.143332  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:36.261777  5070 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:36.247386  5042 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:36.266697  5149 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:36.268783  5115 leader_election.cc:400] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
I20260709 12:23:36.269984  5115 leader_election.cc:403] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:23:36.269033  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:36.273272  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:36.274035  5145 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:36.275475  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:36.271817  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:36.276580  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:36.252707  6535 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:36.250597  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.265825  6507 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.272735  6392 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:36.284477  5945 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:36.293094  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:36.293694  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.266211  6559 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:36.294579  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.295571  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:36.277177  6202 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:36.279304  6574 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:36.298812  5900 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:36.298002  5946 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:36.284195  6593 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:36.281219  6348 raft_consensus.cc:2764] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:23:36.277136  6390 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:36.281575  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:36.286653  5148 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:36.305742  5900 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:36.304152  6559 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:36.286688  6493 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:36.314863  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:36.316037  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:36.275389  6498 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:23:36.318866  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:36.282040  6573 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:23:36.328357  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.332821  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:36.294423  6392 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:36.288379  6564 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:23:36.327598  5074 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:36.339558  5070 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:36.310148  6301 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:36.304461  6574 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.331483  5941 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:36.352171  5075 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 163) took 1400 ms. Trace:
I20260709 12:23:36.354950  5075 rpcz_store.cc:276] 0709 12:23:34.951261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.952637 (+  1376us) service_pool.cc:224] Handling call
0709 12:23:36.344445 (+1391808us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54004484d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:36.352073 (+  7628us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1363401984}
I20260709 12:23:36.298317  6507 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:36.360325  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:36.352075  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:36.364048  5946 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:36.365219  6572 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:36.364753  5114 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:36.309121  6594 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:36.360893  5042 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:36.310772  6555 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:23:36.338819  6348 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:36.345636  6493 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:36.342788  6392 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:36.372668  5190 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:36.382277  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:36.382426  6534 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:36.319244  6497 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:36.386793  6572 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:36.351992  6574 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:36.419390  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.393795  6507 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:36.395100  6459 raft_consensus.cc:2764] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:23:36.326996  6508 consensus_queue.cc:1059] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:23:36.345290  6301 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:36.472491  6301 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:36.472692  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:36.461476  6497 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:36.463598  6573 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:36.474941  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.476270  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.402590  6390 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:36.409976  6610 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.690s	user 0.002s	sys 0.007s
I20260709 12:23:36.412438  6559 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:36.344022  6586 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:23:36.413501  5071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 173) took 1013 ms. Trace:
W20260709 12:23:36.419611  6626 log.cc:927] Time spent T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:23:36.490211  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:36.451514  6311 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:23:36.426661  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 98978.4us
I20260709 12:23:36.391572  6593 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3.
W20260709 12:23:36.451901  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 146278us
W20260709 12:23:36.452041  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 129552us
I20260709 12:23:36.482864  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.517796  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:36.513703  5071 rpcz_store.cc:276] 0709 12:23:35.399985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.927471 (+527486us) service_pool.cc:224] Handling call
0709 12:23:35.927721 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.943208 (+ 15487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.413088 (+469880us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b540058aa54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:36.413365 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33469184}
I20260709 12:23:36.424420  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:36.451984  6534 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.432669  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.462574  5075 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:36.531384  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:36.532090  5075 raft_consensus.cc:2393] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:36.467695  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:36.468834  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:36.476114  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:36.417080  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.481556  6586 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:23:36.425928  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 53388.5us
I20260709 12:23:36.533785  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:36.521389  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:36.522619  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:36.523751  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:36.535444  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:36.451896  6348 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:36.472066  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:36.529485  5898 leader_election.cc:400] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:36.538913  5898 leader_election.cc:403] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:23:36.461359  6552 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:36.536585  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:36.538421  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.460000  6594 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:36.472117  6398 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:36.567364  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.556344  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.561017  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:36.577584  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.568720  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:36.574612  5222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7be74bdf01fe4190992575e84da14996"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:36.525750  6446 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:36.578655  5222 raft_consensus.cc:606] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:23:36.527453  6311 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:36.582840  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:36.583509  5071 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:36.514931  6435 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:36.522703  6572 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 12:23:36.576936  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:36.567584  6555 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:36.541325  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:36.526326  6497 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.531464  6534 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:36.569068  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:36.540354  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:36.594998  5075 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:36.538159  6586 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:36.482795  6602 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:36.474992  6573 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:36.572985  6544 consensus_meta.cc:287] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.007s
I20260709 12:23:36.458480  5148 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:36.589849  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:36.573897  6476 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:36.567268  5945 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:36.592861  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 51632.2us
I20260709 12:23:36.526932  6528 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:36.578038  6552 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:23:36.570518  5898 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:36.600607  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.601495  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.587086  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.569113  5944 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:36.586426  6446 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:36.579520  6622 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:36.614302  5222 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:36.586313  6348 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:23:36.619437  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 49058.3us
W20260709 12:23:36.632488  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 131694us
I20260709 12:23:36.632742  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:36.619925  5948 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:36.579780  6621 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:23:36.590905  6521 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:36.593817  6192 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:36.602312  6398 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:36.597632  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:36.627923  5146 raft_consensus.cc:2368] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:36.631472  5149 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:36.634112  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.624922  6627 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:36.633502  5071 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:36.577749  6493 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:36.607614  6629 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:36.614151  6544 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:36.643982  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:36.638089  5898 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:36.649322  5224 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:36.600186  6497 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:36.638574  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 351) took 1723 ms. Trace:
I20260709 12:23:36.677059  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:36.747751  5069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 172) took 1459 ms (client timeout 1926 ms). Trace:
I20260709 12:23:36.626075  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.760067  6508 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:36.773115  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 129258us
I20260709 12:23:36.773708  6508 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:36.773211  5216 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:36.752869  5071 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
W20260709 12:23:36.652587  6558 consensus_meta.cc:287] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.002s
I20260709 12:23:36.776132  6508 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:36.776599  6558 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:36.779129  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:36.759570  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 121562us
W20260709 12:23:36.771170  5069 rpcz_store.cc:269] 0709 12:23:35.288088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.348553 (+ 60465us) service_pool.cc:224] Handling call
0709 12:23:36.747542 (+1398989us) spinlock_profiling.cc:243] Waited 83.5 ms on lock 0x7b54002ce554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:36.747683 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191964032}
I20260709 12:23:36.632740  5220 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:36.649796  5217 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:36.623091  6476 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:36.639786  6559 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:36.634513  6555 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.738801  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:36.791182  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:36.793092  6550 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:36.649878  6507 consensus_queue.cc:1059] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.121s
I20260709 12:23:36.767043  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:36.948779  5142 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:36.764580  5141 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:36.963605  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.952833  5071 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:36.779539  6627 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:36.786810  6622 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:36.743314  6521 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:36.941525  6508 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.649910  6311 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:36.972347  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.942636  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.974915  5220 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:36.733625  6586 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:36.946647  6299 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.963500  5148 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:23:37.045174  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.046167  5071 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:23:37.048661  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 411803us
I20260709 12:23:36.968704  6622 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.969336  6499 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:37.054210  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:36.764175  6629 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:36.783167  6446 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:37.061156  5115 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:36.957187  6555 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.050557  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 292) took 1594 ms. Trace:
I20260709 12:23:36.766289  5943 rpcz_store.cc:276] 0709 12:23:34.915629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.920826 (+  5197us) service_pool.cc:224] Handling call
0709 12:23:35.039851 (+119025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.420178 (+380327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.638271 (+1218093us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b540000ad54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:36.638492 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542267264}
I20260709 12:23:37.044661  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.055945  6440 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:36.975221  5944 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:37.072304  5041 leader_election.cc:400] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:37.082624  5041 leader_election.cc:403] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 12:23:37.055912  6636 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 12:23:37.083236  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.053694  6550 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:37.084748  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:37.086688  6633 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.451s	user 0.001s	sys 0.000s
I20260709 12:23:37.087308  6555 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.088550  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:36.945782  6348 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:37.089732  6498 raft_consensus.cc:993] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: : Instructing follower 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:23:37.089347  5216 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:37.091177  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.076027  5939 rpcz_store.cc:276] 0709 12:23:35.453580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.461216 (+  7636us) service_pool.cc:224] Handling call
0709 12:23:35.502534 (+ 41318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.585396 (+ 82862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.958761 (+1373365us) spinlock_profiling.cc:243] Waited 82.2 ms on lock 0x7b54002e5254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:37.047822 (+ 89061us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189051648}
I20260709 12:23:37.057008  6507 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.092980  5945 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:37.056032  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 364) took 1455 ms. Trace:
I20260709 12:23:37.093106  6507 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:37.093304  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:37.078960  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:37.093995  5947 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:37.093788  5942 rpcz_store.cc:276] 0709 12:23:35.597984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.603889 (+  5905us) service_pool.cc:224] Handling call
0709 12:23:35.613657 (+  9768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.662715 (+ 49058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:35.690608 (+ 27893us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:35.710192 (+ 19584us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:35.714413 (+  4221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.976584 (+1262171us) spinlock_profiling.cc:243] Waited 40.9 ms on lock 0x7b5400009454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:37.038522 (+ 61938us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.053333 (+ 14811us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94125824}
I20260709 12:23:37.095141  6521 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.096762  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.062711  6642 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:37.143353  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 87838.6us
W20260709 12:23:37.107609  5075 consensus_meta.cc:287] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.512s	user 0.007s	sys 0.004s
W20260709 12:23:37.067387  6631 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
I20260709 12:23:37.109596  5041 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:37.056722  6192 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:23:37.108891  6644 raft_consensus.cc:3055] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:37.059675  6552 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.108378  6558 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:37.116750  6643 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:37.118402  6507 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:37.122031  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:37.193993  5071 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.123356  5948 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:23:37.048962  6559 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:23:37.096902  6269 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:37.088613  6635 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 12:23:37.156436  5075 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:37.218948  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:37.220261  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:37.144704  6602 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:37.100783  6598 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:37.102193  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:37.062603  6622 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:37.167930  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 58467.5us
I20260709 12:23:37.192102  6644 raft_consensus.cc:740] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:23:37.129423  6574 consensus_meta.cc:287] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.670s	user 0.009s	sys 0.004s
I20260709 12:23:37.058072  6476 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:37.168887  6642 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:37.160796  6498 raft_consensus.cc:1081] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Signalling peer 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:23:37.162683  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 154) took 1718 ms. Trace:
W20260709 12:23:37.188378  6301 consensus_meta.cc:287] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.000s
I20260709 12:23:37.143996  6528 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:37.163574  6521 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:37.047148  6508 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.220966  5075 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
W20260709 12:23:37.202160  6535 consensus_meta.cc:287] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.900s	user 0.013s	sys 0.000s
I20260709 12:23:37.232015  5041 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:37.232645  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 67013.2us
I20260709 12:23:37.235894  5217 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:37.254801  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:37.202381  6440 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.236788  6497 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:37.204325  6573 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:37.234285  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.262382  5222 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:37.251876  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 292) took 1159 ms. Trace:
W20260709 12:23:37.235409  5150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 664) took 1785 ms (client timeout 2108 ms). Trace:
W20260709 12:23:37.264365  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:37.264477  5150 rpcz_store.cc:269] 0709 12:23:35.450223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.785498 (+335275us) service_pool.cc:224] Handling call
0709 12:23:37.235162 (+1449664us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b5400239954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.235342 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583783808}
I20260709 12:23:37.202726  6654 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:37.214403  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 306) took 1046 ms. Trace:
I20260709 12:23:37.192095  6552 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:37.247030  5071 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:37.253104  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:37.260113  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:37.270128  5075 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:37.270423  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:37.056825  6299 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:37.271515  5150 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:37.266571  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.223662  6646 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:23:37.272177  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 663) took 1823 ms. Trace:
I20260709 12:23:37.248394  6574 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:37.274994  5075 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:37.275681  5150 rpcz_store.cc:276] 0709 12:23:35.448866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.270923 (+1822057us) service_pool.cc:224] Handling call
0709 12:23:37.271218 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.271926 (+   708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.272123 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.263336  6498 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:37.279834  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:37.250630  6301 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.264025  6647 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:37.255709  6535 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:37.290901  5944 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:37.291283  6508 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.291826  6640 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:37.292444  6640 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.293236  6552 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:37.295671  6299 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:37.297134  6642 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:37.267987  5945 rpcz_store.cc:276] 0709 12:23:36.155670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.096458 (+940788us) service_pool.cc:224] Handling call
0709 12:23:37.201761 (+105303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.289068  5943 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:37.247912  5144 rpcz_store.cc:276] 0709 12:23:35.444336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.960619 (+1516283us) service_pool.cc:224] Handling call
0709 12:23:37.048689 (+ 88070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.062974 (+ 14285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.067385 (+  4411us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:37.072436 (+  5051us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:37.101506 (+ 29070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.111873 (+ 10367us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.144167 (+ 32294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.299194  5945 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:37.299530  5144 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:37.299876  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 310) took 1027 ms. Trace:
I20260709 12:23:37.300242  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 666) took 1844 ms. Trace:
I20260709 12:23:37.300297  5945 rpcz_store.cc:276] 0709 12:23:36.272973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.298441 (+1025468us) service_pool.cc:224] Handling call
0709 12:23:37.298816 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.299604 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.299813 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.300755  5144 rpcz_store.cc:276] 0709 12:23:35.455952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.298998 (+1843046us) service_pool.cc:224] Handling call
0709 12:23:37.299335 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.299957 (+   622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.300185 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.281401  6649 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:37.264274  5939 rpcz_store.cc:276] 0709 12:23:36.092112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.118072 (+1025960us) service_pool.cc:224] Handling call
0709 12:23:37.118557 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.118886 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:23:37.119363 (+   477us) log.cc:844] Serialized 14 byte log entry
0709 12:23:37.233044 (+113681us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:37.241756 (+  8712us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:37.247498 (+  5742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.250385 (+  2887us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.251806 (+  1421us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2531,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":3716,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"thread_start_us":103883,"threads_started":1,"wal-append.queue_time_us":152885}
I20260709 12:23:37.307178  6649 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.307341  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.307704  6649 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.307834  5948 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:37.280969  6521 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:37.308614  6521 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:37.309023  6521 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.309356  6535 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:37.309628  6652 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:23:37.310128  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:37.310952  5144 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:37.311501  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 670) took 2052 ms. Trace:
I20260709 12:23:37.311900  5072 rpcz_store.cc:276] 0709 12:23:35.258816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.924367 (+665551us) service_pool.cc:224] Handling call
0709 12:23:36.748246 (+823879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.748468 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:36.748484 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:36.748522 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:36.748541 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.748579 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.311325 (+562746us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54002ce550. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.311441 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1894011136}
I20260709 12:23:37.303752  6301 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:37.295804  6574 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:37.314170  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 672) took 1374 ms. Trace:
W20260709 12:23:37.314647  6392 consensus_meta.cc:287] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.861s	user 0.007s	sys 0.006s
I20260709 12:23:37.314615  5146 rpcz_store.cc:276] 0709 12:23:35.939540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.999487 (+ 59947us) service_pool.cc:224] Handling call
0709 12:23:37.313970 (+1314483us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b5400238a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.314108 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422816512}
I20260709 12:23:37.315120  6392 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:37.315877  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:37.316094  5073 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.316493  5146 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
W20260709 12:23:37.318054  5147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 671) took 1381 ms (client timeout 1633 ms). Trace:
W20260709 12:23:37.318660  5147 rpcz_store.cc:269] 0709 12:23:35.936681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.952080 (+ 15399us) service_pool.cc:224] Handling call
0709 12:23:37.317879 (+1365799us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b540019f354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.317996 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177810176}
I20260709 12:23:37.323120  5190 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:37.263422  6440 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:37.267959  6644 consensus_queue.cc:260] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:37.286382  5067 consensus_meta.cc:287] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.726s	user 0.014s	sys 0.000s
I20260709 12:23:37.335029  5067 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:37.335417  6644 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.335741  5067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 491) took 1016 ms. Trace:
I20260709 12:23:37.336236  5067 rpcz_store.cc:276] 0709 12:23:36.318939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.361828 (+ 42889us) service_pool.cc:224] Handling call
0709 12:23:37.335664 (+973836us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.306176  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:37.338150  5074 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:37.339458  5074 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:37.340873  5074 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:37.301852  6629 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:37.312556  5941 consensus_meta.cc:287] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.938s	user 0.007s	sys 0.006s
I20260709 12:23:37.285633  6647 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:37.318197  6534 consensus_meta.cc:287] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.716s	user 0.013s	sys 0.000s
I20260709 12:23:37.304196  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 675) took 1363 ms. Trace:
I20260709 12:23:37.342099  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:37.342743  5149 rpcz_store.cc:276] 0709 12:23:35.940576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.271428 (+330852us) service_pool.cc:224] Handling call
0709 12:23:37.303958 (+1032530us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b5400239454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.304128 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":811946752}
I20260709 12:23:37.343557  5071 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:37.344515  5149 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:37.345321  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 714) took 1878 ms. Trace:
I20260709 12:23:37.346258  5149 rpcz_store.cc:276] 0709 12:23:35.466688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.343919 (+1877231us) service_pool.cc:224] Handling call
0709 12:23:37.344273 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.345019 (+   746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.345254 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.348729  5073 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:37.313452  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:37.317498  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:37.358852  5069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49646
I20260709 12:23:37.359687  5069 raft_consensus.cc:606] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:37.360184  5069 raft_consensus.cc:614] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:37.362088  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:23:37.365617  5147 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:37.349136  6647 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:37.366752  5067 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:37.368139  6440 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:37.348917  6499 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:37.369128  6499 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:37.349678  6534 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:37.370538  6640 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.371616  6499 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:37.372980  5217 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:37.374110  5221 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:37.377681  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 299) took 1229 ms. Trace:
I20260709 12:23:37.378114  6629 consensus_queue.cc:1059] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:37.378578  5144 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:37.378984  6567 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.379439  6301 raft_consensus.cc:3055] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:37.379869  6301 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:37.379971  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:37.380591  5899 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:37.381743  6438 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:37.382162  6438 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:37.382552  6438 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:37.387493  6639 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.337s	user 0.003s	sys 0.000s
I20260709 12:23:37.387650  6577 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:37.388302  6577 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.380833  6301 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:37.378121  5946 rpcz_store.cc:276] 0709 12:23:36.148137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.555849 (+407712us) service_pool.cc:224] Handling call
0709 12:23:37.377446 (+821597us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540039b2d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.377607 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1403605376}
W20260709 12:23:37.394928  6637 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.436s	user 0.001s	sys 0.000s
I20260709 12:23:37.396054  5946 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.397246  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 369) took 1797 ms. Trace:
I20260709 12:23:37.397687  5947 rpcz_store.cc:276] 0709 12:23:35.599827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.763580 (+163753us) service_pool.cc:224] Handling call
0709 12:23:36.411432 (+647852us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:36.747078 (+335646us) spinlock_profiling.cc:243] Waited 7.87 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416e9c 00007fd8a5416a5b 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:37.093689 (+346611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.109142 (+ 15453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.397020 (+287878us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54003f0050. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.397182 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":843175296}
I20260709 12:23:37.389614  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.403946  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:23:37.403993  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 52231.1us
I20260709 12:23:37.359814  5146 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:37.349716  6564 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:23:37.450572  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.458628  5069 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:23:37.348299  5941 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:37.395757  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:37.465076  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:37.407236  5148 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:23:37.480365  5144 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:37.371618  6534 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:37.485587  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.460767  6564 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:37.469094  6628 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:37.489516  6564 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.516269  6609 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.519142  6301 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:37.477653  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 303) took 1325 ms (client timeout 1508 ms). Trace:
W20260709 12:23:37.522044  5941 rpcz_store.cc:269] 0709 12:23:36.149247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.157492 (+  8245us) service_pool.cc:224] Handling call
0709 12:23:37.474702 (+1317210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.524652  6306 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.512187  6649 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:37.525919  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.525274  6306 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.527377  6649 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.498843  6392 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:37.532612  6590 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.484593  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:37.532637  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 151) took 2089 ms. Trace:
I20260709 12:23:37.534431  5069 raft_consensus.cc:2368] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:37.461544  6642 raft_consensus.cc:2764] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:23:37.534274  5145 rpcz_store.cc:276] 0709 12:23:35.443513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.283724 (+840211us) service_pool.cc:224] Handling call
0709 12:23:36.299034 (+ 15310us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:36.321096 (+ 22062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:36.321134 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:36.331360 (+ 10226us) log.cc:844] Serialized 52 byte log entry
0709 12:23:36.581952 (+250592us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:36.581997 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.532153 (+950156us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b540045ae54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:37.532215 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:37.532267 (+    52us) raft_consensus.cc:1685] finished
0709 12:23:37.532282 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.532572 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":203,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":12952,"spinlock_wait_cycles":26014976,"thread_start_us":211013,"threads_started":1,"wal-append.queue_time_us":269033}
I20260709 12:23:37.533324  6590 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.542953  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 179) took 1019 ms. Trace:
I20260709 12:23:37.543812  6590 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.544940  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:37.540136  6435 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.546195  5145 raft_consensus.cc:2393] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:37.538046  5940 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:23:37.545945  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:37.546950  5141 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:37.547637  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:37.547919  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:37.548781  5141 raft_consensus.cc:2410] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:23:37.549115  5145 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:37.541155  6306 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.543408  5070 rpcz_store.cc:276] 0709 12:23:36.523513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.596364 (+ 72851us) service_pool.cc:224] Handling call
0709 12:23:36.618250 (+ 21886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.054014 (+435764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.054049 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:37.054104 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:37.054137 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.542685 (+488548us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54004493d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:37.542741 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.542889 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1001632512}
I20260709 12:23:37.551345  5141 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:37.551954  6577 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:37.552160  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 717) took 2085 ms. Trace:
I20260709 12:23:37.554169  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 716) took 2087 ms. Trace:
I20260709 12:23:37.554804  5145 rpcz_store.cc:276] 0709 12:23:35.467023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.548591 (+2081568us) service_pool.cc:224] Handling call
0709 12:23:37.548946 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.553832 (+  4886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.554093 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.559221  5190 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:37.559235  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:37.562148  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:37.563036  5146 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:37.549011  6647 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.582020  5190 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:37.580943  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:37.585275  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:37.585599  6446 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:37.585942  6438 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.586316  6299 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:37.586484  6650 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:37.586688  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.586337  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.587733  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.588188  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:37.579748  6306 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:37.552677  5141 rpcz_store.cc:276] 0709 12:23:35.467184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.549387 (+2082203us) service_pool.cc:224] Handling call
0709 12:23:37.551151 (+  1764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.551872 (+   721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.552102 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.616093  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 665) took 2160 ms. Trace:
I20260709 12:23:37.588981  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.618026  5220 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:37.598961  5115 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:37.599840  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 193988us
I20260709 12:23:37.600128  5946 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:37.597630  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:37.601672  5941 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:37.601714  6650 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:37.602572  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.583979  6390 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:23:37.622727  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:37.604990  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:37.597218  6476 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:37.605901  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.618515  6499 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:37.628954  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 297) took 1223 ms. Trace:
I20260709 12:23:37.630210  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.631093  6499 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:37.630095  5939 rpcz_store.cc:276] 0709 12:23:36.406934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.323735 (+916801us) service_pool.cc:224] Handling call
0709 12:23:37.324073 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.324303 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:23:37.324631 (+   328us) log.cc:844] Serialized 14 byte log entry
0709 12:23:37.627812 (+303181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:37.628579 (+   767us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:37.628637 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.628722 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.628889 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10420,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":2062,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":297604,"threads_started":1}
I20260709 12:23:37.625916  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:37.631815  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.632503  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.633044  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:37.633441  5939 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:37.633850  5144 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:37.634709  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:37.607501  6440 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.635200  5144 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:37.635371  5939 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:37.635919  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 158) took 2028 ms. Trace:
I20260709 12:23:37.635531  6440 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.636760  6564 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:37.609016  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.613914  6306 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:37.637920  6306 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:37.638048  6438 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:37.624070  6446 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:23:37.629102  5217 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:37.618306  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.606402  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.620882  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:37.626083  6435 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:37.640489  6435 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.597378  6622 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.640956  6435 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:37.642606  5149 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:37.643718  6552 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.644515  6552 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:37.646557  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:37.650872  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 290) took 2198 ms. Trace:
I20260709 12:23:37.651593  5940 rpcz_store.cc:276] 0709 12:23:35.452599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.460933 (+  8334us) service_pool.cc:224] Handling call
0709 12:23:35.777984 (+317051us) spinlock_profiling.cc:243] Waited 8.26 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:37.350458 (+1572474us) spinlock_profiling.cc:243] Waited 8.65 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416e9c 00007fd8a5416a5b 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:37.537650 (+187192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.546638 (+  8988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.650490 (+103852us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540000c650. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.650743 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425635200}
I20260709 12:23:37.653990  6390 raft_consensus.cc:3055] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:37.654644  6390 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:37.655803  6390 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.657951  6301 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:37.658459  6301 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.658905  6301 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:37.662074  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 477) took 2478 ms. Trace:
I20260709 12:23:37.662176  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:37.616643  5150 rpcz_store.cc:276] 0709 12:23:35.455542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.276561 (+1821019us) service_pool.cc:224] Handling call
0709 12:23:37.276809 (+   248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.277146 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.277169 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:37.282900 (+  5731us) log.cc:844] Serialized 26 byte log entry
0709 12:23:37.434000 (+151100us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:37.434068 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.434183 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:37.615852 (+181669us) raft_consensus.cc:1685] finished
0709 12:23:37.615875 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.616027 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":5164,"thread_start_us":114999,"threads_started":1,"wal-append.queue_time_us":294222}
I20260709 12:23:37.621289  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 382) took 1032 ms. Trace:
I20260709 12:23:37.664845  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:37.594733  6521 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:37.672300  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 43806.8us
I20260709 12:23:37.616838  5215 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.676146  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:37.627462  6476 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:37.632440  6557 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:37.624280  6550 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:37.615532  5141 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:37.636320  5144 rpcz_store.cc:276] 0709 12:23:35.607616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.634628 (+2027012us) service_pool.cc:224] Handling call
0709 12:23:37.634993 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.635659 (+   666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.635853 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.639325  6349 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:37.662606  5076 rpcz_store.cc:276] 0709 12:23:35.183360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.183539 (+   179us) service_pool.cc:224] Handling call
0709 12:23:35.183964 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.092420 (+908456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.661769 (+1569349us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b540058a554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:37.661979 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2059305344}
I20260709 12:23:37.602927  6299 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:37.617630  6577 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:23:37.667081  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:37.673710  5898 leader_election.cc:400] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:37.708093  5898 leader_election.cc:403] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:37.680085  6629 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:37.713750  6629 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.714306  6629 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:37.710541  6517 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:37.718327  6517 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.718713  6517 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.719074  6650 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:37.720382  6440 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.721493  6577 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:37.722072  6577 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:37.724020  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 141) took 1142 ms. Trace:
I20260709 12:23:37.723660  6577 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:37.725778  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 52264.6us
I20260709 12:23:37.728410  5945 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:37.729492  6628 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:37.680795  6647 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.681303  6476 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:37.665055  5946 rpcz_store.cc:276] 0709 12:23:36.588740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.395633 (+806893us) service_pool.cc:224] Handling call
0709 12:23:37.395910 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.620881 (+224971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.621214 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.632599  5941 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:37.690104  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:37.690557  6459 raft_consensus.cc:3055] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:37.692178  6299 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:37.692458  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 155) took 2247 ms. Trace:
I20260709 12:23:37.674911  6634 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:37.675159  6609 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:37.675406  6311 consensus_queue.cc:1059] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:23:37.684962  6446 raft_consensus.cc:3055] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:37.695452  6306 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:37.698670  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 715) took 2231 ms. Trace:
I20260709 12:23:37.701035  6586 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.677651  6390 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.703078  5939 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:37.701156  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 490) took 1385 ms. Trace:
I20260709 12:23:37.782552  5073 rpcz_store.cc:276] 0709 12:23:36.316025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.318645 (+  2620us) service_pool.cc:224] Handling call
0709 12:23:37.700947 (+1382302us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b5400219b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.701089 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2270475136}
I20260709 12:23:37.706846  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 380) took 1118 ms. Trace:
I20260709 12:23:37.783809  5948 rpcz_store.cc:276] 0709 12:23:36.588163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.635580 (+ 47417us) service_pool.cc:224] Handling call
0709 12:23:37.706559 (+1070979us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b540000a354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.706750 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405056768}
I20260709 12:23:37.785254  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.786116  5948 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:37.725449  6577 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:37.710446  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 722) took 2161 ms. Trace:
I20260709 12:23:37.677901  6499 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:37.681766  6550 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:37.712994  6521 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.724653  5221 rpcz_store.cc:276] 0709 12:23:36.581745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.582258 (+   513us) service_pool.cc:224] Handling call
0709 12:23:37.309278 (+727020us) spinlock_profiling.cc:243] Waited 4.81 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:37.373916 (+ 64638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.374506 (+   590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.723612 (+349106us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b5400273dd0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.723854 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159124224}
I20260709 12:23:37.768921  5189 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:37.773186  5149 rpcz_store.cc:276] 0709 12:23:35.466858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.347411 (+1880553us) service_pool.cc:224] Handling call
0709 12:23:37.347795 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.643204 (+295409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.698378 (+ 55174us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b540014f5d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:37.698592 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677382400}
I20260709 12:23:37.765602  6609 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:37.776078  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 496339us
I20260709 12:23:37.743901  6459 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:37.773700  5940 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:37.787214  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:37.709594  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:37.789795  6564 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:37.772919  6349 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.791687  6535 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:37.795876  6306 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:37.782828  6269 raft_consensus.cc:3055] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:37.780453  6622 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.770015  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:37.770090  6446 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:37.797158  5042 leader_election.cc:400] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:37.842924  5042 leader_election.cc:403] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:37.802443  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 163) took 1486 ms. Trace:
I20260709 12:23:37.744496  6348 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:37.806145  6609 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:37.809996  6550 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.790253  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 176) took 1267 ms. Trace:
W20260709 12:23:37.817729  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.793342  6567 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:37.809262  5041 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:37.821156  5948 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:23:37.824487  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 55704.8us
I20260709 12:23:37.840529  6269 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:37.853272  5189 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:37.854432  6269 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.855788  6269 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:37.855904  6630 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:37.856460  6630 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.856801  6630 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.836398  6409 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:37.858242  6409 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:37.858946  5939 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:37.859256  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 713) took 2392 ms. Trace:
I20260709 12:23:37.859716  5147 rpcz_store.cc:276] 0709 12:23:35.466495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.319594 (+1853099us) service_pool.cc:224] Handling call
0709 12:23:37.319920 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.366206 (+ 46286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.858982 (+492776us) spinlock_profiling.cc:243] Waited 45.4 ms on lock 0x7b54003421d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:37.859187 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104516992}
I20260709 12:23:37.863029  5072 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:37.865167  5940 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:37.776664  6634 raft_consensus.cc:697] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:23:37.830648  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 121190us
I20260709 12:23:37.834894  6590 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:37.874620  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.796193  5145 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:37.875324  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:37.880301  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 161) took 1935 ms. Trace:
I20260709 12:23:37.880663  5145 rpcz_store.cc:276] 0709 12:23:35.944417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.795480 (+1851063us) service_pool.cc:224] Handling call
0709 12:23:37.795849 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.879974 (+ 84125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.880228 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.799825  6499 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.797495  6534 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:23:37.889631  6499 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.834460  6654 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:23:37.895006  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 674) took 1954 ms. Trace:
I20260709 12:23:37.895215  5143 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:37.895519  5142 rpcz_store.cc:276] 0709 12:23:35.940271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.256465 (+316194us) service_pool.cc:224] Handling call
0709 12:23:37.894759 (+1638294us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b540019ee54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.894935 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199730048}
I20260709 12:23:37.896732  5144 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:37.896874  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 152) took 2453 ms. Trace:
I20260709 12:23:37.848407  6535 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:37.843593  5149 rpcz_store.cc:276] 0709 12:23:36.315874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.801316 (+1485442us) service_pool.cc:224] Handling call
0709 12:23:37.801758 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.801983 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.802015 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:37.802089 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:37.802134 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.802205 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.802372 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.897368  5143 rpcz_store.cc:276] 0709 12:23:35.443771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.790488 (+1346717us) service_pool.cc:224] Handling call
0709 12:23:37.894938 (+1104450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.895710 (+   772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.896668 (+   958us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540019ee50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.896811 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2539024128}
I20260709 12:23:37.762836  5148 rpcz_store.cc:276] 0709 12:23:35.444632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.121910 (+1677278us) service_pool.cc:224] Handling call
0709 12:23:37.343294 (+221384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.486997 (+143703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.692173 (+205176us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400239450. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.692371 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468540800}
W20260709 12:23:37.846417  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 49399.6us
I20260709 12:23:37.905412  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:37.839488  6632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.022s	sys 0.055s
I20260709 12:23:37.827502  6440 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:37.910226  5042 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:37.852464  6567 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:37.832682  5146 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
W20260709 12:23:37.875147  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 66025.5us
I20260709 12:23:37.863721  6517 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.933714  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.842311  6459 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:37.934747  6459 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.934831  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.850632  6348 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.850548  6649 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:37.878031  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:37.937680  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.937952  6348 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:37.939056  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.939052  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:37.940786  6392 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:37.941244  6392 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:37.941646  6392 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.877686  6446 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:37.944507  6649 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:37.945099  6446 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:37.947371  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:37.947994  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:37.877978  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.948730  5076 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:37.878188  6507 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:37.729462  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 296) took 1411 ms. Trace:
I20260709 12:23:37.949967  5940 raft_consensus.cc:1275] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:37.899060  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 681) took 1384 ms. Trace:
I20260709 12:23:37.800211  5141 rpcz_store.cc:276] 0709 12:23:35.549350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.614696 (+2065346us) service_pool.cc:224] Handling call
0709 12:23:37.615133 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.683548 (+ 68415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.710366 (+ 26818us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.950590  5076 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:37.951189  5940 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:39870
I20260709 12:23:37.882223  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:37.837226  6572 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.952440  6468 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:23:37.952616  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 70532.5us
I20260709 12:23:37.952755  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:37.931797  5143 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:37.951933  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 731) took 1342 ms. Trace:
I20260709 12:23:37.953691  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 165) took 1427 ms. Trace:
I20260709 12:23:37.851449  6582 raft_consensus.cc:3055] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:37.948119  5072 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:37.833636  6349 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:37.952296  6572 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.962285  6572 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:37.963778  5221 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:37.967178  6556 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:37.968261  4895 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43222 (request call id 140) took 8446 ms. Trace:
I20260709 12:23:37.969195  6556 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:37.969753  6510 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:37.970559  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:37.971779  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 313) took 1444 ms. Trace:
I20260709 12:23:37.972210  5943 rpcz_store.cc:276] 0709 12:23:36.526967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.084427 (+557460us) service_pool.cc:224] Handling call
0709 12:23:37.971572 (+887145us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400003a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:37.971715 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459867392}
I20260709 12:23:37.974053  6535 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:37.976948  4899 catalog_manager.cc:5697] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:37.977697  6634 consensus_queue.cc:237] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:37.978317  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:37.978982  5150 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:37.979071  4899 catalog_manager.cc:5697] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:37.981256  4899 catalog_manager.cc:5697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:37.982800  6521 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:37.984174  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:37.984413  6299 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.985159  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:37.984927  6567 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:37.957130  6534 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:23:37.985802  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.985919  6534 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:37.986349  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.950214  5945 rpcz_store.cc:276] 0709 12:23:36.317814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.362081 (+1044267us) service_pool.cc:224] Handling call
0709 12:23:37.727458 (+365377us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:37.727825 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.729032 (+  1207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.729374 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31774592}
I20260709 12:23:37.986938  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:37.987819  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35624 (request call id 749) took 1367 ms. Trace:
I20260709 12:23:37.988196  5223 rpcz_store.cc:276] 0709 12:23:36.620461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.621114 (+   653us) service_pool.cc:224] Handling call
0709 12:23:37.986739 (+1365625us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:37.987755 (+  1016us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10880384}
I20260709 12:23:37.988722  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:37.928318  5145 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:37.990062  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 679) took 1561 ms. Trace:
I20260709 12:23:37.990411  5145 rpcz_store.cc:276] 0709 12:23:36.428383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.881978 (+1453595us) service_pool.cc:224] Handling call
0709 12:23:37.912705 (+ 30727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.989797 (+ 77092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.990005 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.991377  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:37.954094  5143 rpcz_store.cc:276] 0709 12:23:36.526059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.898374 (+1372315us) service_pool.cc:224] Handling call
0709 12:23:37.927348 (+ 28974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.953393 (+ 26045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.953629 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.867197  5215 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
W20260709 12:23:37.914245  6632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.023s	sys 0.055s
I20260709 12:23:37.954861  5149 rpcz_store.cc:276] 0709 12:23:36.515089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.898171 (+1383082us) service_pool.cc:224] Handling call
0709 12:23:37.898498 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.898699 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.898727 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:37.898777 (+    50us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:37.898810 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.898892 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.898996 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.959141  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:37.986268  6567 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:37.956014  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:37.982937  4899 catalog_manager.cc:5697] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:38.019860  5222 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:37.998795  5190 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:38.020565  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 64635.5us
I20260709 12:23:38.002513  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.005923  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.019974  5220 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:38.012671  5150 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:37.958192  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:38.023756  5143 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:37.852020  5068 rpcz_store.cc:276] 0709 12:23:36.522652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.525305 (+  2653us) service_pool.cc:224] Handling call
0709 12:23:36.577303 (+ 51998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.649726 (+ 72423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:36.735221 (+ 85495us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:36.747318 (+ 12097us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:36.753202 (+  5884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.789981 (+1036779us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b540024efd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:37.790041 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.790184 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116748032}
I20260709 12:23:38.026188  4899 catalog_manager.cc:5697] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 2 to 3, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:38.028316  5042 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:37.900126  6557 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.114s
I20260709 12:23:38.069859  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:38.054493  5150 raft_consensus.cc:1275] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:38.070875  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:38.072201  5150 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:38.061064  4899 catalog_manager.cc:5697] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:38.064900  5149 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:37.940418  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:37.986375  6534 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:37.955240  6654 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:38.083055  5145 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:38.082469  4899 catalog_manager.cc:5697] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 1 to 2, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:37.908867  6440 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:37.968710  4895 rpcz_store.cc:276] 0709 12:23:29.541857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.559698 (+ 17841us) service_pool.cc:224] Handling call
0709 12:23:37.968127 (+8408429us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:34.100274 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:34.288053 (+187779us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:34.288103 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:23:34.288116 (+    13us) tablet.cc:662] Decoding operations
0709 12:23:34.289014 (+   898us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:34.289081 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:34.289110 (+    29us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:23:34.289763 (+   653us) tablet.cc:701] Row locks acquired
0709 12:23:34.289776 (+    13us) write_op.cc:260] PREPARE: finished
0709 12:23:34.289934 (+   158us) write_op.cc:270] Start()
0709 12:23:34.290058 (+   124us) write_op.cc:276] Timestamp: P: 1783599814289885 usec, L: 0
0709 12:23:34.290075 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 12:23:34.290819 (+   744us) log.cc:844] Serialized 2848 byte log entry
0709 12:23:35.021717 (+730898us) op_driver.cc:464] REPLICATION: finished
0709 12:23:35.357505 (+335788us) write_op.cc:301] APPLY: starting
0709 12:23:35.362996 (+  5491us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:35.398819 (+ 35823us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:35.401400 (+  2581us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:35.477823 (+ 76423us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:35.478067 (+   244us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:23:35.478123 (+    56us) write_op.cc:312] APPLY: finished
0709 12:23:35.478477 (+   354us) log.cc:844] Serialized 58 byte log entry
0709 12:23:35.478685 (+   208us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:35.478827 (+   142us) write_op.cc:454] Released schema lock
0709 12:23:35.478883 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:35.479331 (+   448us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:35.479371 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":274684,"apply.run_cpu_time_us":4529,"apply.run_wall_time_us":133952,"num_ops":7,"prepare.queue_time_us":69092,"prepare.run_cpu_time_us":22331,"prepare.run_wall_time_us":436189,"raft.queue_time_us":253993,"raft.run_cpu_time_us":9398,"raft.run_wall_time_us":282559,"replication_time_us":640169,"thread_start_us":569625,"threads_started":4,"wal-append.queue_time_us":278287,"wal-append.run_cpu_time_us":29113,"wal-append.run_wall_time_us":1885594}]]}
I20260709 12:23:38.086217  5143 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:38.088686  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:38.054152  6521 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.993134  6552 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:37.908978  6306 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.120s
I20260709 12:23:38.003417  6582 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:38.079777  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:38.023634  6349 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:38.025072  6622 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.113714  6392 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:38.101181  6521 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:38.120180  6622 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:38.091509  6557 consensus_queue.cc:1243] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer ceec344dd00f40c18ef58414ce147016 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:23:38.116628  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 497) took 1153 ms. Trace:
I20260709 12:23:38.105494  5223 raft_consensus.cc:2368] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:38.092504  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.053841  6559 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:38.124568  5074 rpcz_store.cc:276] 0709 12:23:36.962803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.341865 (+379062us) service_pool.cc:224] Handling call
0709 12:23:37.342175 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.794239 (+452064us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.794275 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:37.794822 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:23:37.914711 (+119889us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:37.914760 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.914833 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:38.116368 (+201535us) raft_consensus.cc:1685] finished
0709 12:23:38.116394 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.116551 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":451633,"raft.queue_time_us":452799,"spinlock_wait_cycles":8970496,"thread_start_us":119484,"threads_started":1,"wal-append.queue_time_us":292571}
I20260709 12:23:38.127995  6559 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:38.128293  6468 raft_consensus.cc:697] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:37.945155  6649 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:38.063915  6507 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:38.068279  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 379) took 1480 ms. Trace:
I20260709 12:23:38.077206  6498 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:37.953873  5141 rpcz_store.cc:276] 0709 12:23:36.609178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.951040 (+1341862us) service_pool.cc:224] Handling call
0709 12:23:37.951361 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.951569 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.951602 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:37.951651 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:37.951687 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.951746 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.951869 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:38.021953  6572 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:23:37.984925  6299 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:38.128537  6559 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:38.053601  6409 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:23:38.111543  6655 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.638s	user 0.003s	sys 0.002s
I20260709 12:23:38.137776  6507 raft_consensus.cc:697] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:38.149554  6498 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:38.147281  6476 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:38.153008  5075 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 689) took 1523 ms. Trace:
I20260709 12:23:38.169718  5075 rpcz_store.cc:276] 0709 12:23:36.629662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.276696 (+647034us) service_pool.cc:224] Handling call
0709 12:23:37.277141 (+   445us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:37.775971 (+498830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.776003 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:37.784048 (+  8045us) log.cc:844] Serialized 52 byte log entry
0709 12:23:38.087650 (+303602us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:38.087694 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.087766 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:38.152648 (+ 64882us) raft_consensus.cc:1685] finished
0709 12:23:38.152680 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.152922 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":502792,"raft.queue_time_us":501699,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":14924,"thread_start_us":247154,"threads_started":1,"wal-append.queue_time_us":336492}
I20260709 12:23:38.154572  6516 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:38.154277  6629 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:38.114388  6517 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:38.075465  5150 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:38.156635  5944 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:38.079298  6644 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:23:38.093426  5941 raft_consensus.cc:2376] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5. Re-sending same reply.
W20260709 12:23:38.148674  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 56262.6us
I20260709 12:23:38.168222  6507 consensus_queue.cc:237] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:38.172814  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:38.172857  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:38.211891  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:38.181732  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.182565  5069 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:38.184579  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:38.214847  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.190758  5070 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:38.194023  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:38.174914  6669 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:23:38.217839  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.195780  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:38.132371  6557 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:38.099387  6435 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:38.220990  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.240615  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.227722  5114 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:38.242698  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 240380us
I20260709 12:23:38.228168  6682 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:38.165920  6590 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:38.243830  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:38.245801  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.173710  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:38.179823  6390 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.252862  5071 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:23:38.279698  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 66038.3us
I20260709 12:23:38.192092  6521 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:38.162503  6301 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:38.299872  5142 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:23:38.237217  6516 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:38.236940  6632 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:38.234002  6650 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:38.270056  6498 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:38.322134  6673 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:23:38.234477  6642 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:38.285635  6535 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:38.271278  6574 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:38.192050  6564 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.101s
I20260709 12:23:38.210878  6348 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.101s
I20260709 12:23:38.271121  6573 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:38.287533  6468 consensus_queue.cc:237] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:38.324796  5220 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:38.208424  6438 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 12:23:38.293324  5215 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:38.346189  5947 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:23:38.315320  6622 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:38.359549  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 725) took 2043 ms. Trace:
I20260709 12:23:38.248543  6557 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:38.291038  6390 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:38.386881  6680 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.161s	user 0.001s	sys 0.004s
I20260709 12:23:38.389642  6435 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:38.351877  6586 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:23:38.392112  6311 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:23:38.347110  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 112700us
W20260709 12:23:38.407612  6648 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 1.142s	user 0.005s	sys 0.011s
W20260709 12:23:38.407871  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 162175us
I20260709 12:23:38.376788  5068 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:23:38.421516  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 137044us
I20260709 12:23:38.403714  5147 rpcz_store.cc:276] 0709 12:23:36.316216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.860737 (+1544521us) service_pool.cc:224] Handling call
0709 12:23:37.861165 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.875726 (+ 14561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.882114 (+  6388us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:37.886905 (+  4791us) log.cc:844] Serialized 26 byte log entry
0709 12:23:38.055037 (+168132us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:38.055090 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.055179 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:38.359193 (+304014us) raft_consensus.cc:1685] finished
0709 12:23:38.359222 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.359466 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12075,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":105,"thread_start_us":167701,"threads_started":1,"wal-append.queue_time_us":275828}
I20260709 12:23:38.249023  6582 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:38.397607  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 761) took 1348 ms. Trace:
I20260709 12:23:38.425233  5041 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:38.427328  6557 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:38.429808  6392 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:38.386354  6535 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:38.141319  5942 rpcz_store.cc:276] 0709 12:23:36.587845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.386402 (+798557us) service_pool.cc:224] Handling call
0709 12:23:37.386779 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.399754 (+ 12975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.399819 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:37.400434 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 12:23:37.762026 (+361592us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:37.771827 (+  9801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.778062 (+  6235us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:37.992719 (+214657us) raft_consensus.cc:1685] finished
0709 12:23:38.014415 (+ 21696us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.053427 (+ 39012us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3240,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":794,"thread_start_us":341954,"threads_started":1,"wal-append.queue_time_us":450736}
I20260709 12:23:38.403687  6269 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:23:38.440448  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.417577  6311 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:38.439814  6507 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.521759  5224 raft_consensus.cc:3055] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:38.524823  5224 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:38.438930  6644 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:38.455732  6534 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:38.438143  5216 rpcz_store.cc:276] 0709 12:23:37.049526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.092105 (+ 42579us) service_pool.cc:224] Handling call
0709 12:23:37.092512 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.124234 (+ 31722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.124289 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:37.125018 (+   729us) log.cc:844] Serialized 26 byte log entry
0709 12:23:37.271819 (+146801us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:37.277447 (+  5628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.281759 (+  4312us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:38.073153 (+791394us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:23:38.303586 (+230433us) raft_consensus.cc:1685] finished
0709 12:23:38.303624 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.377249 (+ 73625us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21428,"raft.queue_time_us":33038,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":23663,"spinlock_wait_cycles":6941568,"thread_start_us":136633,"threads_started":1,"wal-append.queue_time_us":250114}
I20260709 12:23:38.441036  6440 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:38.443559  6582 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:38.452926  6459 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.004s
I20260709 12:23:38.459574  6629 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:38.444532  6476 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:38.445047  6306 raft_consensus.cc:3055] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:23:38.498545  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 58227.7us
I20260709 12:23:38.447060  6689 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.502599  6521 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:38.583063  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 83416us
I20260709 12:23:38.521706  6654 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:38.504536  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 680) took 2049 ms. Trace:
I20260709 12:23:38.505033  5068 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:38.505192  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:38.511897  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 730) took 1903 ms. Trace:
I20260709 12:23:38.599370  5142 rpcz_store.cc:276] 0709 12:23:36.455473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.896443 (+1440970us) service_pool.cc:224] Handling call
0709 12:23:37.896872 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:38.300524 (+403652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.504237 (+203713us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b54003ae3d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:38.504469 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":925947776}
W20260709 12:23:38.512435  6499 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.001s
I20260709 12:23:38.603647  6499 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:38.606040  6499 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:38.555459  6507 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:38.513245  6687 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.607446  6687 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:38.608322  6534 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:38.513437  6683 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.609194  6683 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:38.599686  5146 rpcz_store.cc:276] 0709 12:23:36.608629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.983388 (+1374759us) service_pool.cc:224] Handling call
0709 12:23:37.983683 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.983899 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:23:38.103645 (+119746us) log.cc:844] Serialized 14 byte log entry
0709 12:23:38.511354 (+407709us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b18003f6d60. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a53c8c0a 00007fd8aeb08219 00007fd8a5450a40 00007fd8a544b4e4 00007fd8a544b455 00007fd8a8533060 00007fd8a8545716 00007fd8a8556287 00007fd8a8556208
0709 12:23:38.511539 (+   185us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:38.511596 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:38.511645 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.511716 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.511829 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47277,"prepare.queue_time_us":202672,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":70,"raft.queue_time_us":291262,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":38214528,"thread_start_us":239535,"threads_started":2,"wal-append.queue_time_us":120571}
I20260709 12:23:38.516826  6590 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:38.611433  6687 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:38.522997  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:38.522864  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 89865us
I20260709 12:23:38.612217  6311 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:38.535574  5898 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:38.613433  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 77982.7us
I20260709 12:23:38.598060  6392 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:23:38.614387  6392 raft_consensus.cc:3055] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:38.614805  6392 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:38.615723  6392 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:38.554788  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 327176us
I20260709 12:23:38.616905  6392 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:38.517488  6577 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.709s	user 0.004s	sys 0.004s
I20260709 12:23:38.581727  5041 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:38.503201  6695 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:38.524436  6301 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:38.617944  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.619266  5041 leader_election.cc:400] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:38.619180  5114 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:38.619706  5041 leader_election.cc:403] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:23:38.619946  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:38.541432  6586 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.583405  6693 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.517526  6679 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:38.601912  6689 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:38.602743  6521 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:38.608596  6507 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:38.611265  6550 consensus_meta.cc:287] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.737s	user 0.007s	sys 0.006s
I20260709 12:23:38.551167  6446 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 12:23:38.612429  6676 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.195s	user 0.001s	sys 0.002s
I20260709 12:23:38.620553  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:38.635087  6609 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.763s	user 0.006s	sys 0.007s
I20260709 12:23:38.620656  5074 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:38.621451  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:38.628906  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.631469  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.647718  5073 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:38.630954  6534 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4.
I20260709 12:23:38.634232  6650 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:38.569231  6299 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:38.666580  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:38.606081  6390 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:38.572019  6306 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:23:38.633747  6630 consensus_meta.cc:287] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.776s	user 0.012s	sys 0.003s
I20260709 12:23:38.643852  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.550145  5222 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:38.630399  6459 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:38.671164  6306 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:38.672473  6306 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:38.673854  6630 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:38.676294  6630 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:38.668479  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:38.611497  6683 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:38.678570  6651 raft_consensus.cc:3055] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:38.679226  6651 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:38.631578  6590 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:38.680305  6590 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:38.680756  6590 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:38.628803  6644 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:38.632474  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.633903  6679 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.687175  6679 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:38.687521  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 55103.8us
I20260709 12:23:38.687731  6468 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:38.521786  6535 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:38.688756  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:38.689834  6687 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:38.656796  5898 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:38.691524  6534 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:38.691821  6301 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:38.692276  6534 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.657945  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:38.645517  6556 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:23:38.693168  5947 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:38.693523  6556 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:38.693954  6689 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:38.695379  5215 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:38.695636  6521 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:38.659148  5944 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:23:38.693971  6556 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:38.658953  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:38.635588  6586 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:38.636669  6693 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:38.659935  6696 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:38.650724  6609 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:38.659030  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:38.663475  5073 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:38.644061  5067 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:38.643621  6269 consensus_queue.cc:1243] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:23:38.667403  5220 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:38.645072  6550 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:38.655272  6440 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:38.669939  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:38.673983  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:38.692296  6301 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:38.642374  6642 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.019s
W20260709 12:23:38.645860  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 402117us
I20260709 12:23:38.702375  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:38.687534  5114 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:38.667368  6650 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:38.712126  5947 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:38.718670  5946 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:38.747577  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 45261.7us
I20260709 12:23:38.719535  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:38.720599  6534 raft_consensus.cc:2764] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:23:38.706969  6700 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:38.708201  6644 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:38.741113  5148 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:38.680583  6651 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:38.742702  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 72840.5us
I20260709 12:23:38.685024  6559 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:38.746104  6301 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:38.741112  6499 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:38.739352  5940 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:38.794051  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:38.785740  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.823077  5073 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:38.750896  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:38.752233  5947 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:38.783113  5946 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
W20260709 12:23:38.782238  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 94787.9us
I20260709 12:23:38.753134  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:38.795650  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:38.796057  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:38.796700  5190 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:38.798056  5218 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:38.802734  6269 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:38.841560  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44937.2us
I20260709 12:23:38.839075  5150 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:38.803856  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:38.804585  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:38.788267  6348 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:38.850981  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:38.805830  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:38.806169  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:38.806514  6609 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:38.716539  6556 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:38.807775  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:38.808316  6586 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:38.863911  5150 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:23:38.808933  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:38.865703  5145 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:38.791301  6468 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:38.810142  6559 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:38.839859  5221 raft_consensus.cc:2368] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4.
W20260709 12:23:38.842980  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.840384  5940 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:38.840587  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:38.803623  6572 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.844982  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:38.874650  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:38.845762  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:38.876345  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:38.877193  5073 raft_consensus.cc:2410] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:38.845949  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:38.846799  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:38.843693  5146 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:38.880143  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:38.747255  6696 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:38.781466  6521 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:38.852396  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:38.746783  6440 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:38.854081  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:38.802803  6705 log.cc:927] Time spent T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:23:38.854843  5147 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:38.864693  6687 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:38.853478  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 102706us
I20260709 12:23:38.857115  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:38.791271  6700 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:38.793701  6702 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 12:23:38.842765  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:38.869405  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.749977  6650 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:38.814360  6499 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:38.871379  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:38.871445  5150 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:38.851581  6476 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:38.872977  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:38.851194  6642 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.860598  6348 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:38.791859  6644 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:38.877030  5221 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:38.885742  5218 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:38.886886  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:38.841660  6409 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:23:38.864583  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:38.848642  6654 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:23:38.894834  5041 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:38.895807  5223 raft_consensus.cc:2368] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:38.868865  6609 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.854884  6349 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:23:38.781445  6581 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:23:38.855365  6574 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:23:38.903962  6299 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:38.859686  6564 consensus_queue.cc:1059] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:23:38.885435  6516 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:38.925321  5115 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:38.951396  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:38.927605  5219 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:38.951546  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:38.952065  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:38.955070  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:38.931396  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:38.965601  5223 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:39.013686  5947 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:38.965605  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:38.913511  5146 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:39.015005  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:39.015290  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:38.955859  6688 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:23:38.987126  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:38.919610  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:39.006023  5145 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
W20260709 12:23:38.955509  6674 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:23:38.930689  6630 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 12:23:38.887483  6556 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:39.004669  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:39.013589  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 61591.2us
I20260709 12:23:39.015996  5142 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
W20260709 12:23:39.008297  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 83092.9us
I20260709 12:23:39.057847  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:39.038753  5145 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:38.914939  6687 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:38.927712  6696 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 12:23:38.965116  5945 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:38.966454  6476 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:39.067034  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:39.068032  5147 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:39.050674  5143 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:38.950848  6269 consensus_queue.cc:1059] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:23:39.072021  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:39.075261  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:39.081902  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:39.080904  6692 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:39.083254  5947 raft_consensus.cc:2410] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:39.085990  6559 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:39.091313  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:39.092483  5146 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:39.012156  6701 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:39.038023  6682 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:39.016475  6550 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:38.990996  6642 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.022512  6521 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:39.007915  6573 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 12:23:38.987612  6609 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.066890  5141 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:39.068058  6516 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:39.067478  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:39.117401  6707 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:23:39.041577  6590 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.076043  6392 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:23:39.117406  6709 log.cc:927] Time spent T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 12:23:39.178576  6644 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:39.090226  6564 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:23:39.179230  6521 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:39.117741  6708 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 12:23:39.110366  6348 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.090376  6702 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.115752  6682 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.185698  6682 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:39.185608  6702 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.128574  5217 raft_consensus.cc:2410] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:39.117501  6390 raft_consensus.cc:3055] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:39.158394  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.194846  5221 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:39.161763  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.113509  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:39.165148  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.040831  6440 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:39.171782  6711 log.cc:927] Time spent T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:23:39.175511  6687 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:39.016326  6499 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.176329  6476 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.164296  6392 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.137089  6679 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:39.117228  6692 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:39.009181  6299 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:39.211587  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 316800us
W20260709 12:23:39.217890  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 160133us
I20260709 12:23:39.236974  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:39.087581  6700 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.194671  6390 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:39.253155  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:39.231953  6567 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.225730  6701 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.252437  5188 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:39.238407  6392 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:39.232534  6687 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:39.254827  5190 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:39.241312  6499 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:39.268333  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.290033  5074 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
W20260709 12:23:39.283052  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.253921  5068 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:23:39.298126  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 45053.2us
I20260709 12:23:39.306792  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:39.314910  5188 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:23:39.321302  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 84416.2us
I20260709 12:23:39.322214  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:39.259586  6348 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:39.266674  6409 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:39.265563  6550 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.317202  5144 raft_consensus.cc:2393] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:39.299431  6687 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.268206  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:39.300758  6689 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:23:39.346100  6721 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:23:39.351722  6702 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.352949  6642 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:39.353367  5220 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:23:39.354794  6722 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:23:39.355141  6679 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:39.255154  6516 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.320638  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:39.306998  6717 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:39.326532  6732 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:39.362195  6581 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:39.362954  6581 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.363425  6581 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:39.288630  6701 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:39.346062  6609 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.346350  6729 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:39.347002  6715 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:23:39.269098  6567 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.319335  6269 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:39.364549  5945 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
W20260709 12:23:39.373499  6301 consensus_meta.cc:287] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.522s	user 0.015s	sys 0.000s
I20260709 12:23:39.382242  5141 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:23:39.381289  5940 consensus_meta.cc:287] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.002s
I20260709 12:23:39.385957  6609 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.378079  6559 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:39.399248  6559 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.398344  6609 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.390352  6567 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.393735  6438 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.333020  6348 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:39.406069  6438 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:39.407269  5222 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35624 (request call id 766) took 1338 ms (client timeout 1603 ms). Trace:
I20260709 12:23:39.398289  6299 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.404348  6559 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.369813  6689 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:23:39.427309  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:39.427924  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.391258  6650 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:39.407737  5222 rpcz_store.cc:269] 0709 12:23:38.068935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.069551 (+   616us) service_pool.cc:224] Handling call
0709 12:23:39.407101 (+1337550us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54002d8f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:39.407209 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935390464}
I20260709 12:23:39.378722  6269 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.408862  5940 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.414644  6701 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:39.414662  5067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 195) took 1468 ms. Trace:
I20260709 12:23:39.338631  6550 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.418282  6642 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.418987  6564 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:39.422384  6348 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:39.424314  6725 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.425323  5041 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:39.319450  6692 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:39.408221  6438 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.439311  5222 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:39.435207  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:39.440418  6269 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:39.449204  6581 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.461158  6725 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.474800  6581 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:39.475627  6725 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.480093  5899 leader_election.cc:400] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4.
I20260709 12:23:39.480541  5899 leader_election.cc:403] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:23:39.480439  6559 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:39.481844  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 400) took 1489 ms. Trace:
I20260709 12:23:39.481694  6689 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:39.482434  5943 rpcz_store.cc:276] 0709 12:23:37.992576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.013941 (+ 21365us) service_pool.cc:224] Handling call
0709 12:23:38.101966 (+ 88025us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:38.303988 (+202022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:38.311210 (+  7222us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:38.431519 (+120309us) log.cc:844] Serialized 26 byte log entry
0709 12:23:38.664877 (+233358us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:38.667102 (+  2225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.481402 (+814300us) spinlock_profiling.cc:243] Waited 87.7 ms on lock 0x7b5400438854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:39.481456 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.481502 (+    46us) raft_consensus.cc:1685] finished
0709 12:23:39.481521 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.481779 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10653,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":24956,"spinlock_wait_cycles":201771264,"thread_start_us":158959,"threads_started":1,"wal-append.queue_time_us":259805}
I20260709 12:23:39.410336  6301 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:39.454367  5942 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 3. Ops: []
I20260709 12:23:39.487421  6301 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.488503  6609 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:39.445308  5067 rpcz_store.cc:276] 0709 12:23:37.945960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.946111 (+   151us) service_pool.cc:224] Handling call
0709 12:23:37.946503 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:38.724449 (+777946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.414323 (+689874us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b5400196254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:39.414569 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574966528}
I20260709 12:23:39.492810  6550 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:39.494197  6299 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.494765  6299 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.494688  6559 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4.
I20260709 12:23:39.495215  6438 raft_consensus.cc:3055] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:39.497498  6299 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.497907  6438 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:39.376721  6729 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.502295  6438 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:39.503718  6438 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:39.505930  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:39.506968  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:39.507230  5899 leader_election.cc:400] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:39.440855  6573 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:23:39.507675  5146 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:39.513332  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 88142.1us
I20260709 12:23:39.514483  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:39.507650  5899 leader_election.cc:403] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:23:39.526515  5041 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:39.527427  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:39.528026  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.529120  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:39.535002  6687 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.455744  6564 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.522677  6739 raft_consensus.cc:3055] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:39.537518  6564 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.537787  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:39.547282  5144 raft_consensus.cc:2393] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:39.548285  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:39.539129  5142 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:23:39.550652  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:39.505335  6729 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:39.510399  6498 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.553646  5899 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:23:39.555898  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.559320  5041 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:39.452090  6642 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.645077  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:39.577811  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:39.656329  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 339932us
I20260709 12:23:39.544310  6651 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:39.525705  6717 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:23:39.657023  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 334876us
I20260709 12:23:39.628755  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:39.645291  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:39.551255  6609 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:39.668242  6574 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:23:39.669132  6556 consensus_meta.cc:287] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.003s
I20260709 12:23:39.631817  6650 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.670286  6642 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.542862  6701 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.183s
I20260709 12:23:39.673358  5215 raft_consensus.cc:2410] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:39.674250  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 729) took 3066 ms. Trace:
I20260709 12:23:39.676277  6687 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.654771  6582 raft_consensus.cc:3055] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:39.651733  6390 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.653059  6649 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:39.641826  6693 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.684080  5940 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:39.630854  6682 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.687685  6700 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.670572  6392 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.632321  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:39.602025  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 44051.6us
W20260709 12:23:39.695935  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 136761us
I20260709 12:23:39.571393  5146 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
W20260709 12:23:39.708706  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 155298us
I20260709 12:23:39.714062  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:39.714712  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:39.715858  5071 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:39.715747  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.644960  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:39.552341  6581 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:39.600039  5216 consensus_meta.cc:287] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.004s
I20260709 12:23:39.552769  6299 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:39.538079  6739 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:39.573820  6732 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:39.569845  6348 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:39.578751  6301 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:39.611654  6702 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:39.571786  6725 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:39.563299  5144 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:39.750165  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.751355  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.722437  6654 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:39.707612  6742 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.726049  6349 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:39.728040  6609 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.729861  6440 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.728407  6574 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:39.727857  6717 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:39.744676  5148 rpcz_store.cc:276] 0709 12:23:36.607872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.930901 (+1323029us) service_pool.cc:224] Handling call
0709 12:23:37.960084 (+ 29183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:38.789353 (+829269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.673902 (+884549us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b5400185954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:39.674152 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1795686272}
I20260709 12:23:39.764854  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:39870 (request call id 318) took 1826 ms. Trace:
I20260709 12:23:39.769189  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.767899  5899 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:39.768555  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:39.775722  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.756644  6499 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.744972  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.779924  5220 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:39.780289  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:39.822297  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:39.820843  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:39.823531  5221 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:39.780606  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:39.781991  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:39.825783  5218 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:39.726591  6550 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:23:39.842000  6739 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.842969  5947 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:23:39.843314  6739 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:39.845918  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 78133.9us
I20260709 12:23:39.847622  6299 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:39.848116  5899 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:39.708334  6498 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.756235  6582 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:39.778124  5942 raft_consensus.cc:2410] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:23:39.799438  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.851095  6498 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:39.851254  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.807559  5072 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:39.811821  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.815043  5071 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:39.806381  6725 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.787340  6390 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.816504  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.854637  6725 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:39.817819  5076 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:39.811885  5940 rpcz_store.cc:276] 0709 12:23:37.934418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.951031 (+ 16613us) service_pool.cc:224] Handling call
0709 12:23:39.682545 (+1731514us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b54002d78d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a8532a3d 00007fd8ad502cd0 00007fd8a6a87254 00007fd8a6a871e2 00007fd8a6a87111 00007fd8a6a8708c 00007fd8a6a86342 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:39.760964 (+ 78419us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811033984}
I20260709 12:23:39.759578  6687 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.855593  6582 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.856951  6582 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:39.787787  5216 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:39.858029  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 153) took 1060 ms. Trace:
I20260709 12:23:39.858091  6687 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.858420  5216 rpcz_store.cc:276] 0709 12:23:38.797152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.807479 (+ 10327us) service_pool.cc:224] Handling call
0709 12:23:39.857951 (+1050472us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:39.811257  6702 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:39.860353  6724 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.643s	user 0.002s	sys 0.000s
I20260709 12:23:39.860710  6550 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:39.861044  5076 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:39.810619  6649 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:23:39.861045  5145 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:39.862604  6649 raft_consensus.cc:3055] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:39.862993  6649 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:39.863497  5899 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:39.759938  6269 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.863839  6649 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.864948  6649 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:23:39.865242  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:39.866050  6299 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:39.831243  5943 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:39.866639  6299 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.866959  5067 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:39.816015  6609 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:39.803525  6301 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:39.820809  6742 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:39.868058  6301 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:39.869853  6742 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:39.869928  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:39.869616  6717 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.870667  5070 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:39.817517  6192 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.872515  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:39.873870  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:39.821142  6440 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.838579  6574 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.839588  6499 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.864348  6564 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:39.883044  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:39.833163  5948 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:23:39.883741  5948 pending_rounds.cc:85] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 1
I20260709 12:23:39.883997  5948 pending_rounds.cc:107] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:23:39.884229  5948 raft_consensus.cc:2889] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:39.867069  6299 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:39.865988  6269 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:39.867865  5223 raft_consensus.cc:2368] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:39.834820  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:39.888149  5946 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 1.
I20260709 12:23:39.817251  6349 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:39.875006  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.877408  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:39.777877  6534 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:23:39.893316  5115 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:39.816582  6689 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.867002  6682 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:39.908262  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 696) took 1015 ms. Trace:
I20260709 12:23:39.909780  5217 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:39.886838  6687 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.878377  6702 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:39.910509  6687 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:39.911099  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:39.899175  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:39.912844  6609 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.872766  6192 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:39.879688  6742 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:39.924075  6581 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:23:39.967357  6742 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:39.908819  5149 rpcz_store.cc:276] 0709 12:23:38.893256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.350559 (+457303us) service_pool.cc:224] Handling call
0709 12:23:39.533047 (+182488us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:39.533302 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:39.533780 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:39.533809 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:39.534361 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:23:39.738421 (+204060us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:39.738485 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.738591 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.907829 (+169238us) raft_consensus.cc:1685] finished
0709 12:23:39.907851 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.908167 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1935,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":123,"spinlock_wait_cycles":25667968,"thread_start_us":203598,"threads_started":1,"wal-append.queue_time_us":287159}
I20260709 12:23:39.970278  6559 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:39.970959  6559 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:39.975409  6742 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:39.881716  6654 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:39.978675  6654 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:39.913959  5071 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:23:39.984802  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 91368.7us
I20260709 12:23:39.915545  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:39.922832  5940 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:39.926119  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:39.778393  6700 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:39.925601  5070 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
W20260709 12:23:39.899933  6723 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.010s	sys 0.024s
I20260709 12:23:39.934921  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:39.990839  5948 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
W20260709 12:23:39.950168  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.992202  5223 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:23:39.992959  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:39.951845  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.993739  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:39.996778  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:39.953353  6750 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.149s	user 0.002s	sys 0.001s
I20260709 12:23:39.963954  5072 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:23:40.019469  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 93454.1us
I20260709 12:23:39.995548  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:40.020908  5218 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:39.879649  6745 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:39.892586  6349 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.914676  6574 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:39.912863  6687 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:40.018872  6723 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.010s	sys 0.024s
I20260709 12:23:39.926894  6732 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.927129  6702 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:23:39.913911  6573 consensus_queue.cc:1175] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9847a7797ddc4df689a1ab0911af8785" responder_term: 7 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: [6.3-6.3]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:40.091459  5947 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
W20260709 12:23:40.098153  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.075495  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:40.105824  5940 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:40.106932  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:40.107829  5947 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:23:40.102854  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 72749us
I20260709 12:23:40.117744  6581 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:23:40.104691  6711 log.cc:927] Time spent T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:23:40.130805  5940 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:39.923507  6689 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:40.132181  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:40.038825  6534 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 12:23:40.077934  6729 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:40.046134  6507 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.078s
I20260709 12:23:40.102880  6720 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:40.088935  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:40.046458  6742 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 12:23:40.135221  5940 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:40.103374  6687 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:40.119773  6683 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:40.118497  6440 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:40.131037  5944 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:40.166656  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:40.167428  5944 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:23:40.180932  5899 leader_election.cc:400] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:40.181701  5899 leader_election.cc:403] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:40.136945  6760 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:40.190295  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:40.094327  6745 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:40.150308  6720 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:40.203053  6754 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:23:40.203334  6720 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:40.207823  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:40.208606  5940 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:40.213716  5189 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:40.161110  5216 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:40.104626  6732 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:40.221897  6499 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:40.161974  6192 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:40.149906  6718 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.067s
W20260709 12:23:40.218605  6752 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:23:40.218787  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:40.220148  6700 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:40.212425  6392 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:40.149111  6729 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:40.248979  6729 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:40.229651  5219 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:40.185640  6564 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:40.260094  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 163) took 1119 ms. Trace:
I20260709 12:23:40.260223  6446 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:40.260527  6564 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:40.230837  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:40.261240  6446 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:40.263150  6564 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:40.207523  6557 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:40.249118  6311 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:40.263935  6446 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:40.177147  6687 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:40.265558  6689 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:40.250324  5224 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:40.258612  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:40.267287  5224 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:40.272310  6687 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:40.274369  6745 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:40.284286  6590 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:40.246219  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 65534.2us
I20260709 12:23:40.233449  6269 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:23:40.231860  6725 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:40.289897  6732 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:40.277633  6609 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:40.287329  6446 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.318301  6446 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:40.293743  5945 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:40.312717  6559 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:40.324043  5945 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:23:40.316702  6567 raft_consensus.cc:3055] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:40.277904  6301 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:40.335857  6725 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:40.335875  6567 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:40.338008  6301 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:40.337970  6567 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:40.342418  6567 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:40.342556  6446 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:40.347334  6306 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.348205  6306 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:40.352129  6306 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:40.331350  6299 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:40.325832  6720 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:40.351902  6349 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:40.356078  6720 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:40.399151  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.267594  5215 rpcz_store.cc:276] 0709 12:23:39.141236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.290420 (+149184us) service_pool.cc:224] Handling call
0709 12:23:40.259741 (+969321us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b5400553a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:40.259972 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722928384}
I20260709 12:23:40.416482  5222 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:40.326376  5945 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:40.418787  5941 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
W20260709 12:23:40.420994  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.318768  5943 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:40.302707  5944 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
W20260709 12:23:40.469514  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.470927  6349 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:40.474597  6590 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:40.475957  6590 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:40.477701  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 123421us
I20260709 12:23:40.491556  6582 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:40.502075  6306 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:40.506495  6348 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:40.516376  6609 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:40.562247  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 348483us
I20260709 12:23:40.574262  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:40.574080  6609 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:40.582582  6609 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:40.595494  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.537638  6720 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:40.597638  6720 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:40.600790  6574 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.602316  6574 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:40.608749  6720 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:40.609730  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.625762  5940 raft_consensus.cc:2393] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:40.628858  5942 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:40.632473  5224 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:23:40.633750  6640 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:23:40.635980  6299 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:40.637363  6572 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.637718  5944 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:40.638089  6572 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:40.639263  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.640509  5074 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:40.643349  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:40.643677  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.644007  6572 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:40.644927  5067 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:40.645784  6306 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.645906  6640 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.647908  6306 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:40.648099  6640 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:40.650395  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 86634.4us
I20260709 12:23:40.651793  6306 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:40.651679  6640 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:40.654888  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:40.655643  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:40.656411  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.657297  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:40.658308  5076 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:40.660547  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.663627  6550 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.664171  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:40.664592  6550 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:40.665580  6348 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:40.666170  6348 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.666656  6348 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:40.667147  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.668762  6550 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:40.669597  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:40.670246  5899 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.68:43449 timed out after 1.664s (SENT) [suppressed 7 similar messages]
I20260709 12:23:40.672703  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.673058  6574 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:40.674037  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.675930  5075 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:40.671341  5067 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:40.679419  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.680267  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.654495  6640 consensus_queue.cc:1059] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:40.646868  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.641208  6590 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:40.686612  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.680377  5067 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:40.713860  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.690284  6574 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:40.723500  6572 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.724105  5075 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:40.714761  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.712697  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 771) took 2593 ms. Trace:
I20260709 12:23:40.724591  6572 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:40.731334  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:40.731837  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.732671  5075 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:40.734318  6696 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:40.734373  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 80807.1us
I20260709 12:23:40.735436  6696 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:40.743428  5041 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:40.744521  6311 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.745518  6311 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:40.745978  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.747159  5070 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
W20260709 12:23:40.752229  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.748881  6311 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:40.723683  6574 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:40.760962  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.764508  5076 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:40.779382  5069 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 3: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:40.729393  5224 rpcz_store.cc:276] 0709 12:23:38.090258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.090453 (+   195us) service_pool.cc:224] Handling call
0709 12:23:38.090864 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:38.521649 (+430785us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54003401a0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a84b2dc0 00007fd8a853d1c0 00007fd8a853e231 00007fd8a853ad8f 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c
0709 12:23:40.249855 (+1728206us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54003401a0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a84b906f 00007fd8a8531f60 00007fd8a85337de 00007fd8a853db49 00007fd8a853e416 00007fd8a853ad8f 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62
0709 12:23:40.633179 (+383324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:40.683290 (+ 50111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":960894080}
I20260709 12:23:40.787833  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:40.789003  5941 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:23:40.789937  6696 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:40.791460  6306 raft_consensus.cc:3055] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:40.791940  6306 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:40.792675  5075 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:40.793726  6640 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.794536  6640 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:40.797049  6640 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:40.798038  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:40.800293  5940 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:23:40.800328  5946 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.018s	sys 0.077s
W20260709 12:23:40.801304  5946 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.019s	sys 0.077s
W20260709 12:23:40.804018  5947 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.022s	sys 0.005s
I20260709 12:23:40.785310  6720 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:40.812487  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:40.782051  6348 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:40.844635  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:40.845747  5076 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:23:40.847342  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.824430  6720 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:40.847998  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:40.836563  5947 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.022s	sys 0.006s
W20260709 12:23:40.855588  6688 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.002s	sys 0.008s
W20260709 12:23:40.856319  5221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.000s	sys 0.028s
W20260709 12:23:40.856453  6688 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.003s	sys 0.009s
W20260709 12:23:40.856796  5221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.000s	sys 0.028s
W20260709 12:23:40.857710  5948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.020s	sys 0.027s
W20260709 12:23:40.857914  6654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.009s	sys 0.016s
W20260709 12:23:40.858158  5948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.020s	sys 0.027s
W20260709 12:23:40.858322  6654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.010s	sys 0.016s
W20260709 12:23:40.859313  5073 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.018s	sys 0.044s
W20260709 12:23:40.859754  5073 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.018s	sys 0.044s
W20260709 12:23:40.855441  6650 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.013s	sys 0.021s
I20260709 12:23:40.859814  6654 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:40.855698  6701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.081s	sys 0.134s
I20260709 12:23:40.850843  6720 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:40.793462  6306 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:40.868350  6701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.081s	sys 0.135s
W20260709 12:23:40.860711  6650 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.013s	sys 0.022s
I20260709 12:23:40.887866  6640 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:40.888767  6640 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:40.872776  6438 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.013s	sys 0.012s
W20260709 12:23:40.894862  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 105818us
I20260709 12:23:40.887852  5898 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:40.892278  6696 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.905843  6696 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:40.905367  6521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.002s	sys 0.022s
I20260709 12:23:40.887956  6550 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:40.906255  6409 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.000s	sys 0.023s
W20260709 12:23:40.907096  6521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.002s	sys 0.023s
I20260709 12:23:40.868335  6654 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:40.917655  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.900561  6572 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:40.908317  6438 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.013s	sys 0.013s
W20260709 12:23:40.920424  6409 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.000s	sys 0.025s
I20260709 12:23:40.918107  6574 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:40.933688  6590 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.935173  6582 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:23:40.959090  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 197638us
I20260709 12:23:40.938001  6550 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:40.957199  6582 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.964690  5898 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:40.967128  6348 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:40.982594  6654 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:40.984747  6654 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:40.961705  6574 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:23:40.986234  6696 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:40.987396  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:40.989456  6550 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:40.989682  6306 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.004576  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 186) took 1499 ms. Trace:
I20260709 12:23:41.025754  5141 rpcz_store.cc:276] 0709 12:23:39.504480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.504621 (+   141us) service_pool.cc:224] Handling call
0709 12:23:39.505062 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:39.505509 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:39.505717 (+   208us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:40.118904 (+613187us) spinlock_profiling.cc:243] Waited 6.25 ms on lock 0x7b2c00193f58. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:23:40.171110 (+ 52206us) log.cc:844] Serialized 26 byte log entry
0709 12:23:40.884722 (+713612us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:40.900388 (+ 15666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:40.919483 (+ 19095us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:41.003958 (+ 84475us) raft_consensus.cc:1685] finished
0709 12:23:41.003987 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:41.004306 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":687,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"spinlock_wait_cycles":14371200,"thread_start_us":370634,"threads_started":1,"wal-append.queue_time_us":725829}
I20260709 12:23:41.028795  6609 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:41.005234  6640 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:41.007364  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:40.967159  6701 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.033399  6701 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:41.034116  6687 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:41.036103  6701 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:40.980568  6765 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.161s	user 0.002s	sys 0.000s
W20260709 12:23:41.013262  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 61143.3us
W20260709 12:23:41.030977  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 97870.9us
I20260709 12:23:40.978809  6582 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:41.029430  6609 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:41.053459  6717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.083s	user 0.010s	sys 0.012s
W20260709 12:23:41.050727  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 380525us
I20260709 12:23:41.015169  6590 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:41.050431  6574 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:41.066373  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:41.065056  6717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.190s	user 0.011s	sys 0.012s
I20260709 12:23:41.056452  6609 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:41.067783  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:41.063102  6550 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:23:41.078809  6757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.010s	sys 0.017s
I20260709 12:23:41.068760  6574 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:23:41.105369  6757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.011s	sys 0.017s
I20260709 12:23:41.098285  6550 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:41.108410  6521 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:41.148247  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 140968us
W20260709 12:23:41.137495  6779 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.054s	user 0.001s	sys 0.002s
W20260709 12:23:41.150254  5115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.049s	user 0.065s	sys 0.140s
I20260709 12:23:41.141883  6299 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:41.192562  5115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.172s	user 0.065s	sys 0.141s
I20260709 12:23:41.194648  5070 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:41.124401  6521 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:41.202422  6567 consensus_meta.cc:287] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.815s	user 0.018s	sys 0.006s
W20260709 12:23:41.242750  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.197265  5941 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
W20260709 12:23:41.224143  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 1.20339e+06us
W20260709 12:23:41.234295  6780 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:23:41.233392  6349 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.226454  6299 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:41.268615  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:41.259756  6590 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:41.279186  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 406) took 1289 ms. Trace:
W20260709 12:23:41.295370  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 82585.2us
I20260709 12:23:41.261214  6720 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:41.298189  6567 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:23:41.299460  6783 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 12:23:41.301107  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 98195.5us
W20260709 12:23:41.302554  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.303740  6609 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:41.308182  6640 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:41.290952  5946 rpcz_store.cc:276] 0709 12:23:39.981375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.981696 (+   321us) service_pool.cc:224] Handling call
0709 12:23:39.982160 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:39.982536 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:23:40.802448 (+819912us) log.cc:844] Serialized 14 byte log entry
0709 12:23:41.191735 (+389287us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:41.198162 (+  6427us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:41.239722 (+ 41560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:41.257000 (+ 17278us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:41.267862 (+ 10862us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47519,"prepare.queue_time_us":169643,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":19444,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":12888,"thread_start_us":1073232,"threads_started":2}
I20260709 12:23:41.266783  6574 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:41.289973  6550 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:41.309303  6696 consensus_queue.cc:1059] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:41.319669  6777 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:23:41.329736  5076 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:41.317667  6567 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:41.297139  6348 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:41.346282  6574 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:41.331813  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 785) took 1151 ms. Trace:
I20260709 12:23:41.347074  5221 rpcz_store.cc:276] 0709 12:23:40.172475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.172697 (+   222us) service_pool.cc:224] Handling call
0709 12:23:40.173146 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:40.174098 (+   952us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:40.174127 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:40.174619 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:23:40.857452 (+682833us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:40.857596 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:40.857731 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:41.312575 (+454844us) raft_consensus.cc:1685] finished
0709 12:23:41.312602 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:41.323332 (+ 10730us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4463,"raft.queue_time_us":730,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":16501,"thread_start_us":682324,"threads_started":1,"wal-append.queue_time_us":879036}
I20260709 12:23:41.349664  6567 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:41.351382  6609 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:41.353750  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:41.322392  6732 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:41.355867  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:41.356144  6732 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:41.357348  6732 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:41.358728  6521 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:41.310966  6349 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:41.362404  6590 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:41.363492  6590 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:41.341804  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:41.366035  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:41.366314  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 230) took 1455 ms. Trace:
I20260709 12:23:41.366744  5073 rpcz_store.cc:276] 0709 12:23:39.910895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.914049 (+  3154us) service_pool.cc:224] Handling call
0709 12:23:39.914353 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:39.914582 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:23:40.860254 (+945672us) log.cc:844] Serialized 14 byte log entry
0709 12:23:41.312063 (+451809us) log.cc:844] Serialized 14 byte log entry
0709 12:23:41.316401 (+  4338us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:41.322554 (+  6153us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:41.322611 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:41.330492 (+  7881us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:41.366226 (+ 35734us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68911,"prepare.queue_time_us":220481,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":3961,"raft.queue_time_us":7717,"raft.run_cpu_time_us":478,"raft.run_wall_time_us":24836,"thread_start_us":1164806,"threads_started":2}
W20260709 12:23:41.323371  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 436343us
I20260709 12:23:41.371119  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 781) took 1380 ms. Trace:
I20260709 12:23:41.371649  5217 rpcz_store.cc:276] 0709 12:23:39.990599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.019633 (+ 29034us) service_pool.cc:224] Handling call
0709 12:23:40.264443 (+244810us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:40.264718 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:40.464218 (+199500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:40.464311 (+    93us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:40.465702 (+  1391us) log.cc:844] Serialized 26 byte log entry
0709 12:23:40.962099 (+496397us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:41.032902 (+ 70803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:41.063183 (+ 30281us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:41.370561 (+307378us) raft_consensus.cc:1685] finished
0709 12:23:41.370600 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:41.370901 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":172077,"raft.queue_time_us":208608,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":385,"spinlock_wait_cycles":45405568,"thread_start_us":483050,"threads_started":1,"wal-append.queue_time_us":745985}
I20260709 12:23:41.345990  6348 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:41.374084  6720 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:41.374720  6696 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:41.364030  6590 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:41.366060  5115 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:41.390894  6521 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.349400  6582 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:41.401736  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.422991  6649 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:41.423437  6521 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:41.423776  6649 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:41.395639  6696 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:41.402657  6609 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:41.435017  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.430646  6572 raft_consensus.cc:3055] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:41.430950  6717 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:41.460522  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 413) took 1346 ms. Trace:
W20260709 12:23:41.459982  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 501864us
W20260709 12:23:41.468881  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:41.469734  6438 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.013s	sys 0.026s
W20260709 12:23:41.482494  6438 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.013s	sys 0.026s
I20260709 12:23:41.488502  6609 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:41.489291  6609 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:41.454123  6572 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:41.484156  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:41.514604  6306 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:41.517766  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 209257us
I20260709 12:23:41.462698  5947 rpcz_store.cc:276] 0709 12:23:40.113960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.114278 (+   318us) service_pool.cc:224] Handling call
0709 12:23:40.114692 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:40.114988 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:23:40.899210 (+784222us) log.cc:844] Serialized 14 byte log entry
0709 12:23:41.419580 (+520370us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:41.432857 (+ 13277us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:41.441220 (+  8363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:41.451225 (+ 10005us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:41.460440 (+  9215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17008,"prepare.queue_time_us":79316,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"thread_start_us":1174746,"threads_started":2}
I20260709 12:23:41.492919  6649 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:41.520642  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:41.524960  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 167562us
I20260709 12:23:41.518375  6609 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:41.498950  6717 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:41.544658  6701 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:41.523239  6696 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:41.526778  6348 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:41.542544  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 82353.9us
I20260709 12:23:41.556898  5899 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:41.565737  6766 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:41.555953  6582 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:41.524869  6590 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:41.557898  6609 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.566090  6799 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.573041  6299 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:41.565903  6801 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:41.589846  6650 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:41.590294  6803 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.586566  6701 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:23:41.590795  6766 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:41.592640  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 226738us
I20260709 12:23:41.607306  6807 consensus_queue.cc:1059] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:41.618481  6732 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:41.619199  5188 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:41.624377  6299 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:41.610385  6650 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:41.630044  6650 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:41.625269  6299 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:41.622380  6609 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:41.643899  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:41.642843  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:41.635205  6688 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:41.645299  5947 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:41.646330  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:41.648613  6804 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.649123  5944 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:41.649449  6804 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:41.650333  5149 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:41.629944  6701 consensus_queue.cc:1243] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:23:41.611224  6572 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:41.637820  6582 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:41.648881  5941 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:41.652644  6649 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:41.642832  6574 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:41.662112  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:41.658020  6808 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:23:41.621821  6803 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:41.660167  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 126019us
I20260709 12:23:41.626920  6799 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:41.647181  6717 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:41.651134  6688 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:41.667526  6696 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:41.668816  5947 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
W20260709 12:23:41.674556  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.631350  6801 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:41.668160  6649 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:41.683907  6649 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:41.679493  6574 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:41.660578  6807 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:23:41.687252  6572 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:41.677134  6688 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:41.689332  6438 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.689980  6438 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:41.691762  5150 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:23:41.675621  6582 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:41.696709  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.698906  6299 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:41.699601  5149 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:41.702718  6799 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:41.702458  6567 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:41.708315  6567 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:41.710507  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:41.711053  5944 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:41.714483  5148 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:23:41.713016  6806 raft_consensus.cc:3055] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:41.719408  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:41.715523  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:41.721995  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:41.722340  5142 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:41.723170  6521 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:41.705605  6803 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:41.714959  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:41.729954  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:41.678773  6696 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:41.753446  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 109618us
I20260709 12:23:41.719703  5144 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:41.723652  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:41.759130  5941 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:41.762095  6311 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:23:41.763070  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.753794  6649 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:41.764743  6311 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:41.765138  6649 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:41.766515  6804 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:41.768288  6649 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:41.747148  6306 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:41.740320  6806 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:41.771119  6702 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:41.779886  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:41.787365  6702 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:41.787596  6609 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:41.787998  5142 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:41.790511  6582 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:41.788703  6702 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:41.753939  6799 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:41.791900  6799 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:41.765338  6311 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:41.774075  6574 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:41.776319  5946 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:23:41.778098  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:41.786768  5150 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:41.779388  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:41.778762  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:41.798023  6702 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:41.798604  5939 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:41.757668  6696 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:41.790292  6806 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:41.807644  6806 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:41.808349  6766 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:41.777324  5940 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:41.810657  5149 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:41.785769  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:41.819674  5947 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
W20260709 12:23:41.824793  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 147993us
I20260709 12:23:41.818725  5188 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:23:41.830870  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.833477  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:41.834820  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 77995.3us
I20260709 12:23:41.802078  6438 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:41.843389  5946 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:41.840703  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:41.774497  6797 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:41.871866  6797 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:41.872846  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:41.873126  6799 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:41.856389  6306 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:41.862305  6801 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:41.864492  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:41.865736  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 784) took 1815 ms. Trace:
I20260709 12:23:41.868651  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:41.844156  6717 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:41.840358  6521 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:41.830035  6349 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:41.852056  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.898324  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:41.875967  5219 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:23:41.874025  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:41.874361  6797 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:41.887313  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 44545us
I20260709 12:23:41.889396  5149 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:41.900620  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:41.905336  5075 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:41.899353  5219 pending_rounds.cc:85] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 2
I20260709 12:23:41.910238  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:41.888178  6306 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:41.924113  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:41.918056  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:41.947140  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.913595  5219 pending_rounds.cc:107] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:23:41.888654  5218 rpcz_store.cc:276] 0709 12:23:40.029098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.029243 (+   145us) service_pool.cc:224] Handling call
0709 12:23:40.029657 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:40.030096 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:23:40.564160 (+534064us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b180024af00. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8abf333c7 00007fd8a85459a7 00007fd8a85550bf 00007fd8a8555038
0709 12:23:40.931174 (+367014us) log.cc:844] Serialized 14 byte log entry
0709 12:23:41.634681 (+703507us) log.cc:844] Serialized 14 byte log entry
0709 12:23:41.653251 (+ 18570us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:41.735650 (+ 82399us) log.cc:844] Serialized 26 byte log entry
0709 12:23:41.740140 (+  4490us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:41.740184 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:41.756916 (+ 16732us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:41.829859 (+ 72943us) raft_consensus.cc:1685] finished
0709 12:23:41.829887 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:41.844050 (+ 14163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8755,"prepare.queue_time_us":880702,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":103,"raft.queue_time_us":178359,"raft.run_cpu_time_us":683,"raft.run_wall_time_us":53363,"spinlock_wait_cycles":6549376,"thread_start_us":1031877,"threads_started":3,"wal-append.queue_time_us":572259}
I20260709 12:23:41.961426  5219 raft_consensus.cc:2889] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:41.973798  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.974110  5149 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:41.976176  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:41.980405  4898 catalog_manager.cc:5697] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 3 to 5, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 5 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:41.981704  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:41.982012  6438 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:41.982585  6438 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:41.984838  6438 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:41.908883  6521 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:41.986285  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:41.986502  6521 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:41.982604  5220 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:41.899709  5944 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:41.987022  5144 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:41.988505  5041 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:41.989899  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.992244  5142 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:42.002197  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.002628  6311 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:42.003239  6816 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.003924  6816 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:42.004616  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.004937  6311 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:42.005960  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.006233  5075 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:41.923782  6306 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:42.014621  5041 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:42.018116  5221 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:42.021252  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:42.026899  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.992611  6590 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.988760  6812 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:42.017313  6574 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:41.978066  6717 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:42.022861  6311 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.059139  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:42.060346  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.060848  5070 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:42.041220  6799 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:42.070844  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.071659  5142 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:42.021458  6650 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:42.048347  6574 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:42.079346  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.067170  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.091102  5144 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:42.045053  6720 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:23:42.072940  6799 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.047514  6590 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:42.091609  6814 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:23:42.072729  6311 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:42.113758  5945 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:42.113924  6804 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.113984  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.114459  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.116191  6590 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:42.089955  6574 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:42.084297  6567 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.120584  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.125844  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.138765  5142 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:42.128187  5069 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:42.107800  6799 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:42.141852  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:42.145713  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:42.061769  6812 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:42.153712  6348 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:42.154295  6348 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.154733  6348 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:42.154970  6825 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:23:42.155557  6812 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:42.114174  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:42.156366  6817 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:23:42.156566  5068 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:42.157704  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.157814  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.112352  6816 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:42.158619  5067 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:42.159184  5223 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:42.155615  6804 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:42.161556  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.164805  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.166738  6804 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:42.166991  6717 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:42.119029  6688 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:42.149325  5220 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:42.167855  6717 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:42.168428  6717 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:42.169266  6688 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:42.168888  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.170473  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.171200  5068 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:42.171816  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:42.174563  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.176146  6702 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.176798  6702 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:42.178895  6822 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:23:42.179157  6702 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:42.180513  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.181771  5215 raft_consensus.cc:2410] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:23:42.171968  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.196193  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.198637  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.204859  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:42.206175  6801 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:42.206820  6801 raft_consensus.cc:697] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:42.207517  5220 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
W20260709 12:23:42.208271  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.208451  5114 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:42.208757  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.210474  6311 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:42.209836  4898 catalog_manager.cc:5697] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 3 to 4, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 4 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:23:42.210106  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:42.211516  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 64886.8us
I20260709 12:23:42.215960  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.212186  4898 catalog_manager.cc:5697] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 2 to 3, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
W20260709 12:23:42.207249  4899 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49944 (request call id 76) took 12665 ms (client timeout 15000 ms). Trace:
I20260709 12:23:42.210108  6306 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:42.214030  5216 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:42.214972  6696 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:42.220057  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.221298  5072 raft_consensus.cc:2410] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:23:42.222410  5898 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:42.225008  5070 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:42.226153  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:42.220670  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 185402us
I20260709 12:23:42.226240  5222 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:42.182952  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:42.228542  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.227962  5898 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:42.224367  4898 catalog_manager.cc:5697] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 2 to 3, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 3 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:23:42.245962  6564 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.246632  6564 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:42.247893  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.248991  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.251967  5219 raft_consensus.cc:2393] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:42.145051  6567 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:42.225502  6702 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:42.223948  6804 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.240039  6550 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:42.255357  5041 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:42.229449  5068 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 1.
I20260709 12:23:42.241346  6650 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:42.264572  4898 catalog_manager.cc:5697] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 2 to 3, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 3 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:42.250665  6720 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.264113  6550 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:42.249022  6564 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:42.260878  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.295403  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.296271  6797 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:42.296392  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.297202  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.365170  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.366158  5899 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:42.366688  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.367183  6306 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:42.300041  6829 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:42.284515  6572 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.368297  6829 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.368801  6829 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:42.368842  6572 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:42.302733  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.303766  6311 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.371678  5149 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:42.305455  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.306733  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.307633  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:42.373584  5072 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:42.301648  4898 catalog_manager.cc:5697] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 4, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 4 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:42.374154  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.313277  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.377033  5223 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:42.314164  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.314957  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:42.295409  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.320732  5143 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:42.322065  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.322955  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.386765  5145 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:23:42.329048  5939 consensus_meta.cc:287] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.002s
I20260709 12:23:42.284291  6299 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:23:42.227610  6348 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:42.258436  6582 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:42.358776  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:42.262960  6702 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:42.378334  5224 raft_consensus.cc:2368] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:42.374881  4898 catalog_manager.cc:5697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 2 to 4, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 4 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:42.410657  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.389237  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.392127  5146 raft_consensus.cc:2393] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 2: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:42.373219  5141 raft_consensus.cc:2393] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:42.356983  6550 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:42.400929  5221 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
W20260709 12:23:42.417454  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 85644us
I20260709 12:23:42.268860  6804 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:42.421990  5149 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:42.391820  5939 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
W20260709 12:23:42.448355  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.423269  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.426411  5143 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:42.428372  4898 catalog_manager.cc:5697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 2, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 2 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:23:42.429782  5147 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:42.430876  6567 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:42.432245  6306 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:42.432778  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:42.386590  5067 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:42.434032  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:42.379153  6720 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:42.386313  5144 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:42.371712  6311 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:42.439278  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.439582  6348 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:42.406857  6438 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:42.413646  6702 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:42.441262  6696 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:42.487540  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.443039  5145 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:42.444669  5222 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:42.258054  6801 consensus_queue.cc:237] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:42.456933  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.464924  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.462754  5219 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
W20260709 12:23:42.473481  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 98359.4us
I20260709 12:23:42.473737  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.495803  5147 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
W20260709 12:23:42.475632  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 144556us
I20260709 12:23:42.454887  6306 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:42.217645  4899 rpcz_store.cc:269] 0709 12:23:29.541488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.541779 (+   291us) service_pool.cc:224] Handling call
0709 12:23:42.206995 (+12665216us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:38.406845 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:38.413456 (+  6611us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:38.413525 (+    69us) write_op.cc:435] Acquired schema lock
0709 12:23:38.413589 (+    64us) tablet.cc:662] Decoding operations
0709 12:23:38.582630 (+169041us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:38.629254 (+ 46624us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:38.630808 (+  1554us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:23:38.651600 (+ 20792us) tablet.cc:701] Row locks acquired
0709 12:23:38.651627 (+    27us) write_op.cc:260] PREPARE: finished
0709 12:23:38.651939 (+   312us) write_op.cc:270] Start()
0709 12:23:38.652121 (+   182us) write_op.cc:276] Timestamp: P: 1783599818651797 usec, L: 0
0709 12:23:38.652169 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 12:23:38.653258 (+  1089us) log.cc:844] Serialized 2848 byte log entry
0709 12:23:39.217814 (+564556us) op_driver.cc:464] REPLICATION: finished
0709 12:23:39.734669 (+516855us) spinlock_profiling.cc:243] Waited 9.96 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a54310ac 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8abf71ef0
0709 12:23:40.173658 (+438989us) write_op.cc:301] APPLY: starting
0709 12:23:40.178233 (+  4575us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:40.199339 (+ 21106us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:40.199367 (+    28us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:40.200308 (+   941us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:40.200557 (+   249us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:23:40.200609 (+    52us) write_op.cc:312] APPLY: finished
0709 12:23:40.200916 (+   307us) log.cc:844] Serialized 58 byte log entry
0709 12:23:41.230580 (+1029664us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:41.230840 (+   260us) write_op.cc:454] Released schema lock
0709 12:23:41.230958 (+   118us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:41.231796 (+   838us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:41.231834 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":309396,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":922220,"apply.run_cpu_time_us":31110,"apply.run_wall_time_us":1074463,"num_ops":7,"prepare.queue_time_us":154867,"prepare.run_cpu_time_us":17037,"prepare.run_wall_time_us":290226,"raft.queue_time_us":208835,"raft.run_cpu_time_us":35373,"raft.run_wall_time_us":883297,"replication_time_us":565474,"spinlock_wait_cycles":22909312,"thread_start_us":2002803,"threads_started":5,"wal-append.queue_time_us":1170475,"wal-append.run_cpu_time_us":12574,"wal-append.run_wall_time_us":2432603}]]}
I20260709 12:23:42.421206  6574 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:42.437366  6799 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:42.498368  5143 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:42.493275  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.504227  5074 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:42.504607  5141 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:42.508054  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.510829  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.503515  5146 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:42.514425  5147 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:42.500613  6572 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:42.503046  6306 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:42.497993  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.499843  6717 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:42.500339  5216 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:42.516109  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.512424  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.525344  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.525344  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.535789  5072 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:42.554296  5899 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:42.521090  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.551403  5143 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 5
W20260709 12:23:42.559100  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 303808us
I20260709 12:23:42.562045  5146 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:23:42.565989  5149 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:42.552419  5144 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:42.591737  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.549693  5220 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:42.601989  5147 raft_consensus.cc:2393] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 2: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:42.567266  6582 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:42.614676  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.616927  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:42.621996  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 67822.2us
I20260709 12:23:42.631533  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.637655  5189 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:42.638175  5147 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:42.635454  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:42.646719  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:42.647393  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:42.647677  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:42.648289  6572 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:42.617679  6564 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:42.647850  5944 raft_consensus.cc:2393] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:42.630749  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:42.652937  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.657579  5147 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:42.657349  5899 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:42.648607  5943 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 2: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:42.666369  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.668128  5145 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:42.680784  6717 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:42.681469  6574 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:42.621629  5220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:42.683379  5220 raft_consensus.cc:606] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:42.687541  5041 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:42.688616  5222 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:42.670333  6720 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:42.700906  6720 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:42.701650  6720 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:42.702224  6720 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:42.650655  6806 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:23:42.689774  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:42.708449  5218 raft_consensus.cc:2410] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:23:42.708606  6654 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:42.691250  6840 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:23:42.694720  6834 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:42.695200  6832 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:23:42.697045  6572 consensus_queue.cc:1059] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:42.710391  6572 raft_consensus.cc:3055] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:23:42.710454  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 67584.4us
I20260709 12:23:42.710860  6572 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:42.711866  6572 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:42.713182  6572 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:42.666885  6804 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:42.713423  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:42.724655  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.722949  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.726006  5069 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 1.
I20260709 12:23:42.726365  5899 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:42.707001  5147 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:42.720630  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:42.719215  6834 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.722292  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:42.721403  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.698697  5189 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:42.720039  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.736603  5150 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:23:42.735999  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:42.743892  5899 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:23:42.746141  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.667205  6802 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:42.747941  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.748446  5143 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:42.755206  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.758049  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.709451  6654 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:42.766402  5143 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:42.766371  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.708482  6717 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.768309  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.773862  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.784868  5219 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:42.775256  5944 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:42.777753  5220 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:42.742262  6806 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:42.750828  5067 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:42.779682  6720 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:42.781643  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.739461  6843 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:42.809064  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.779220  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.787071  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:42.779881  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:42.796274  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:42.857019  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.750365  6812 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:42.804605  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.797966  6806 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.802651  5899 leader_election.cc:400] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:42.800717  5150 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:42.868559  5899 leader_election.cc:403] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:42.838570  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.840559  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:42.842417  5220 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:42.750550  6834 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:42.850365  5216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:42.853029  6799 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:42.853518  6654 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:42.854280  6838 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:23:42.854909  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:42.894376  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.855122  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:42.874132  5945 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:42.857786  5948 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:42.859035  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.860496  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.857692  5218 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:42.806304  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:42.864295  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 77367.1us
I20260709 12:23:42.801776  6349 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:42.723340  6582 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:42.874182  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.874974  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.883798  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:42.885636  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.890266  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.894969  5072 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:42.894424  5071 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:42.914294  5150 raft_consensus.cc:2376] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3. Re-sending same reply.
I20260709 12:23:42.794416  6717 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:42.879483  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.832165  6843 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:42.856389  6550 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:42.906278  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:42.932512  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.900334  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:42.914300  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 150225us
I20260709 12:23:42.871647  6797 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:42.934866  5143 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:42.919873  5145 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:42.832751  6702 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:42.921319  5142 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:42.921967  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.938608  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.933926  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:42.939216  5070 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:42.934286  5899 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:42.943204  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.919916  5146 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:42.871305  6806 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:42.898456  6829 consensus_meta.cc:287] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.529s	user 0.015s	sys 0.000s
I20260709 12:23:42.923754  5076 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:42.910930  5217 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:42.901610  6802 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:23:42.965778  6609 consensus_queue.cc:1059] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:42.966868  6609 raft_consensus.cc:3055] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:42.967226  6609 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:42.967686  6834 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:42.912967  6349 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:42.969870  6349 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:42.967448  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 111797us
I20260709 12:23:42.973235  6550 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:42.944442  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:42.950335  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.974085  5940 raft_consensus.cc:2393] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:42.974290  5147 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:42.899977  6311 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:42.954952  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:42.916607  5941 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:42.978288  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:42.921823  6438 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:42.980034  6438 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.980455  6438 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:42.981259  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:42.982013  5074 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:42.983430  6650 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:42.983932  6650 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.984095  6348 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:42.984335  6650 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:42.984635  6348 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.985080  6348 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:42.901274  5216 raft_consensus.cc:606] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:42.987341  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.961220  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.988168  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:42.988719  5150 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:42.988198  5143 raft_consensus.cc:2368] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:42.975600  5075 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:42.951462  5144 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
W20260709 12:23:42.987581  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.955176  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:42.992112  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:42.992600  5141 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:42.996759  5899 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:42.999714  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50946.2us
I20260709 12:23:42.993512  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:43.000636  5188 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:43.001658  5188 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:43.004001  5142 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:43.009795  5067 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:42.931370  6843 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:42.956739  6574 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:43.013353  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:43.016474  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:42.957998  6804 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:43.020059  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:43.021863  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:43.022373  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.034202  5146 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:42.957162  6844 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:43.012607  6311 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:43.049466  5142 raft_consensus.cc:2393] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:42.959564  6654 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
W20260709 12:23:43.047174  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 91064.8us
I20260709 12:23:43.007982  6801 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:23:43.019394  6834 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:43.022554  6299 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:42.996234  6829 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:43.097560  5899 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:43.019163  6696 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:43.035817  6844 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.025300  6688 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:43.020674  6574 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:43.111248  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.169931  6574 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.170447  5142 raft_consensus.cc:2368] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:43.036172  5068 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:43.116159  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.112897  6649 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.061s
W20260709 12:23:43.118872  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 65504.7us
I20260709 12:23:43.174391  6649 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:43.174808  6649 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:43.174817  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:43.175544  5190 leader_election.cc:336] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.516s (SENT)
I20260709 12:23:43.175791  6649 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:43.176151  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:43.177131  6649 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:43.177632  6812 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:43.180099  5189 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:23:43.180737  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.114240  5041 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:43.098867  6797 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:43.182339  6797 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:43.182479  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 82773.2us
I20260709 12:23:43.182883  6797 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:43.049610  6311 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.191234  6311 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.190923  6348 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:43.123543  6564 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:43.126253  6720 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:43.127018  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.108321  5216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:43.194882  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.202100  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:43.203007  5189 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:43.195622  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 174391us
I20260709 12:23:43.196497  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.196930  5148 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
W20260709 12:23:43.208042  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:43.198406  5900 leader_election.cc:336] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.597s (SENT)
I20260709 12:23:43.135885  6802 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:43.234112  5948 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:43.138757  6717 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:43.141664  6609 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:43.147783  6702 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:43.157836  6766 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:43.163602  6829 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:43.164129  6688 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.058568  6834 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:43.165654  6844 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:43.203963  6732 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:43.189713  6299 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:43.189827  6804 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:43.182526  6851 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:43.240428  5947 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:43.242805  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:43.248471  5146 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:43.284605  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.283723  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:43.286480  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.256140  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:43.287602  6688 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.288412  6766 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:43.167544  6696 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:43.314401  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.318608  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.193502  6348 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:43.249424  5216 raft_consensus.cc:606] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:43.340643  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:43.351527  5146 raft_consensus.cc:2393] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:43.353277  5148 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:23:43.343667  6306 consensus_meta.cc:287] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.786s	user 0.015s	sys 0.000s
I20260709 12:23:43.354326  6306 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:43.340268  5145 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:43.312013  6834 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:43.193991  6650 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:43.309589  6812 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:23:43.347244  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 110983us
I20260709 12:23:43.233893  6438 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:43.364029  6306 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:43.365176  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 166814us
I20260709 12:23:43.316855  6732 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:43.367120  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:43.369822  5945 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:43.387036  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.373840  5215 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:43.375209  6348 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:23:43.383466  5900 leader_election.cc:336] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.655s (SENT)
I20260709 12:23:43.338971  6804 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:43.345060  6609 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:43.335997  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:43.338532  6802 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 12:23:43.398797  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 195852us
I20260709 12:23:43.387809  5146 raft_consensus.cc:2376] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6. Re-sending same reply.
I20260709 12:23:43.400022  5189 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:43.392387  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.396494  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:43.401583  6306 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:43.377859  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:43.394196  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:43.401564  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:43.404026  5948 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:43.404116  6306 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:43.404924  6306 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:43.401830  5148 raft_consensus.cc:2376] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5. Re-sending same reply.
I20260709 12:23:43.400944  5145 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
W20260709 12:23:43.408293  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.389158  6732 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.418602  6306 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:43.422675  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:43.426136  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:43.427098  5145 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:23:43.429998  6720 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:43.432466  6720 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:43.432921  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:43.434955  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:43.442548  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 64014.1us
I20260709 12:23:43.443202  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:43.443439  5188 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:43.448226  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:43.450209  5948 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:23:43.491953  5148 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 2: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:43.416704  6564 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:43.458940  6717 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:43.493964  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50621us
I20260709 12:23:43.499202  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:43.500479  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:43.500783  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 57689.5us
I20260709 12:23:43.435461  6654 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:43.447273  6720 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:43.420856  6349 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:43.509235  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.402818  6766 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:43.508638  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:43.510293  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.510607  5146 raft_consensus.cc:2410] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:23:43.511934  5142 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:43.494530  6829 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:43.478427  6806 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:43.558264  5948 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:23:43.556169  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.443899  6696 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:43.571727  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:43.575474  5148 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:23:43.572924  6862 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:23:43.573444  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:43.580189  5222 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:43.582505  5940 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:43.584326  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:43.584620  6564 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:43.585243  5217 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
W20260709 12:23:43.586443  6868 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.075s	user 0.000s	sys 0.002s
I20260709 12:23:43.587273  6877 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:43.558131  6829 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.511631  6843 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:43.594728  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.576236  6863 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:43.614842  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:43.568439  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:43.563453  6873 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:43.617456  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:43.619836  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 119747us
I20260709 12:23:43.621135  5940 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
W20260709 12:23:43.623353  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 54129.5us
I20260709 12:23:43.624368  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:43.627812  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:43.628679  5939 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:43.609552  6829 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:43.610205  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:43.640798  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:43.619432  6863 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.641619  5070 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:43.646490  5222 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:43.624785  6873 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:43.661091  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.623202  6877 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:23:43.667775  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 165535us
I20260709 12:23:43.627940  6349 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:43.636729  6871 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:43.667544  5945 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:43.647403  6863 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:43.636416  6875 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.647066  6572 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:43.665108  6873 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:43.644418  6688 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:43.701100  5221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:43.701901  5221 raft_consensus.cc:606] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:23:43.689891  5942 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:43.706827  5070 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:43.641400  6878 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:23:43.691452  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:43.716398  6688 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:43.714730  6574 consensus_meta.cc:287] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.002s
I20260709 12:23:43.725641  6806 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:43.726526  6877 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:43.727361  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:43.728291  6649 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.004s
I20260709 12:23:43.729555  5947 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:43.751070  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:43.748230  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:43.730876  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:43.741788  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 60122.9us
I20260709 12:23:43.692664  6572 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:43.757494  5041 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:43.752256  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:43.769019  5947 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
W20260709 12:23:43.762648  6797 consensus_meta.cc:287] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.005s
I20260709 12:23:43.691745  6875 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:43.790524  5945 raft_consensus.cc:2368] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5.
I20260709 12:23:43.747466  6732 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:43.747419  6574 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:43.788049  6861 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:43.803216  5942 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:43.815711  5221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:43.814481  6843 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:43.793650  6816 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
W20260709 12:23:43.829957  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 67258.8us
I20260709 12:23:43.821151  5221 raft_consensus.cc:606] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:23:43.832314  5070 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.815320  6801 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:43.822801  6799 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:43.835549  5041 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:43.838663  6882 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:43.839350  6882 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:43.839988  6834 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:43.841626  6812 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:43.842209  6812 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:43.842443  6834 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:43.845944  6801 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:43.802546  6797 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:43.858793  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:43.838701  5041 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:43.863879  5145 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:43.865559  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:43.849282  6801 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:43.878445  6834 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:43.878955  6834 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:43.879231  6887 log.cc:927] Time spent T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:23:43.879355  6834 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:43.882581  6572 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:43.883028  6438 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:43.884096  6871 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:43.854760  6720 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:43.887907  5070 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:43.865427  6799 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:23:43.878963  6888 log.cc:927] Time spent T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:23:43.884500  6871 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:43.896219  5068 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:43.915792  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:43.922863  6311 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.731s	user 0.009s	sys 0.005s
I20260709 12:23:43.926582  6797 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:43.941581  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:43.932783  6861 raft_consensus.cc:697] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:43.840536  6882 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:43.935279  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:43.939193  6609 consensus_meta.cc:287] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.536s	user 0.004s	sys 0.010s
W20260709 12:23:43.949234  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.951421  5941 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:43.971678  5945 raft_consensus.cc:2393] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:43.935966  6311 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:43.972052  5943 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:43.947433  6893 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:23:43.988576  6883 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.199s	user 0.002s	sys 0.000s
I20260709 12:23:43.989159  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35624 (request call id 792) took 1544 ms. Trace:
I20260709 12:23:43.989694  5215 rpcz_store.cc:276] 0709 12:23:42.443177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.452023 (+  8846us) service_pool.cc:224] Handling call
0709 12:23:43.959240 (+1507217us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b540023b9d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:43.983221 (+ 23981us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1868776192}
I20260709 12:23:43.965585  6799 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:43.991370  6574 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:43.994035  6732 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:43.947892  6871 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:43.988852  5948 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:44.005226  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.003978  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:44.009383  5042 leader_election.cc:336] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.896s (SENT)
I20260709 12:23:44.010056  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:43.996964  6893 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:44.002702  6717 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:44.025489  6898 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
W20260709 12:23:44.027558  6804 consensus_meta.cc:287] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.615s	user 0.013s	sys 0.000s
I20260709 12:23:44.028254  6804 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.029853  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.031507  6804 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:44.032152  5945 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:44.038062  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.039670  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.050025  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.052336  6688 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:44.054840  5221 raft_consensus.cc:1240] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:44.055673  5216 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:23:44.054894  6829 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.056824  6816 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:44.057456  6311 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:44.058166  6311 raft_consensus.cc:993] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: : Instructing follower ceec344dd00f40c18ef58414ce147016 to start an election
I20260709 12:23:44.056017  6812 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.058705  6311 raft_consensus.cc:1081] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Signalling peer ceec344dd00f40c18ef58414ce147016 to start an election
I20260709 12:23:44.059811  5067 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:23:44.059948  6875 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:44.060365  6812 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:44.061058  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.000569  6834 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.062559  5041 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:44.064730  6732 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:44.065394  6732 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:44.065864  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.066476  6732 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:44.068826  6650 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:44.033782  6893 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:44.041710  6861 consensus_queue.cc:237] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:44.076439  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:44.077883  6878 raft_consensus.cc:993] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: : Instructing follower 9847a7797ddc4df689a1ab0911af8785 to start an election
W20260709 12:23:44.078313  6844 consensus_meta.cc:287] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.734s	user 0.008s	sys 0.007s
I20260709 12:23:44.078802  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:44.078743  6844 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:44.080143  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:44.081511  6844 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:44.083632  6806 raft_consensus.cc:993] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: : Instructing follower 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:23:44.084689  6306 raft_consensus.cc:1081] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Signalling peer 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:23:44.085979  6897 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:44.086345  6897 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:44.087213  6897 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:44.087711  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.088497  6897 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:44.091629  5900 leader_election.cc:336] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.939s (SENT)
I20260709 12:23:44.075465  6870 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:44.092170  6870 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:23:44.092931  5900 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.992s (SENT)
I20260709 12:23:44.093065  6870 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.095171  6870 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:44.095870  6766 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:44.046157  6892 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:23:44.089802  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.085973  6306 raft_consensus.cc:1081] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Signalling peer 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:23:44.054852  6688 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:44.099015  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:44.099438  6901 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:23:44.099972  6898 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:44.109836  5221 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:44.101070  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.100901  6311 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:44.101506  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.101913  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.102325  6878 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:23:44.104269  6885 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.382s	user 0.003s	sys 0.000s
I20260709 12:23:44.100730  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.110505  5947 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []
W20260709 12:23:44.119854  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:44.131331  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.112385  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:44.136718  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 127384us
I20260709 12:23:44.144299  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:44.153018  5221 raft_consensus.cc:2368] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4.
I20260709 12:23:44.108428  6688 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:44.157053  6869 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:44.168701  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.140692  5947 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:39878
W20260709 12:23:44.161864  5115 connection.cc:256] RPC call timeout handler was delayed by -2.4095534130001397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1834s for a response
W20260709 12:23:44.175451  5115 connection.cc:256] RPC call timeout handler was delayed by -1.9577534129998639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2336s for a response
I20260709 12:23:44.176128  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.141373  5939 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:39878
I20260709 12:23:44.222671  5939 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.223091  5939 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:44.083630  6863 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.148145  6889 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:23:44.175112  5947 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.247745  5947 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:44.160974  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.170212  6801 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.203349  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:44.225265  5115 connection.cc:256] RPC call timeout handler was delayed by -1.740853412999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2577s for a response
W20260709 12:23:44.264125  5115 connection.cc:256] RPC call timeout handler was delayed by -1.4996534130000327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28450000000000003s for a response
I20260709 12:23:44.214303  6869 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:23:44.229583  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.171686  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.150866  6878 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:44.262385  5939 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.227999  5948 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
W20260709 12:23:44.246490  6867 log.cc:927] Time spent T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: Append to log took a long time: real 0.678s	user 0.004s	sys 0.002s
I20260709 12:23:44.279093  5939 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:44.269603  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:44.282797  5940 raft_consensus.cc:2368] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:44.286417  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:44.254511  5941 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:44.312181  5190 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:44.323174  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:44.317672  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 208164us
I20260709 12:23:44.326164  5947 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:44.328097  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:44.330432  5945 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:44.331859  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.333657  5224 raft_consensus.cc:2368] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3.
I20260709 12:23:44.334352  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:44.341272  6903 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:23:44.341701  6609 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:44.330736  5217 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:44.335989  5215 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:44.363157  5947 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:44.331617  6877 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:44.332042  6696 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:44.316228  6702 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:23:44.368031  5041 leader_election.cc:400] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3.
I20260709 12:23:44.418249  5041 leader_election.cc:403] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 12:23:44.418888  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 51045.6us
I20260709 12:23:44.370637  6863 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:44.268410  6688 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:44.373133  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:44.374138  6873 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.000s
I20260709 12:23:44.376510  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:44.377040  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:44.379113  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:44.380190  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:44.424201  5073 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:44.381831  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:44.382416  6889 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:44.283914  6801 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:44.425427  5076 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 3: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:44.386106  6913 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:44.427165  6801 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:44.390216  6908 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:23:44.390372  5068 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:44.392149  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.401077  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.416103  5219 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:44.420074  5071 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:44.421663  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:44.365856  6879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.004s	sys 0.008s
I20260709 12:23:44.380713  6892 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.432782  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:44.431715  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 97480.3us
I20260709 12:23:44.434016  5219 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:23:44.433032  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 105007us
I20260709 12:23:44.434840  5190 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:44.436153  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:44.437037  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:44.439975  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:44.431527  6879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.004s	sys 0.008s
I20260709 12:23:44.446858  6799 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:44.440606  6892 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.422248  6869 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.429589  6873 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:44.415380  6861 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 12:23:44.449415  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.493758  5215 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:44.496450  6898 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:44.450495  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:44.497152  6898 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.498448  6702 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:44.453680  6799 raft_consensus.cc:697] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:44.501077  6799 consensus_queue.cc:237] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:44.463840  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:44.479009  5222 raft_consensus.cc:2368] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:44.481910  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:44.505594  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 55230.8us
I20260709 12:23:44.423659  6717 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:44.508658  5115 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:44.485833  6696 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:44.512986  6696 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:44.513089  6882 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:44.513463  6696 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:44.459741  6878 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:44.513922  6882 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:44.490116  6879 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:44.515251  6879 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:44.516639  6882 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:44.462091  6892 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:44.472930  6915 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.519922  6915 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:44.468432  5223 raft_consensus.cc:2410] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:23:44.491815  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.475272  6801 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:44.522372  6801 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:44.522611  6878 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.450239  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.523597  5221 raft_consensus.cc:2368] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4.
I20260709 12:23:44.523270  6878 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.524623  6801 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:44.499859  6898 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:44.572080  6892 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:44.536037  6702 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:44.490638  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.575524  6892 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.522662  6915 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:44.530396  6879 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:44.477443  6873 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:44.525460  6878 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:44.557147  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:44.578796  6892 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:44.477977  6688 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:44.582019  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 46524.3us
I20260709 12:23:44.485797  6893 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:44.524639  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.587373  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.587610  6893 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:44.588214  5221 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:44.548527  6766 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.591212  6766 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:44.537283  5148 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:44.577706  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.591912  6915 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:44.593730  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:44.594681  5942 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:44.594966  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.595924  5219 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:44.596453  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.594089  6915 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:44.597579  5946 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:44.597680  6915 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:44.600356  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.601271  5217 raft_consensus.cc:3055] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:44.601660  5217 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:23:44.603778  5190 leader_election.cc:336] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.525s (SENT)
I20260709 12:23:44.604633  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:44.608211  6766 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:44.608862  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.611027  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.612120  5222 raft_consensus.cc:2368] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3.
I20260709 12:23:44.613682  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.604835  6696 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:44.619092  6696 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:44.620842  6871 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:44.620842  5222 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
W20260709 12:23:44.622187  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.609047  6766 consensus_queue.cc:1059] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
W20260709 12:23:44.623508  5190 leader_election.cc:336] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.751s (SENT)
I20260709 12:23:44.623637  6801 raft_consensus.cc:3055] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:44.624603  6801 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:44.624073  6874 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:44.624552  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.628437  5222 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:23:44.630829  5042 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.516s (SENT)
I20260709 12:23:44.591636  6877 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:44.632750  5221 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
W20260709 12:23:44.634034  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 50342.9us
I20260709 12:23:44.634207  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.625742  6801 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:44.638540  6829 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:44.639447  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.640803  5222 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:44.639566  6801 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:44.642885  5219 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:44.644069  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.645882  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:44.646098  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.644385  5115 leader_election.cc:400] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4.
I20260709 12:23:44.647461  5219 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:44.643168  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.648842  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.651051  5220 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:44.654712  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.655445  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.656396  5222 raft_consensus.cc:2410] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:44.657473  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:44.657763  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.657884  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.658633  5222 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 3: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:44.659288  5219 raft_consensus.cc:2410] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:23:44.659497  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:44.663981  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:44.664335  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.664850  5221 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:23:44.665163  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.666968  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:44.668452  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:44.668736  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:44.669622  6876 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:44.670471  6876 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:44.670701  5115 leader_election.cc:400] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4.
I20260709 12:23:44.677521  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:44.680794  5115 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:44.684504  5221 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:44.681952  6875 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:44.690635  5221 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:44.694564  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:44.695940  5076 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:44.699044  6702 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:44.700325  6880 raft_consensus.cc:993] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac: : Instructing follower 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:23:44.700652  6834 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:44.704433  6717 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3.
W20260709 12:23:44.705593  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 365231us
I20260709 12:23:44.706221  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 267) took 1437 ms. Trace:
I20260709 12:23:44.706873  6913 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:44.706648  5068 rpcz_store.cc:276] 0709 12:23:43.268586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.285209 (+ 16623us) service_pool.cc:224] Handling call
0709 12:23:44.706000 (+1420791us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5400589654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:44.706121 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1098425216}
I20260709 12:23:44.602962  5217 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.708714  5217 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:44.713774  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.716300  5215 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:44.718314  5224 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:44.719343  6438 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:44.698921  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:44.729765  5900 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:44.698186  5115 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:44.737983  5074 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:44.740028  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:44.740397  5076 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:44.720634  5220 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:44.741295  5115 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:44.682662  6877 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:44.743444  6886 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:44.743520  6650 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:44.744011  6886 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:44.744714  6915 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:44.746199  6876 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:44.746886  6609 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:44.747519  6915 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:44.748636  5068 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:44.742472  6880 raft_consensus.cc:1081] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Signalling peer 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:23:44.751515  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.752244  5216 raft_consensus.cc:2376] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4. Re-sending same reply.
I20260709 12:23:44.752611  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 229) took 1171 ms. Trace:
I20260709 12:23:44.753065  5148 rpcz_store.cc:276] 0709 12:23:43.584402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.609833 (+ 25431us) service_pool.cc:224] Handling call
0709 12:23:44.752394 (+1142561us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54006e2154. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:44.752513 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2116344192}
I20260709 12:23:44.756800  5222 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:44.757148  6877 raft_consensus.cc:697] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:44.739950  6702 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:44.758445  6877 consensus_queue.cc:237] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:44.736937  6861 consensus_queue.cc:1175] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 2 } }
W20260709 12:23:44.761778  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 324948us
I20260709 12:23:44.768028  6438 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:44.771298  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:44.772173  5947 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:44.773963  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:44.774857  5071 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:44.745055  6886 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:44.776796  6886 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:44.780413  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.781774  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.782719  6702 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:44.785776  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.788561  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:44.789247  5942 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:44.789754  6650 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:44.792268  5188 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:44.794588  6878 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:44.794988  6878 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:44.795315  6878 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:44.798611  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:44.800869  6861 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:44.802276  6688 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:44.803037  5042 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.754s (SENT)
I20260709 12:23:44.803746  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:44.806226  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 97587.5us
W20260709 12:23:44.808225  5042 leader_election.cc:336] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.790s (SENT)
I20260709 12:23:44.808743  5042 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:44.805388  6889 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:44.801312  6861 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:44.811928  5900 leader_election.cc:400] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:44.812423  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:44.815455  5900 leader_election.cc:403] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:23:44.815876  5145 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:44.811007  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:44.809841  5218 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:44.813836  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:44.820775  5041 leader_election.cc:400] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:44.820473  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:44.820955  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:44.821163  5041 leader_election.cc:403] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:23:44.821599  5146 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:44.756880  5217 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:44.819801  6915 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:44.819897  6844 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:44.826864  6915 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:44.827157  6844 raft_consensus.cc:697] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:44.828325  6831 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:44.828239  6915 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:44.829635  6702 raft_consensus.cc:3055] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:44.830055  6702 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:44.818945  6861 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:44.830968  6702 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:44.832490  6702 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:44.826924  6843 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Leader election lost for term 2. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:44.813567  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:44.835839  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 462807us
I20260709 12:23:44.836062  5943 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:44.837124  5076 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:44.837615  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:44.838472  5944 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:44.834367  6829 consensus_queue.cc:1059] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:23:44.834151  6720 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:44.810429  6438 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:44.843590  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:44.844334  5946 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:44.828719  6844 consensus_queue.cc:237] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.849098  6876 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:44.849974  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:44.861114  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.855513  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:44.856760  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:44.857395  5188 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:44.863592  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:44.864218  5939 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:44.865620  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:44.867795  5148 raft_consensus.cc:2410] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:23:44.868115  5042 leader_election.cc:336] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.570s (SENT)
I20260709 12:23:44.872599  6829 raft_consensus.cc:3055] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:44.873025  5940 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:23:44.869557  5946 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:44.875169  5189 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:44.881301  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:44.884161  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:44.885039  5146 raft_consensus.cc:2376] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6. Re-sending same reply.
I20260709 12:23:44.892097  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:44.892493  6834 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:44.893163  5145 raft_consensus.cc:2410] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:44.866701  6874 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:23:44.878855  6829 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:44.895503  5041 leader_election.cc:400] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3.
I20260709 12:23:44.896118  5041 leader_election.cc:403] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:23:44.896834  6732 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:44.896488  6829 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.897266  6732 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:44.897712  6732 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:44.897969  6829 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:44.899346  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:44.873689  6812 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:44.900947  6834 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:44.897717  6717 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3.
I20260709 12:23:44.901575  6834 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:44.881809  6871 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:44.879053  6688 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:44.902525  6834 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:44.900978  6812 raft_consensus.cc:697] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:44.900336  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:44.906535  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:44.916626  5939 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:44.909518  5223 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:44.879906  4899 catalog_manager.cc:5697] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
W20260709 12:23:44.929003  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 121978us
I20260709 12:23:44.911587  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.946059  5943 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:44.947309  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:44.948192  5946 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:44.950063  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:44.950095  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:44.951175  5145 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:44.910553  6720 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:44.910573  6717 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:44.961858  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:44.964931  5146 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:44.969373  4899 catalog_manager.cc:5697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.970237  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:44.971529  4899 catalog_manager.cc:5697] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:23:44.975198  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:44.976027  5146 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:44.978255  5900 leader_election.cc:400] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:44.978811  5900 leader_election.cc:403] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:23:44.983902  5190 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.009s (SENT)
I20260709 12:23:45.001572  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:44.942672  6649 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:23:45.014868  4899 catalog_manager.cc:5697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:45.010573  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:45.025735  4899 catalog_manager.cc:5697] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 5, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 5 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
W20260709 12:23:45.027565  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.027420  4899 catalog_manager.cc:5697] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:23:45.028520  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:44.911623  6812 consensus_queue.cc:237] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:45.035524  4899 catalog_manager.cc:5697] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 4, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 4 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:23:45.001911  6843 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3.
I20260709 12:23:45.037938  5218 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:45.038162  5071 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:45.041059  4899 catalog_manager.cc:5697] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 4, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 4 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:45.043463  5898 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:45.044085  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.044719  5145 raft_consensus.cc:2376] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5. Re-sending same reply.
I20260709 12:23:45.046769  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:45.048640  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.049435  5148 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:45.039746  6299 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:45.032656  6438 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:45.052292  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:23:45.053725  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.043385  4899 catalog_manager.cc:5697] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 4, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 4 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:45.055367  4899 catalog_manager.cc:5697] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 3, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 3 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:23:45.057461  4899 catalog_manager.cc:5697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 3, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 3 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:23:45.058938  4899 catalog_manager.cc:5697] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 3, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 3 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:45.060973  4899 catalog_manager.cc:5697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 5, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 5 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:45.070909  5146 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:45.062867  4898 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43226 (request call id 128) took 7198 ms. Trace:
I20260709 12:23:45.074360  4899 catalog_manager.cc:5697] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 5, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:23:45.076169  4899 catalog_manager.cc:5697] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 3, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 3 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:45.096024  5946 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:39878
I20260709 12:23:45.052273  5944 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:45.097872  5946 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:45.107925  5946 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:45.108990  5145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3"
dest_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:56156
W20260709 12:23:45.098984  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 46793.2us
I20260709 12:23:45.109752  5145 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:45.110080  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:45.111056  6438 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:45.111470  6438 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:45.111848  6438 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:45.112124  5147 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:45.102006  6878 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:45.102084  6928 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:23:45.050716  6654 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:23:45.110172  5145 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:45.097553  4899 catalog_manager.cc:5697] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:45.123982  4899 catalog_manager.cc:5697] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:45.124293  5946 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:45.112190  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:45.104822  5190 leader_election.cc:336] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.101s (SENT)
I20260709 12:23:45.125761  5945 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:45.128070  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.128537  4899 catalog_manager.cc:5697] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 4, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 4 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:23:45.130198  4899 catalog_manager.cc:5697] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:23:45.074963  4898 rpcz_store.cc:276] 0709 12:23:37.862029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.863092 (+  1063us) service_pool.cc:224] Handling call
0709 12:23:45.057470 (+7194378us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:42.958267 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:42.990051 (+ 31784us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:42.990105 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:23:42.990124 (+    19us) tablet.cc:662] Decoding operations
0709 12:23:42.991126 (+  1002us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:42.991235 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:42.991268 (+    33us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:23:42.992048 (+   780us) tablet.cc:701] Row locks acquired
0709 12:23:42.992101 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:23:42.992283 (+   182us) write_op.cc:270] Start()
0709 12:23:42.992446 (+   163us) write_op.cc:276] Timestamp: P: 1783599822992224 usec, L: 0
0709 12:23:42.992481 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:23:43.035927 (+ 43446us) log.cc:844] Serialized 3233 byte log entry
0709 12:23:44.104513 (+1068586us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:23:44.480650 (+376137us) op_driver.cc:464] REPLICATION: finished
0709 12:23:44.675122 (+194472us) write_op.cc:301] APPLY: starting
0709 12:23:44.675250 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:44.675533 (+   283us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:44.675573 (+    40us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:44.677211 (+  1638us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:44.677743 (+   532us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:23:44.677841 (+    98us) write_op.cc:312] APPLY: finished
0709 12:23:44.701825 (+ 23984us) log.cc:844] Serialized 64 byte log entry
0709 12:23:44.702922 (+  1097us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:44.703083 (+   161us) write_op.cc:454] Released schema lock
0709 12:23:44.703138 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:44.703595 (+   457us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:44.703622 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":425991,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":194162,"apply.run_cpu_time_us":4249,"apply.run_wall_time_us":28462,"num_ops":8,"prepare.queue_time_us":211415,"prepare.run_cpu_time_us":32587,"prepare.run_wall_time_us":497847,"raft.queue_time_us":136830,"raft.run_cpu_time_us":13999,"raft.run_wall_time_us":170429,"replication_time_us":1487964,"spinlock_wait_cycles":9592576,"thread_start_us":662738,"threads_started":4,"wal-append.queue_time_us":461750}]]}
I20260709 12:23:45.125962  5142 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:45.134114  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.134531  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:45.134862  5143 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:45.135967  5190 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:45.116039  6804 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:45.104988  5942 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:45.142727  5900 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:45.155506  6843 raft_consensus.cc:2764] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:23:45.158046  6843 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:45.160354  6921 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.160997  6921 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:45.162485  6804 raft_consensus.cc:697] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:45.163371  6921 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:45.163623  6804 consensus_queue.cc:237] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:45.165607  6927 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:45.166368  5218 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:45.166184  6927 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:45.167927  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.168984  6927 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:45.169939  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:45.170964  5073 raft_consensus.cc:2410] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:23:45.145922  5190 leader_election.cc:336] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.151s (SENT)
I20260709 12:23:45.131769  6349 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:23:45.147241  6877 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:45.147722  6878 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:45.148280  6844 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:45.148627  6933 log.cc:927] Time spent T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:23:45.149286  6931 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:23:45.150200  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:45.192169  5071 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:45.197495  5944 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:45.199594  5075 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:45.179420  5041 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:45.190887  5946 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:45.191103  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 45239.2us
W20260709 12:23:45.205029  5190 leader_election.cc:336] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.192s (SENT)
I20260709 12:23:45.207518  5071 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:45.209108  5188 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:45.214478  5899 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:45.218017  5899 leader_election.cc:400] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:45.218607  5899 leader_election.cc:403] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:23:45.221356  5899 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:45.223443  5190 leader_election.cc:336] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.211s (SENT)
I20260709 12:23:45.223970  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:45.224438  6869 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:45.224785  5076 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:45.225836  6732 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:45.224080  6889 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:23:45.228178  6732 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:45.229624  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.230248  5142 raft_consensus.cc:2376] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6. Re-sending same reply.
I20260709 12:23:45.228781  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:45.234149  5941 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:45.234844  5899 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:45.235448  5944 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:23:45.236229  6299 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:45.237365  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.238098  5150 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:45.239060  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.240404  5899 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:45.241906  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.242556  5141 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:45.223994  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:45.246482  5148 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:45.219504  5145 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:45.248318  5147 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:45.255749  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.256887  5148 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:45.258601  5076 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:45.218060  6720 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:45.264577  5150 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:45.271890  5899 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:45.235535  6886 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:45.221711  6927 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:45.275221  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.272693  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.250432  6843 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:45.290429  5150 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:45.291363  5144 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:45.259292  6913 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:45.301429  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.266523  6869 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:45.273074  6816 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:45.314491  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.316998  5150 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:45.312767  5141 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:45.295411  6720 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:45.289147  5148 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:45.225836  6349 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 12:23:45.307790  5145 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:45.343259  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.343946  5145 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:45.344923  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.312184  5147 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:45.338214  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.347808  5141 raft_consensus.cc:2376] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4. Re-sending same reply.
I20260709 12:23:45.306318  6886 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:45.313848  6844 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:45.340713  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.349521  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:45.351711  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 79959.8us
I20260709 12:23:45.352759  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.355712  5899 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:45.356065  5150 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:45.358264  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:45.359624  5148 raft_consensus.cc:2368] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3.
I20260709 12:23:45.363241  6886 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:45.365654  6349 consensus_queue.cc:1059] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:45.372545  5223 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:45.374310  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:45.374648  6299 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:45.375427  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.322563  6913 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:45.376422  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:45.379031  5042 leader_election.cc:336] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.145s (SENT)
I20260709 12:23:45.394450  5224 raft_consensus.cc:2410] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:45.385957  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.395406  5147 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:45.388495  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:45.389024  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:45.398023  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:45.393038  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 173573us
I20260709 12:23:45.330992  6869 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:45.396257  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.400251  5147 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:45.403264  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:45.421159  5148 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
W20260709 12:23:45.425910  5190 leader_election.cc:336] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.215s (SENT)
W20260709 12:23:45.403585  6861 consensus_meta.cc:287] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.572s	user 0.017s	sys 0.000s
I20260709 12:23:45.429212  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.432960  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.432952  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:45.435055  5148 raft_consensus.cc:2376] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5. Re-sending same reply.
I20260709 12:23:45.435300  5222 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:45.438661  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 281355us
I20260709 12:23:45.443320  5190 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:45.450947  5150 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:45.478736  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:45.479596  5220 raft_consensus.cc:3055] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:45.479923  5220 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:45.480810  5220 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:45.482311  5220 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:45.487519  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44309.7us
I20260709 12:23:45.490167  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:45.491906  5042 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:45.492370  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.493245  5150 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:45.494757  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:45.430780  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:45.363164  6720 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:45.508086  5076 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:45.430603  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:45.510103  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:45.509516  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:45.511099  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.499571  5147 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:23:45.394531  6438 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:45.512843  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.513707  5150 raft_consensus.cc:2376] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4. Re-sending same reply.
I20260709 12:23:45.516866  5042 leader_election.cc:400] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 2 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 2. Current term is 3.
I20260709 12:23:45.502264  5220 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:45.436518  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.511998  5148 raft_consensus.cc:2393] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 3: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:45.442926  6650 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:45.437237  6654 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.074s
W20260709 12:23:45.581413  5042 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.054s (SENT)
W20260709 12:23:45.626109  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44810.9us
I20260709 12:23:45.528394  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.603312  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.515441  6732 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:23:45.609479  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.439311  6861 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:45.628827  5147 raft_consensus.cc:2410] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:45.606653  6311 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:23:45.509500  6880 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:23:45.603775  6829 consensus_meta.cc:287] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.705s	user 0.016s	sys 0.005s
I20260709 12:23:45.642076  5144 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:45.643296  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.643630  6880 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:45.627590  5145 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:45.630887  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.634394  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.634443  5222 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
W20260709 12:23:45.550895  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.644398  6880 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:45.526042  6806 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:45.514937  6812 raft_consensus.cc:2764] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:23:45.658936  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.528805  6831 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:45.660080  5150 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:45.644671  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.638037  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.671850  5147 raft_consensus.cc:2376] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3. Re-sending same reply.
I20260709 12:23:45.662200  5148 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:45.665822  5220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2725c773e7344020b5752861d00b493e"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:45.504069  6878 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.136s
I20260709 12:23:45.627568  6927 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:45.501399  6874 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:45.633975  6650 raft_consensus.cc:697] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:45.664288  5216 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:45.675062  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.680490  5145 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:45.676635  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.669788  5944 raft_consensus.cc:2410] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:45.527129  6913 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:45.694304  5220 raft_consensus.cc:606] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:23:45.695013  5220 raft_consensus.cc:614] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:45.684024  5147 raft_consensus.cc:2410] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:45.641273  6732 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:45.678943  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.696579  5150 raft_consensus.cc:2376] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5. Re-sending same reply.
I20260709 12:23:45.687922  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:45.696031  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.681648  6871 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:45.697952  5148 raft_consensus.cc:2376] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5. Re-sending same reply.
I20260709 12:23:45.698199  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.700935  5145 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:45.680040  5144 raft_consensus.cc:2376] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6. Re-sending same reply.
I20260709 12:23:45.714114  5147 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:45.687222  6874 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:45.721621  5148 raft_consensus.cc:3055] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:45.722405  5148 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:45.707545  6732 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:45.670200  6869 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:45.730104  5150 pending_rounds.cc:85] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 3
I20260709 12:23:45.730577  5150 pending_rounds.cc:107] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:23:45.725697  5145 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:45.699154  6299 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:45.730928  5150 raft_consensus.cc:2889] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:45.741993  5148 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:45.724287  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 374857us
I20260709 12:23:45.723914  6844 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:45.754204  5148 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:45.714532  6871 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:45.729009  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 370816us
I20260709 12:23:45.759779  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:45.772825  5222 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:45.773950  6927 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:45.775076  6927 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:45.735646  6650 consensus_queue.cc:237] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:45.778446  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:45.786016  6968 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:45.786511  5074 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:45.786684  6968 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:23:45.785571  5900 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.602s (SENT)
I20260709 12:23:45.790351  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:45.804064  5189 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:45.781235  6946 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:23:45.821808  5189 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:45.788730  6438 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:45.824122  5148 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:23:45.835577  5148 pending_rounds.cc:85] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 1
I20260709 12:23:45.835886  5148 pending_rounds.cc:107] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:23:45.836170  5148 raft_consensus.cc:2889] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:45.838606  5148 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:45.844583  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:45.845501  5223 raft_consensus.cc:3055] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:45.845938  5223 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:45.852387  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:45.821151  6654 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.822453  6861 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:45.846939  5223 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:45.868495  5223 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:45.860447  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:45.840673  6939 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:23:45.828908  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:45.892683  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.893487  5142 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:45.894876  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:45.895891  5142 raft_consensus.cc:2376] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5. Re-sending same reply.
W20260709 12:23:45.889420  6944 log.cc:927] Time spent T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:23:45.925329  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.811841  6874 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:45.828876  6966 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:23:45.885038  5944 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:45.941804  5223 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:45.945233  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.862329  6929 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:23:45.892899  6967 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:23:45.947755  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 162260us
W20260709 12:23:45.933423  6948 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 12:23:45.841490  6299 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:45.876837  6654 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:45.958830  5115 leader_election.cc:400] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5.
I20260709 12:23:45.959379  5115 leader_election.cc:403] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:45.812222  6968 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:45.960793  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:45.961967  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.937933  6878 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:45.978392  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.981890  5142 raft_consensus.cc:2376] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5. Re-sending same reply.
I20260709 12:23:45.984009  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.947551  6843 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:45.954648  6965 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:23:45.992429  5900 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.551s (SENT)
I20260709 12:23:45.892236  6844 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:45.986351  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:45.995988  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:45.994731  5900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.678s (SENT) [suppressed 66 similar messages]
I20260709 12:23:45.999222  5148 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:23:46.000211  5900 leader_election.cc:336] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.685s (SENT)
W20260709 12:23:45.985914  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 152462us
I20260709 12:23:45.997941  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:46.013516  5148 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:23:45.988754  6971 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.030416  5148 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:46.033115  5145 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
W20260709 12:23:46.036232  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.035060  5149 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:46.048497  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:46.004079  6843 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:23:46.056376  5042 leader_election.cc:336] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.078s (SENT)
I20260709 12:23:46.057020  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:46.016073  6871 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:46.051636  6973 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:23:46.072450  6720 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:46.036396  6972 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:23:46.077943  6929 raft_consensus.cc:3055] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:46.076335  6806 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:46.112447  6654 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:46.113408  6880 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:46.115118  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:46.116310  6975 log.cc:927] Time spent T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:23:46.036576  6971 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:46.119234  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.102667  6929 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:46.137933  5149 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
W20260709 12:23:46.142163  6980 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:23:46.142630  6732 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:46.143534  5042 leader_election.cc:336] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.164s (SENT)
I20260709 12:23:46.144064  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:46.146030  6978 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:23:46.111230  6956 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:46.116885  6880 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:46.145368  6732 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:46.149283  6956 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:46.124709  6843 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:46.139173  6981 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5.
I20260709 12:23:46.156785  6995 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:23:46.222201  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 274397us
I20260709 12:23:46.233006  5115 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:46.245426  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 434649us
I20260709 12:23:46.221239  6989 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.149938  6806 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:46.248528  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.259258  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:46.273154  6988 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:23:46.256376  6991 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:23:46.279418  6976 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:23:46.256703  6989 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:46.186672  6929 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:46.294811  6997 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:23:46.397118  5946 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:46.429565  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 181077us
I20260709 12:23:46.431288  5115 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.435599  6987 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.435951  6984 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.436276  6987 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:46.436467  6984 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:46.438766  6987 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:46.435282  7009 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:46.444923  5074 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:23:46.450423  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.454839  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:46.404268  6732 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:46.455791  5074 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:46.456660  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 248768us
W20260709 12:23:46.458556  5042 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.806s (SENT)
I20260709 12:23:46.460893  5946 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:46.440914  6929 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.466862  5071 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:46.466807  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.442504  6720 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:46.471202  5042 leader_election.cc:336] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.905s (SENT)
I20260709 12:23:46.472086  5042 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:46.459833  6971 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:46.476475  5042 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.451s (SENT)
I20260709 12:23:46.477223  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:46.493619  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.464398  6732 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:46.497334  5944 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.003s
I20260709 12:23:46.500586  5944 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:46.476891  6968 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:46.513383  5074 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:46.497474  5115 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.480052  7009 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:46.472762  6349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.106s	user 0.027s	sys 0.026s
I20260709 12:23:46.473819  6720 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:46.499748  7004 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:23:46.530967  7008 log.cc:927] Time spent T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 12:23:46.532007  6998 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:23:46.533710  6968 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:46.520879  6876 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:46.520650  6349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.154s	user 0.027s	sys 0.027s
W20260709 12:23:46.583158  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 64295.4us
I20260709 12:23:46.584529  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:46.586493  5141 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:46.587671  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 729) took 1354 ms. Trace:
I20260709 12:23:46.589591  5141 rpcz_store.cc:276] 0709 12:23:45.233383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.396925 (+163542us) service_pool.cc:224] Handling call
0709 12:23:46.584343 (+1187418us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:46.587580 (+  3237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27749632}
I20260709 12:23:46.594900  5224 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:46.595103  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.596799  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.598934  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 832) took 1181 ms. Trace:
I20260709 12:23:46.599819  5224 rpcz_store.cc:276] 0709 12:23:45.417344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.421798 (+  4454us) service_pool.cc:224] Handling call
0709 12:23:46.585475 (+1163677us) spinlock_profiling.cc:243] Waited 8.87 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:46.594685 (+  9210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:46.595631 (+   946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.598868 (+  3237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20403968}
I20260709 12:23:46.598608  6968 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.556190  7009 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:23:46.600566  6991 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:46.623907  5115 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.633340  6989 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:46.633929  6984 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:46.634425  6989 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:46.635700  6720 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:46.608439  6968 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:46.623288  6732 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:46.624462  7021 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:46.628328  7009 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:46.628474  7024 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:23:46.650883  7010 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:23:46.651911  6311 raft_consensus.cc:3055] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:46.652488  6311 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:46.653674  6311 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:46.655121  6311 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:46.662755  5115 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.646461  7009 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:46.665015  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:46.667318  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 190887us
I20260709 12:23:46.669162  5074 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:46.666322  7024 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:46.669612  5217 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:23:46.670065  7021 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:46.670377  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 640) took 1300 ms. Trace:
I20260709 12:23:46.660148  6929 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:23:46.673285  5074 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:46.671829  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:46.664513  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 751) took 1383 ms. Trace:
I20260709 12:23:46.670837  6968 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:46.678087  6929 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:46.681602  6929 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:46.682663  6929 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:46.678500  5144 rpcz_store.cc:276] 0709 12:23:45.279962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.727706 (+447744us) service_pool.cc:224] Handling call
0709 12:23:45.794578 (+ 66872us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:46.214168 (+419590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:46.216951 (+  2783us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:46.399259 (+182308us) log.cc:844] Serialized 26 byte log entry
0709 12:23:46.533696 (+134437us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:46.534181 (+   485us) log.cc:844] Serialized 14 byte log entry
0709 12:23:46.597314 (+ 63133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.601150 (+  3836us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:46.659384 (+ 58234us) raft_consensus.cc:1685] finished
0709 12:23:46.660474 (+  1090us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.663574 (+  3100us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":986,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":152043,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":17430,"spinlock_wait_cycles":3060864,"thread_start_us":460471,"threads_started":3,"wal-append.queue_time_us":164007}
I20260709 12:23:46.683861  6953 raft_consensus.cc:3055] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:46.683991  6929 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:46.684366  6953 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:46.688807  7020 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:46.691315  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:46.698467  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.699693  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:46.670820  5217 rpcz_store.cc:276] 0709 12:23:45.370440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.375895 (+  5455us) service_pool.cc:224] Handling call
0709 12:23:46.669039 (+1293144us) spinlock_profiling.cc:243] Waited 9.24 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416eee 000000000036eb39 00007fd8ad48a1e0 00007fd8ad5530c4 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:46.669216 (+   177us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:46.670081 (+   865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.670309 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21240576}
I20260709 12:23:46.707026  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.704676  5074 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:46.690454  7009 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:46.697909  7024 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:46.702251  7025 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:46.723378  6989 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:46.744721  6947 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 1.026s	user 0.002s	sys 0.001s
I20260709 12:23:46.701987  7020 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:46.728475  7004 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:23:46.740456  7021 consensus_queue.cc:1175] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "90e5220a62ed42fe83c82d0302d18fac" responder_term: 5 status { last_received { term: 4 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:46.740936  6953 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:46.747377  6953 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.701018  7005 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:23:46.741803  7009 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:46.752521  5074 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:46.700801  6869 consensus_queue.cc:1059] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
W20260709 12:23:46.756425  5042 leader_election.cc:336] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.728s (SENT)
I20260709 12:23:46.759225  5042 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:46.752046  6349 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:46.748399  7020 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:46.771337  6349 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:46.777155  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 70288.4us
I20260709 12:23:46.767092  6968 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:46.766744  7026 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:46.779808  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:46.780037  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:46.783001  5147 raft_consensus.cc:2376] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6. Re-sending same reply.
I20260709 12:23:46.783083  5148 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:46.780390  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:46.785058  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 830) took 1604 ms. Trace:
I20260709 12:23:46.785509  5218 rpcz_store.cc:276] 0709 12:23:45.180533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.180719 (+   186us) service_pool.cc:224] Handling call
0709 12:23:45.181142 (+   423us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:45.186760 (+  5618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:45.186819 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:45.201515 (+ 14696us) log.cc:844] Serialized 52 byte log entry
0709 12:23:45.439466 (+237951us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:45.506594 (+ 67128us) log.cc:844] Serialized 14 byte log entry
0709 12:23:45.506833 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.506919 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:46.691149 (+1184230us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b1800050800. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:23:46.744063 (+ 52914us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:23:46.784822 (+ 40759us) raft_consensus.cc:1685] finished
0709 12:23:46.784846 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.784995 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":64098,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":61,"raft.queue_time_us":3956,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":8342,"spinlock_wait_cycles":21864704,"thread_start_us":179164,"threads_started":1,"wal-append.queue_time_us":458774}
I20260709 12:23:46.787732  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:46.787904  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:46.785961  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:46.776173  6649 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.791285  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:46.791778  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:46.792228  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:46.792443  5144 raft_consensus.cc:2376] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6. Re-sending same reply.
I20260709 12:23:46.792919  5941 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:46.793406  6720 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:23:46.793476  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 341) took 1415 ms (client timeout 1821 ms). Trace:
I20260709 12:23:46.794008  6720 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:46.794536  6720 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:46.794098  5941 rpcz_store.cc:269] 0709 12:23:45.377737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.402259 (+ 24522us) service_pool.cc:224] Handling call
0709 12:23:46.792055 (+1389796us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:46.793394 (+  1339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8105856}
I20260709 12:23:46.796636  5900 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:46.797418  6717 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:46.798519  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 395) took 1017 ms. Trace:
I20260709 12:23:46.791039  6732 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:46.799050  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:46.799464  6717 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:46.801770  7005 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:46.802240  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:46.802377  7005 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:46.803046  6869 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:46.809828  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:46.810081  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:46.811018  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:23:46.799566  6994 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.584s	user 0.001s	sys 0.004s
I20260709 12:23:46.809875  6989 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.784852  6968 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:46.816568  7024 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:46.821565  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57004 (request call id 7) took 1176 ms. Trace:
I20260709 12:23:46.826378  7024 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:46.827608  6968 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:46.817392  5141 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:23:46.829569  5141 pending_rounds.cc:85] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 2
I20260709 12:23:46.830008  5141 pending_rounds.cc:107] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:23:46.830367  5141 raft_consensus.cc:2889] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:46.826574  5220 rpcz_store.cc:276] 0709 12:23:45.645646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.665469 (+ 19823us) service_pool.cc:224] Handling call
0709 12:23:46.821331 (+1155862us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b5400446bd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a8535853 00007fd8ad50360a 00007fd8a6a897d4 00007fd8a6a89762 00007fd8a6a89691 00007fd8a6a8960c 00007fd8a6a888c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:46.821458 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48331008}
I20260709 12:23:46.807320  6869 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:46.814030  6831 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.826917  7024 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:46.844971  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 50419.7us
I20260709 12:23:46.807264  6649 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:46.848065  6831 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:46.839036  7005 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:46.856769  6649 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:46.811630  7004 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:23:46.836771  6989 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:46.849030  6986 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:46.851724  6940 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 1.488s	user 0.005s	sys 0.000s
I20260709 12:23:46.818326  7029 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:23:46.798960  5942 rpcz_store.cc:276] 0709 12:23:45.781300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.801627 (+ 20327us) service_pool.cc:224] Handling call
0709 12:23:46.781087 (+979460us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:23:46.788881 (+  7794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:46.798081 (+  9200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:46.798120 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:46.798186 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:46.798227 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.798302 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.798453 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26061184}
I20260709 12:23:46.863307  5069 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:46.866344  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:46.867384  5076 raft_consensus.cc:2410] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:23:46.871021  5145 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:46.875927  5147 raft_consensus.cc:2410] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:23:46.882532  6834 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.004s
W20260709 12:23:46.886961  6957 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.947s	user 0.004s	sys 0.000s
W20260709 12:23:46.888697  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.889438  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:46.889873  7004 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.890338  6799 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:46.890456  7014 raft_consensus.cc:3055] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:46.890805  6799 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:46.891036  7014 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:46.891322  6799 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.892282  7014 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:46.899185  5042 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:46.900504  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:46.902033  5946 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:46.911216  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:46.912911  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.925071  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 797) took 1840 ms. Trace:
W20260709 12:23:46.931398  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 62801us
I20260709 12:23:46.912352  6966 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:46.925571  5146 rpcz_store.cc:276] 0709 12:23:45.084778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.097643 (+ 12865us) service_pool.cc:224] Handling call
0709 12:23:45.098132 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:45.102957 (+  4825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:45.103008 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:45.103766 (+   758us) log.cc:844] Serialized 26 byte log entry
0709 12:23:45.201963 (+ 98197us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:45.202042 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.202114 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:46.850834 (+1648720us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:23:46.912515 (+ 61681us) raft_consensus.cc:1685] finished
0709 12:23:46.912540 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.924999 (+ 12459us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3005,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":237,"spinlock_wait_cycles":36765056,"thread_start_us":97569,"threads_started":1,"wal-append.queue_time_us":221073}
I20260709 12:23:46.918717  7014 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:46.912086  6921 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.944666  5218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5244acc0775748b7933286d588ec2b56"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:46.921209  6861 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:46.980165  6861 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:46.980683  6861 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:46.956806  6965 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.936034  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:46.890460  7004 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:46.981451  6965 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:46.935797  6929 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.983857  6965 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:46.983903  6929 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:46.984330  6956 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:46.986853  6956 raft_consensus.cc:476] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 12:23:46.923739  6886 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
W20260709 12:23:46.990154  7033 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:23:46.990289  6831 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:46.992667  7036 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:23:46.994199  6349 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:46.968977  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:46.995826  5146 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
W20260709 12:23:46.997404  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 61465.7us
I20260709 12:23:46.998155  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:46.998289  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:47.000703  6877 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:47.001832  6806 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:47.002303  6806 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:46.917450  6688 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.978929  5218 raft_consensus.cc:606] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:47.008750  5218 raft_consensus.cc:614] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:47.004503  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.002725  6806 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:47.008993  5042 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:46.956753  6921 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:47.032974  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.056252  5218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:47.056982  5218 raft_consensus.cc:606] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:23:47.057339  5218 raft_consensus.cc:614] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:47.073362  6349 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.074350  6349 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.078478  6349 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.081668  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.085014  5068 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:47.013420  6874 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:23:47.015098  6688 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.026405  6877 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:23:47.015923  7021 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.091650  6688 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.091949  7021 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:47.094228  7021 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.096866  7022 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:47.056165  6987 raft_consensus.cc:3055] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:47.097592  7022 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.098166  6987 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:47.098937  6929 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.099939  7022 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:47.099406  6987 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:47.101128  6987 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:47.028822  6979 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:47.101905  6720 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:47.107234  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:47.113103  5146 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:47.085175  6989 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:47.117444  5218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
W20260709 12:23:47.082882  7037 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:23:47.118301  5218 raft_consensus.cc:606] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:47.118700  5218 raft_consensus.cc:614] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:47.119239  5898 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:47.124945  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:47.108165  6979 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.123428  7029 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:47.123881  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:47.138603  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 134666us
I20260709 12:23:47.128938  6979 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:47.116531  6861 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.145659  5218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:47.146324  5218 raft_consensus.cc:606] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:47.146679  5218 raft_consensus.cc:614] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:47.130360  7024 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:47.137410  7029 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.151634  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:47.152576  5148 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:47.153396  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:23:47.156478  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.158947  5150 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:47.141366  6816 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:23:47.162328  5142 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:47.147972  7001 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.157325  6989 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.171325  7050 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.171706  6989 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:47.172050  7050 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.144761  6688 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:23:47.174305  6989 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:47.175853  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.160549  7026 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:47.155938  7029 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:47.177783  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.187116  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:47.178638  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:47.188408  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:47.188570  5148 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:47.180372  5073 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:47.149358  6799 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.191821  5072 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:47.194293  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.195230  5070 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:47.171324  6720 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:47.199528  5074 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:47.202529  6806 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.202600  6921 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:47.241816  5073 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:47.222844  5943 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:47.231616  7050 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.234047  7024 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.234196  6799 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:47.236370  7004 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:47.238909  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.240095  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.240417  6861 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:47.241991  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.258170  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:47.259070  5944 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:47.181211  7001 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.254683  5067 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:47.266713  5190 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:47.274080  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:47.277666  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.278704  5072 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 2.
I20260709 12:23:47.279217  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:47.288384  5944 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:47.289757  5114 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:47.290149  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:47.291685  5114 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:47.292384  7001 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.292891  6964 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:47.274554  7050 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:23:47.261919  6943 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
W20260709 12:23:47.295451  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.299273  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:47.300829  5072 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:47.267620  6921 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:47.286203  6874 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:47.280282  6816 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:23:47.293365  6964 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.284449  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:47.288661  6952 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:47.293943  7006 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:47.293411  6991 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:47.288318  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:47.346609  5943 raft_consensus.cc:2410] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:47.336838  6953 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:47.348147  6953 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:47.350589  6953 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:47.334743  7006 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.351547  7006 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:47.294551  7014 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:23:47.320946  6964 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:47.317862  6299 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:47.323223  6311 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:47.365720  7055 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:47.366305  7055 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.366681  7055 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:47.368196  6299 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.370764  6299 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.366294  6311 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:47.374698  6311 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:47.375352  6311 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:47.335561  5144 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:47.315762  6921 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.381363  6952 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.385010  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 398) took 1601 ms. Trace:
I20260709 12:23:47.385643  5945 rpcz_store.cc:276] 0709 12:23:45.784076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.784333 (+   257us) service_pool.cc:224] Handling call
0709 12:23:45.784783 (+   450us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:45.955053 (+170270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:45.955093 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:47.156056 (+1200963us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b2c00152dc8. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:23:47.165572 (+  9516us) log.cc:844] Serialized 52 byte log entry
0709 12:23:47.252325 (+ 86753us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:47.252371 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.252472 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:47.384523 (+132051us) raft_consensus.cc:1685] finished
0709 12:23:47.384560 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:47.384922 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85418,"raft.queue_time_us":156850,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":17305,"spinlock_wait_cycles":10926336,"thread_start_us":140487,"threads_started":2,"wal-append.queue_time_us":181488}
I20260709 12:23:47.389333  6952 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.382359  6861 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.394400  6861 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.394747  7050 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:47.395359  7050 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.396814  6861 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:47.387311  7042 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.217s	user 0.000s	sys 0.003s
I20260709 12:23:47.397439  7050 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:47.408898  6349 raft_consensus.cc:3055] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:47.412873  6349 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:47.414049  6349 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:47.373389  6979 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.408684  7029 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.359414  6806 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:47.420828  6979 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:47.375861  6311 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.429275  6311 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:47.438710  5220 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:47.440500  5218 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:47.442226  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:47.443266  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:47.444455  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:47.445130  5218 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
W20260709 12:23:47.449024  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.446314  6877 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:47.451210  6877 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.453776  6877 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.456046  6983 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:47.456480  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:47.456673  6983 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:47.457203  5217 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:47.445992  6967 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.459061  6983 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:47.458920  6967 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.464877  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:47.465844  5945 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:47.464797  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:47.431761  6349 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:47.467305  6943 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:47.469987  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:47.447888  6311 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.472077  6311 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:47.474000  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.474476  6311 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:47.476343  6943 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:47.476908  6943 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:47.478102  6943 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.480401  5218 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:47.482038  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:47.485257  7058 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.165s	user 0.000s	sys 0.003s
I20260709 12:23:47.486459  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.487033  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:47.519371  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:47.520635  5939 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:47.523200  6299 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:47.524401  7006 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.525008  6984 consensus_queue.cc:1059] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:47.525527  6964 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:47.526849  7006 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:47.461261  6967 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:47.516165  6965 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.494452  6952 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.584419  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:47.537639  6984 raft_consensus.cc:3055] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:23:47.591741  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 77016.9us
I20260709 12:23:47.566991  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.573608  6964 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.636487  5940 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:47.643678  5948 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:23:47.665155  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.675558  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:47.600819  7029 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.601394  6984 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:47.677966  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:47.681416  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:47.688163  5072 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:47.690554  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:47.691414  5075 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:47.692364  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.688988  6831 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:23:47.694867  6984 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.696180  6984 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:47.589958  6965 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:47.598907  6952 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:47.706507  6967 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:47.711563  6967 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.687831  5941 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:47.688410  5144 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:47.702250  7029 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.747795  6965 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:47.750643  6952 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:47.752877  5944 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:47.687706  6717 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:23:47.758028  5945 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:23:47.821898  6967 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.760020  7029 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:47.889889  5149 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.010s	sys 0.071s
I20260709 12:23:47.759678  7055 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:47.892189  5149 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.011s	sys 0.071s
W20260709 12:23:47.896586  5189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.018s	sys 0.065s
W20260709 12:23:47.899637  5189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.018s	sys 0.066s
W20260709 12:23:47.899021  5188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.031s	sys 0.053s
W20260709 12:23:47.901468  5188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.032s	sys 0.053s
W20260709 12:23:47.903832  5190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.037s	sys 0.037s
W20260709 12:23:47.904268  5190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.037s	sys 0.037s
W20260709 12:23:47.904575  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 603145us
I20260709 12:23:47.910782  5948 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:47.905278  6967 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.911563  7064 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.915736  6812 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:47.917717  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:47.920483  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:47.914266  6688 consensus_queue.cc:1059] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:23:47.925845  7064 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:47.931573  6877 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.911948  7056 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:23:47.944416  7064 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:47.953019  7064 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:47.955390  7066 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:47.955874  7066 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:47.926151  6967 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:47.949777  6688 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:47.956845  7066 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:47.952054  6877 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:47.963418  5075 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:47.971040  6688 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:47.972426  6688 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:47.974120  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.974784  7050 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.975420  7050 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.975719  6717 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.976552  6717 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.977936  7050 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.985044  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:47.985951  5075 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:47.966295  7064 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:47.990047  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:47.991847  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 692624us
I20260709 12:23:47.992198  6929 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:47.993958  7025 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:47.994187  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:47.994626  6967 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:47.994793  7025 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:47.995261  7030 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.995412  7025 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:47.995779  7030 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:47.996227  6877 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:47.997260  7064 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:47.980981  6717 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:47.991683  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:48.001178  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 83583.1us
I20260709 12:23:48.003502  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:48.003669  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 83254.1us
I20260709 12:23:47.997754  6876 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:48.002028  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:48.034643  5076 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:47.988708  7066 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:48.024869  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:47.989415  7029 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:47.984551  7055 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:47.998023  7030 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:48.029691  5188 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:48.031981  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:47.993026  6929 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:48.035293  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:48.041669  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.999953  6991 raft_consensus.cc:697] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:48.039314  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:48.041601  5224 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 2.
I20260709 12:23:48.036581  5072 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:23:48.043390  5220 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:48.034906  6967 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.045759  5223 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:48.034407  6876 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:48.039325  5218 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:48.051165  5076 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:48.050469  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.053158  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.053745  5222 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:48.055866  5188 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:48.044639  7030 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.058220  5188 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:48.043748  7025 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:48.060981  7055 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:48.061441  7009 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.062490  7050 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.064800  5222 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:48.066152  5220 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:48.068248  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.068581  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.069546  5219 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:48.045630  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.052461  6876 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:48.071900  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.074379  5222 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:48.075194  5221 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:48.079252  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.080001  5221 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:48.060447  7075 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:48.083010  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.072098  7030 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:48.084573  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:48.085225  7030 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:48.052265  6967 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:48.091303  5218 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:48.086503  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:48.072263  7071 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:23:48.089999  7070 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:23:48.121593  5221 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:48.123401  5220 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 12:23:48.079960  7025 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.130102  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.108877  5217 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:48.099356  6990 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:48.090618  5216 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:48.098829  7006 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:48.134006  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:48.155985  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:48.159406  7049 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.989s	user 0.000s	sys 0.012s
I20260709 12:23:48.160669  6929 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:48.163216  5071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 307) took 1170 ms. Trace:
I20260709 12:23:48.169932  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:48.170379  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.170553  5075 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:48.170919  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.171526  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:48.172353  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.172562  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:48.172937  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.174431  6717 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:48.174808  6816 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.174882  6717 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:48.175451  6717 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:48.175400  6816 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:48.176793  5898 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:48.085709  6991 consensus_queue.cc:237] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:48.179673  6349 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:48.180416  6349 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:48.182376  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.138976  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.183233  5220 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:48.184073  5899 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:48.184345  5146 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:48.139899  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.163751  5071 rpcz_store.cc:276] 0709 12:23:46.993104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.993873 (+   769us) service_pool.cc:224] Handling call
0709 12:23:46.994291 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:46.994732 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:46.994763 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:46.995339 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 12:23:47.133966 (+138627us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:47.134025 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.134124 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:48.102834 (+968710us) spinlock_profiling.cc:243] Waited 8.31 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:23:48.162796 (+ 59962us) raft_consensus.cc:1685] finished
0709 12:23:48.162828 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:48.163134 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1755,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"spinlock_wait_cycles":19116928,"thread_start_us":138197,"threads_started":1,"wal-append.queue_time_us":160666}
I20260709 12:23:48.186080  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:48.186748  5145 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:48.131034  5072 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:48.114261  7023 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:48.081260  7009 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:48.115029  6649 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:48.190282  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.188304  5075 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
W20260709 12:23:48.169301  7069 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:23:48.186895  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:48.187611  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.185452  5221 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:48.196213  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.196918  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.190971  5146 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:48.120941  7071 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:48.191280  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:48.192080  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.202492  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.203438  5071 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:23:48.204205  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.204788  5147 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:48.205711  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:48.206264  5221 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:48.211247  5145 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:48.135708  7025 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:48.218674  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.189595  6929 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:48.198858  5150 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:23:48.187717  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.198129  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.195881  6816 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:48.199263  6970 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:48.196825  6987 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.156612  6990 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:48.230150  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:48.193714  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:48.241665  5220 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:48.254124  5150 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:48.211335  6650 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:23:48.244972  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 58156.4us
I20260709 12:23:48.206399  7071 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:48.249111  5221 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:48.251247  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:48.254499  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 60909.6us
I20260709 12:23:48.158254  7006 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:48.211122  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:48.257653  5220 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:48.190407  7023 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:48.196985  6649 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:48.273573  7025 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:48.238473  7005 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:23:48.238292  6970 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:23:48.275557  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.257280  6987 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:48.268714  7014 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:48.280383  5220 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:48.210688  6654 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:48.221926  6984 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:48.281298  5114 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:48.200006  6831 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:23:48.271133  5141 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:48.200037  6927 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:23:48.222078  6889 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:48.275152  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.242969  6929 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:48.304863  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 119625us
I20260709 12:23:48.306495  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.307618  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:48.286898  6650 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:48.228348  6974 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:23:48.287748  6816 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:23:48.331457  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 129809us
I20260709 12:23:48.306411  6970 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:48.273379  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:48.355510  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 74287us
I20260709 12:23:48.359632  5899 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:48.385766  5114 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:48.344187  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.383985  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:48.379151  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:48.302701  6812 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.329589  6654 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:48.301362  7064 consensus_queue.cc:1059] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 12:23:48.332767  6984 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:48.332005  7014 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:48.463608  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:48.464237  5900 leader_election.cc:336] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.678s (SENT)
W20260709 12:23:48.441699  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 82171.1us
I20260709 12:23:48.469231  5899 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:48.469858  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.358850  6995 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:48.362988  6991 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:48.418953  6967 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:48.454157  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.385779  7030 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.385571  7080 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:48.495348  7030 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:48.458212  6654 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:48.377419  7025 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:23:48.504154  7082 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 12:23:48.504385  7096 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:48.459208  7021 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.383400  6921 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:48.357586  6889 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:48.386945  6349 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:48.491083  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 105401us
I20260709 12:23:48.450096  6985 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:48.461510  6984 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:48.450141  6812 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:48.453007  7086 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 12:23:48.520315  7021 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:48.520843  7087 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:23:48.531032  5941 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:23:48.531481  5941 pending_rounds.cc:85] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 1
I20260709 12:23:48.531724  5941 pending_rounds.cc:107] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:23:48.531993  5941 raft_consensus.cc:2889] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:48.522356  6649 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:48.468869  7014 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:48.486373  6949 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 2.797s	user 0.010s	sys 0.010s
I20260709 12:23:48.484994  5141 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:48.518389  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.479733  6953 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:48.567130  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.518565  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:48.533254  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.537492  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:48.538697  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 804) took 3339 ms. Trace:
I20260709 12:23:48.522737  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.497746  7080 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:48.501951  6995 raft_consensus.cc:697] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:48.573654  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.574084  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.568507  5149 raft_consensus.cc:2376] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6. Re-sending same reply.
I20260709 12:23:48.579679  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.479518  6886 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:23:48.519095  6831 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:48.581800  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.569383  5143 rpcz_store.cc:276] 0709 12:23:45.198870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.200871 (+  2001us) service_pool.cc:224] Handling call
0709 12:23:45.245417 (+ 44546us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:45.264095 (+ 18678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:45.264143 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:45.289975 (+ 25832us) log.cc:844] Serialized 52 byte log entry
0709 12:23:45.428934 (+138959us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:45.763594 (+334660us) log.cc:844] Serialized 14 byte log entry
0709 12:23:46.234088 (+470494us) log.cc:844] Serialized 14 byte log entry
0709 12:23:46.236133 (+  2045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.236255 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:47.204792 (+968537us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b1800434b00. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a541c2a6 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db
0709 12:23:48.286729 (+1081937us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:23:48.538416 (+251687us) raft_consensus.cc:1685] finished
0709 12:23:48.538442 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:48.538609 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57331,"prepare.queue_time_us":333640,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":4010,"raft.queue_time_us":1887,"raft.run_cpu_time_us":508,"raft.run_wall_time_us":12334,"spinlock_wait_cycles":37277824,"thread_start_us":798610,"threads_started":3,"wal-append.queue_time_us":355632}
I20260709 12:23:48.584312  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.585140  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.585318  6995 consensus_queue.cc:237] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:48.585608  6717 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:48.587288  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.588177  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.620911  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.586400  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.626636  5149 raft_consensus.cc:2410] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:23:48.544049  7023 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:48.523330  6889 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:48.541685  7009 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:48.627720  5150 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
I20260709 12:23:48.490206  6991 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:48.628535  5145 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:48.527446  6812 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:48.574154  6953 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:48.586400  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.586696  6989 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:48.586930  6843 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:48.633786  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 254681us
I20260709 12:23:48.587177  7029 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.635915  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.657740  5041 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:48.647504  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 184006us
I20260709 12:23:48.628473  6990 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:48.518648  6650 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:48.655357  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.629503  7096 raft_consensus.cc:697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:48.619647  7097 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 12:23:48.656826  7029 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:48.666263  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:48.666917  5148 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:48.649426  6953 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:48.646430  7023 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.675227  5146 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:48.664830  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:48.652944  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 407) took 1067 ms. Trace:
I20260709 12:23:48.682199  5942 rpcz_store.cc:276] 0709 12:23:47.583764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.588439 (+  4675us) service_pool.cc:224] Handling call
0709 12:23:48.584968 (+996529us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:48.651298 (+ 66330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7228288}
I20260709 12:23:48.682943  5943 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:48.690392  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.676441  6717 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:48.691545  7098 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:48.692401  7097 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:48.693188  7097 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:48.685747  6889 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:48.688681  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 404) took 1117 ms. Trace:
I20260709 12:23:48.663676  6843 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:48.696628  7080 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:48.697440  6929 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:48.694830  6889 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:48.698390  7096 consensus_queue.cc:237] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:48.700069  6889 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:48.700217  6984 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:48.702798  6991 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:48.683636  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:48.765499  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 85512.4us
W20260709 12:23:48.784094  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 108289us
I20260709 12:23:48.785898  5041 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:48.788300  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:48.800776  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.810778  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:48.834558  5148 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:48.669510  7025 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:48.670992  7030 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:48.843205  5945 raft_consensus.cc:2393] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:48.843608  5900 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:48.844802  5900 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:48.847090  5900 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:48.674785  5144 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:48.789471  7087 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.859300  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:48.826021  7098 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:48.840085  7088 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:48.829922  6349 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:48.789425  5941 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 2.
I20260709 12:23:48.854255  6985 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:48.766528  7023 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:48.912703  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.908706  6927 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:48.853601  6969 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:48.915318  5148 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:23:48.927260  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 68035.9us
I20260709 12:23:48.911226  7101 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:48.905503  6886 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:48.697772  5943 rpcz_store.cc:276] 0709 12:23:47.570331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.572195 (+  1864us) service_pool.cc:224] Handling call
0709 12:23:48.273230 (+701035us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416eee 000000000036eb39 00007fd8ad48a1e0 00007fd8ad5530c4 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:48.687977 (+414747us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17339136}
I20260709 12:23:48.935921  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 408) took 1345 ms. Trace:
I20260709 12:23:48.947860  5946 rpcz_store.cc:276] 0709 12:23:47.590894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.591107 (+   213us) service_pool.cc:224] Handling call
0709 12:23:48.912495 (+1321388us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:48.935843 (+ 23348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31106944}
W20260709 12:23:48.928673  7071 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.651s	user 0.014s	sys 0.005s
W20260709 12:23:48.916599  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 69791.6us
I20260709 12:23:48.939560  5148 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 12:23:48.950001  5148 pending_rounds.cc:85] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 1
I20260709 12:23:48.950366  5148 pending_rounds.cc:107] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:23:48.950752  5148 raft_consensus.cc:2889] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:48.951705  5148 pending_rounds.cc:107] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:23:48.952054  5148 raft_consensus.cc:2889] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:48.953409  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.953526  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.954728  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.955828  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:48.939210  7025 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:48.957213  7100 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 12:23:48.958298  7063 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 1.052s	user 0.007s	sys 0.002s
I20260709 12:23:48.936802  6990 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:48.938283  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.966027  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.911666  7103 consensus_queue.cc:1059] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:23:48.967598  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.967957  7103 raft_consensus.cc:3055] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:48.968575  7103 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:48.967600  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:48.971472  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.972961  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:48.937580  6927 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:48.919418  7087 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:48.976612  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.965278  5942 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
W20260709 12:23:48.950754  6876 consensus_meta.cc:287] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.851s	user 0.013s	sys 0.000s
I20260709 12:23:48.983124  6886 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:23:48.986753  7104 log.cc:927] Time spent T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:23:48.989252  6717 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:23:48.991441  6970 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.570s	user 0.006s	sys 0.005s
I20260709 12:23:48.978739  6969 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.003218  6970 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:49.003727  6990 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.978435  6349 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:23:49.004335  6990 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:49.005854  5076 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:48.999431  6876 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:49.007428  6970 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:48.987264  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.008584  6876 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:49.009354  5073 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:49.010113  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.012468  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.013851  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.014046  7023 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:49.014299  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.015622  7014 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:49.016392  7107 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:23:48.969034  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:48.993854  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 43863.7us
I20260709 12:23:49.017566  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:49.017716  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:49.018276  7014 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:49.018702  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:49.019343  5150 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:49.019718  5114 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:48.951345  7101 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:49.020344  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:49.020953  5150 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:49.022194  7101 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:49.022608  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.022876  7101 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:49.023840  5068 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:48.998721  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:49.002322  6717 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:48.970548  6985 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:23:48.936103  7106 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.019168  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:49.024292  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.024519  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:49.046052  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.043313  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.048152  5946 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:49.003507  6969 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.035372  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 405) took 1459 ms. Trace:
I20260709 12:23:49.049935  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.043871  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.051126  5070 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:49.051424  6969 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:49.051489  6717 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:49.042403  7105 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:23:49.036947  7006 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:49.044816  7106 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.043377  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.053712  6953 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.054177  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.054617  7106 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:49.055003  5940 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:49.047469  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.055905  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:48.982249  7103 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:49.046056  5069 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.008503  6990 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:49.033984  7021 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:48.981688  6927 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:48.981600  5148 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:48.923832  7098 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:49.031332  7072 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.993s	user 0.004s	sys 0.016s
I20260709 12:23:49.048337  5141 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.052171  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:49.029703  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.050902  5068 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:48.978057  7071 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:49.040010  7087 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:49.050025  5944 rpcz_store.cc:276] 0709 12:23:47.575725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.575910 (+   185us) service_pool.cc:224] Handling call
0709 12:23:47.624424 (+ 48514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:47.760503 (+136079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.034987 (+1274484us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400006754. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:49.035280 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294180096}
W20260709 12:23:49.107074  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.073417  5073 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:49.058974  5900 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:49.091195  5147 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:49.117887  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:49.074832  5898 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:49.118718  5073 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.107299  5071 raft_consensus.cc:2410] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:49.107657  5114 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.119480  5141 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:49.120292  5114 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.061383  5943 raft_consensus.cc:1275] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:49.121155  7021 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:49.121634  7021 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:49.059527  7029 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:49.122104  7021 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.122665  7025 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:49.123148  7025 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:49.123519  7025 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.065267  5148 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:49.130196  6984 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:49.060065  6987 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:49.135427  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 128482us
I20260709 12:23:49.136863  5223 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
W20260709 12:23:49.138378  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 79484.5us
I20260709 12:23:49.140180  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:49.057554  6831 consensus_queue.cc:1059] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 12:23:49.142401  6299 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:23:49.137939  6654 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.639s	user 0.014s	sys 0.000s
I20260709 12:23:49.142900  6299 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:49.142251  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.143414  6299 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:49.143260  7021 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:49.143281  6654 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:49.147832  6654 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:49.148509  5070 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:49.149762  7098 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:49.150595  5070 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:23:49.151126  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 76429.2us
W20260709 12:23:49.151623  7115 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:23:49.151667  7071 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:49.152276  5070 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:49.140175  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.154780  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.155925  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:49.158404  5073 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:49.160953  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:49.161770  7021 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:49.180369  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.183139  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:49.184238  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:23:49.185920  5948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 413) took 1510 ms (client timeout 1891 ms). Trace:
I20260709 12:23:49.186317  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:49.186408  5948 rpcz_store.cc:269] 0709 12:23:47.675745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.677685 (+  1940us) service_pool.cc:224] Handling call
0709 12:23:49.185750 (+1508065us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b54002d78d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:49.185856 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212154240}
I20260709 12:23:49.187076  5219 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:49.187574  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.167462  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:49.188642  6806 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.188817  5190 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.189736  7050 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.190447  7050 consensus_queue.cc:1243] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer ceec344dd00f40c18ef58414ce147016 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:23:49.190959  6843 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:49.192466  7050 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:49.192919  7050 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:49.185969  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.195864  5224 raft_consensus.cc:2410] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:23:49.195868  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.150619  6962 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.169926  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:49.208040  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:49.208438  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:49.154150  6943 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:49.215674  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.216464  5215 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
W20260709 12:23:49.217906  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 566628us
I20260709 12:23:49.219147  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.219342  6843 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:49.219918  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.220682  5189 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.225718  6962 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:49.227240  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:49.228118  6962 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:49.228425  5142 raft_consensus.cc:2393] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:49.229144  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 843) took 1016 ms. Trace:
I20260709 12:23:49.232550  6927 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.234061  6688 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:49.235134  7071 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:49.234892  7025 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:49.235977  7071 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:49.056074  6953 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:49.165783  6831 consensus_queue.cc:1243] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Peer ceec344dd00f40c18ef58414ce147016 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:23:49.242254  5142 rpcz_store.cc:276] 0709 12:23:48.212446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.212819 (+   373us) service_pool.cc:224] Handling call
0709 12:23:49.227001 (+1014182us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:49.229049 (+  2048us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5440896}
I20260709 12:23:49.171339  6969 consensus_queue.cc:1059] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:49.172119  6886 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:49.160971  6717 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:49.153723  6804 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:49.173674  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:49.174252  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:49.174449  7029 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:49.168857  6654 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:49.157047  6349 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:49.176645  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 852) took 1047 ms. Trace:
W20260709 12:23:49.271678  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 97516.4us
I20260709 12:23:49.276678  5221 rpcz_store.cc:276] 0709 12:23:48.129086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.270033 (+140947us) service_pool.cc:224] Handling call
0709 12:23:48.270342 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:48.270589 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:48.272859 (+  2270us) log.cc:844] Serialized 14 byte log entry
0709 12:23:48.380206 (+107347us) log.cc:844] Serialized 14 byte log entry
0709 12:23:49.176059 (+795853us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b18002c7ca0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a53c8c0a 00007fd8aeb08219 00007fd8a5450a40 00007fd8a544b4e4 00007fd8a544b455 00007fd8a8533060 00007fd8a8545716 00007fd8a8556287 00007fd8a8556208
0709 12:23:49.176256 (+   197us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:49.176320 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:49.176369 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.176453 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:49.176579 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":791694,"prepare.queue_time_us":1939,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":3075,"raft.queue_time_us":796852,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":4100,"spinlock_wait_cycles":6187904,"thread_start_us":82199,"threads_started":1,"wal-append.queue_time_us":304319}
I20260709 12:23:49.280057  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.177230  7064 raft_consensus.cc:3055] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:49.198066  5069 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:49.198942  5141 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:49.201568  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.283320  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.209053  5224 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:49.284252  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 82792.7us
I20260709 12:23:49.285359  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.285892  6962 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.286504  6962 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:49.287070  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.289059  6962 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:49.289265  6816 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:49.289880  7106 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:49.290380  7073 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:49.290553  7106 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:49.291391  7073 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:49.292089  7073 raft_consensus.cc:697] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:49.193315  7050 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:49.293514  7073 consensus_queue.cc:237] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:49.211699  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.210232  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:49.239435  5148 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:49.238232  5946 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:49.215378  5222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 851) took 1126 ms. Trace:
I20260709 12:23:49.282953  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.239718  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.299413  5069 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
W20260709 12:23:49.299906  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 60275us
I20260709 12:23:49.300885  5115 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:49.302646  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:49.249603  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 53798.6us
I20260709 12:23:49.237231  7025 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:49.254516  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:49.189204  6806 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:49.257031  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:49.265997  7103 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:49.217481  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.268776  6299 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:49.270416  7029 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:49.289938  6816 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:49.304442  7066 consensus_queue.cc:1059] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:49.248303  7021 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:49.297463  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 85871.8us
I20260709 12:23:49.302763  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.268812  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 221) took 1204 ms. Trace:
I20260709 12:23:49.305508  5041 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.314513  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:49.291983  7106 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:49.271363  6877 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.326547  5224 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:49.324527  5041 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.313215  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.309409  5073 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:49.322769  5223 rpcz_store.cc:276] 0709 12:23:48.064584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.064694 (+   110us) service_pool.cc:224] Handling call
0709 12:23:48.064959 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.137365 (+1072406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.268515 (+131150us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400553a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:49.268741 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2464586496}
I20260709 12:23:49.298858  5222 rpcz_store.cc:276] 0709 12:23:48.089104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.093869 (+  4765us) service_pool.cc:224] Handling call
0709 12:23:48.094308 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:48.094505 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:23:48.191389 (+ 96884us) log.cc:844] Serialized 14 byte log entry
0709 12:23:49.215007 (+1023618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:49.215068 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:49.215121 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.215188 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:49.215317 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":590121,"prepare.queue_time_us":36747,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3612,"raft.queue_time_us":590985,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":529588,"threads_started":2,"wal-append.queue_time_us":576839}
I20260709 12:23:49.328622  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:49.322312  5900 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.329609  5067 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 6: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:49.328751  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.328658  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.330317  5222 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:49.332274  5067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 801) took 1165 ms. Trace:
I20260709 12:23:49.330282  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:49.331532  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:49.334389  5223 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.276465  6349 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:49.338706  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.332676  5067 rpcz_store.cc:276] 0709 12:23:48.166865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.173887 (+  7022us) service_pool.cc:224] Handling call
0709 12:23:49.328444 (+1154557us) spinlock_profiling.cc:243] Waited 8.88 ms on lock 0x7b5800560198. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:49.332212 (+  3768us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20419456}
I20260709 12:23:49.336606  5041 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.317307  7006 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:49.342010  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:49.317826  7025 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:49.311844  7064 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:49.367141  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.365042  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.317169  7029 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:49.365042  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:49.262121  6717 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:49.323905  6816 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.328612  7114 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:49.343765  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.310976  6843 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:49.392100  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:49.371635  5223 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:49.391521  5072 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:49.392825  5149 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.368688  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.326570  6962 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:49.308899  6806 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:49.396369  5221 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:49.396960  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 83877.7us
I20260709 12:23:49.331528  6804 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:49.311169  6654 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:49.399263  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 786) took 1184 ms. Trace:
I20260709 12:23:49.399380  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:49.396896  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.386731  7006 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.344537  6349 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:49.330137  7021 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:49.343453  7014 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:49.369325  7032 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:49.397262  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.331395  6877 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.391275  7066 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:49.402503  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:49.467530  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 65099.6us
I20260709 12:23:49.405458  6843 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:49.378760  5942 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:49.428575  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.451179  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:49.469738  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.470057  5223 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:49.470592  7055 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:49.471078  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.472723  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.473425  5222 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 2.
I20260709 12:23:49.473603  7055 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:49.474134  7055 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:49.475020  7055 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.366465  6927 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:49.400787  6962 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:49.425150  6876 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.402221  5145 rpcz_store.cc:276] 0709 12:23:48.214492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.640110 (+425618us) service_pool.cc:224] Handling call
0709 12:23:48.640614 (+   504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:48.649238 (+  8624us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:23:48.969994 (+320756us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b18003725e0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8abf333c7 00007fd8a85459a7 00007fd8a85550bf 00007fd8a8555038
0709 12:23:49.102722 (+132728us) log.cc:844] Serialized 14 byte log entry
0709 12:23:49.258491 (+155769us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:49.312825 (+ 54334us) log.cc:844] Serialized 26 byte log entry
0709 12:23:49.313031 (+   206us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:49.313071 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.313156 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:49.391977 (+ 78821us) raft_consensus.cc:1685] finished
0709 12:23:49.399018 (+  7041us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:49.399184 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17307,"prepare.queue_time_us":342346,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":27753,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":11674,"spinlock_wait_cycles":65719040,"thread_start_us":246850,"threads_started":2,"wal-append.queue_time_us":204259}
I20260709 12:23:49.427404  7006 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:49.453354  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.390201  6974 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:23:49.341143  6970 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:49.331769  6720 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.117s
W20260709 12:23:49.509655  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.402666  7114 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:23:49.514688  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 97693.3us
I20260709 12:23:49.476635  7056 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:49.378010  7087 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 12:23:49.490869  5224 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:49.493279  5072 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:49.484716  6877 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:49.466641  7096 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:49.404816  6804 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.488829  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.477428  7014 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:49.488040  6962 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:49.538285  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.539578  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 233) took 1437 ms. Trace:
I20260709 12:23:49.540041  5217 rpcz_store.cc:276] 0709 12:23:48.102228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.157220 (+ 54992us) service_pool.cc:224] Handling call
0709 12:23:49.538043 (+1380823us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:49.539486 (+  1443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56772864}
I20260709 12:23:49.541668  7066 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:49.542181  7066 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:49.542727  7066 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.542099  7114 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:49.544270  6299 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:49.546798  7021 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:49.518155  5069 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:49.531473  6804 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.485754  5215 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:49.501703  5221 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
W20260709 12:23:49.535356  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 58029.4us
I20260709 12:23:49.537007  7014 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.481419  6654 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:49.549799  7119 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:23:49.560988  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.563258  5069 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:49.520007  6970 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:49.492281  6876 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.477433  7029 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:49.581984  7029 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.587338  7066 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.479090  7064 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:49.588919  7064 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:49.599649  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.471781  6843 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:49.623617  6311 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:49.630681  6311 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:49.624315  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.641904  5074 raft_consensus.cc:1275] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:49.656636  5149 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:49.643975  7088 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:49.665364  6311 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:49.643989  6717 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:49.671578  7126 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:23:49.673627  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.675451  6349 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:49.651213  6970 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:49.682178  7128 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 12:23:49.683871  7134 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:23:49.646775  7050 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:49.691615  7050 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:49.692507  7106 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.693118  7106 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.696627  7021 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.697417  7021 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.699674  7021 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:49.702970  5222 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:49.695657  7106 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:49.706593  5222 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:49.714107  5217 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:49.722601  7066 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:49.723865  6876 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:49.737649  6876 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.738498  6876 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:49.743043  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.743891  5067 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:49.751833  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.753122  5218 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:49.754401  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 263) took 1299 ms. Trace:
I20260709 12:23:49.755017  5147 rpcz_store.cc:276] 0709 12:23:48.455282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.029429 (+574147us) service_pool.cc:224] Handling call
0709 12:23:49.029930 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.118220 (+ 88290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.754033 (+635813us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b54000ebd54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:49.754318 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140176768}
I20260709 12:23:49.756704  5188 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.758121  7066 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:49.761420  5188 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.768121  6876 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:49.768518  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.772097  6804 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:49.773631  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.774933  5067 raft_consensus.cc:2410] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:23:49.776827  6876 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.777657  6876 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.780522  6876 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:49.780998  5069 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:49.784729  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:49.786327  5069 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:23:49.795047  5067 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 6: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:49.798820  6962 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.802824  5071 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:49.804932  6962 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:49.763490  6349 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:49.712217  6816 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:49.805842  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.789912  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.807996  5071 raft_consensus.cc:2393] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:49.788646  5222 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:23:49.809687  5222 pending_rounds.cc:85] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 1
I20260709 12:23:49.811154  5222 pending_rounds.cc:107] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:23:49.810957  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:49.811622  5222 raft_consensus.cc:2889] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:49.811014  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.816529  5218 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:49.786485  7088 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.821516  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:49.828915  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.830763  5114 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.812939  6806 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:49.794446  7006 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.825924  7056 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:49.836632  7023 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:49.840409  7099 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:49.835831  6876 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:23:49.824537  7088 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:49.856093  5041 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.834722  7021 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:23:49.834460  7055 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:23:49.830896  7064 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:23:49.861631  6804 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:49.849085  7023 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:49.869273  7096 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:49.870134  7096 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:49.869395  7023 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:49.871479  7006 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:49.866096  6970 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:49.853400  7099 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:49.877178  7088 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:49.877663  7099 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:49.880540  6816 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:49.883533  5075 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:49.884261  6969 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:49.884727  6969 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:49.886193  6943 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:49.886941  6943 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:49.889235  5900 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:49.890610  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.891450  5222 raft_consensus.cc:2393] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:23:49.892355  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.893567  6804 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:49.894126  6804 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:49.895025  6804 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.895673  5216 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:49.897207  7106 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:49.900415  6969 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.866907  5041 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.904435  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.906255  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.909835  5189 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.911345  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.912223  5948 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:49.911414  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:49.915310  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.918702  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:49.901896  6969 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.936830  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.938076  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.938951  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.939129  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.939689  5946 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:49.940094  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:49.940526  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:49.941121  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:49.928129  5041 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.901187  6943 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:49.932931  5070 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:23:49.934173  5115 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:49.946564  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:49.947630  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.948067  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.952445  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:49.953491  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.954077  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.956033  5898 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:49.959543  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.959117  5899 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.960728  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:49.946451  5070 pending_rounds.cc:85] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 2
I20260709 12:23:49.961433  5070 pending_rounds.cc:107] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:23:49.961818  5070 raft_consensus.cc:2889] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:49.962231  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.957481  7006 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:23:49.966112  6966 consensus_queue.cc:1059] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:49.966602  6438 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:49.967067  6438 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:49.967432  6966 raft_consensus.cc:3055] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:49.967748  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.967907  6966 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:49.969498  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.970081  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.970384  6966 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:49.970916  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:49.971158  7101 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:49.971544  7101 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:49.971958  7101 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:49.971750  5939 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:49.972553  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 415) took 2233 ms. Trace:
W20260709 12:23:49.974583  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 64873.3us
I20260709 12:23:49.970952  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.975594  5944 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:23:49.977246  5899 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.978683  5899 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:49.980811  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.981519  5115 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.984732  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.986040  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:49.986192  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.988643  5948 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:49.988709  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:49.971701  6966 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:49.935065  7098 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:49.945731  6831 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:49.973026  5947 rpcz_store.cc:276] 0709 12:23:47.719156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.720589 (+  1433us) service_pool.cc:224] Handling call
0709 12:23:47.725849 (+  5260us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:48.334823 (+608974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:48.334872 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:48.379969 (+ 45097us) log.cc:844] Serialized 52 byte log entry
0709 12:23:48.628362 (+248393us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:48.639769 (+ 11407us) log.cc:844] Serialized 14 byte log entry
0709 12:23:48.661378 (+ 21609us) log.cc:844] Serialized 14 byte log entry
0709 12:23:48.661649 (+   271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.123033 (+461384us) spinlock_profiling.cc:243] Waited 88.3 ms on lock 0x7b1800215f80. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a541c2a6 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e522 00007fd8a54507c1
0709 12:23:49.124205 (+  1172us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b1800215f80. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e522 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:23:49.952252 (+828047us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54001a0254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:49.952310 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:49.952361 (+    51us) raft_consensus.cc:1685] finished
0709 12:23:49.952374 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:49.952537 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":355563,"prepare.queue_time_us":495570,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":333340,"raft.run_cpu_time_us":388,"raft.run_wall_time_us":26921,"spinlock_wait_cycles":643213312,"thread_start_us":298621,"threads_started":2,"wal-append.queue_time_us":255559,"wal-append.run_cpu_time_us":3842,"wal-append.run_wall_time_us":1253052}
I20260709 12:23:49.943645  7066 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:50.010133  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:50.010658  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:50.009310  5946 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:49.967434  6438 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:49.958240  6654 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:50.034615  5946 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:49.957736  6299 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.040735  6299 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:49.987679  6720 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:49.991664  7071 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:50.008286  7098 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:49.987044  7006 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:50.015811  5224 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:50.041041  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:50.053922  5947 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:49.981176  6974 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:49.959054  6943 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:50.011027  5948 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:49.985421  7103 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 12:23:49.999323  7025 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:49.996119  6876 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:23:50.060108  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.985653  6995 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 12:23:50.066412  6299 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:50.057987  7098 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:50.010977  7074 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 12:23:50.066411  7009 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:50.071071  7023 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:50.120414  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 141814us
W20260709 12:23:50.128153  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 139538us
I20260709 12:23:50.130048  5115 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:50.084836  6974 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:50.131783  5899 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:23:50.136538  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 179569us
I20260709 12:23:50.142745  5899 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:50.138500  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:50.148180  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:50.127877  7025 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:50.137848  6886 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:50.133100  6974 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.150784  5943 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:50.142356  6876 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:50.151659  7021 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:50.163169  6886 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.166356  6974 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:50.191259  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 52236.4us
I20260709 12:23:50.193358  6299 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:50.194944  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:50.176649  6886 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:50.166671  7025 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:50.218008  7152 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:23:50.224005  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:50.244768  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:50.245922  6299 raft_consensus.cc:697] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:50.226024  6831 raft_consensus.cc:3055] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:50.227317  7025 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:50.227473  7009 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:50.228307  7023 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:50.187960  7021 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:23:50.258464  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:50.258857  7166 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:23:50.201558  7029 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:50.259848  7103 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:50.272310  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:50.274351  5071 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:50.238852  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 335) took 1171 ms. Trace:
I20260709 12:23:50.277174  5068 rpcz_store.cc:276] 0709 12:23:49.067940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.145342 (+ 77402us) service_pool.cc:224] Handling call
0709 12:23:49.786739 (+641397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.797784 (+ 11045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:49.797837 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:49.797921 (+    84us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:49.797972 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.798060 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:50.238595 (+440535us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b5400196250. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:50.238759 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1471001856}
I20260709 12:23:50.269693  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:50.214956  6876 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:50.255025  6299 consensus_queue.cc:237] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:50.281813  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:50.288719  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:50.250406  7074 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:50.292060  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 115605us
I20260709 12:23:50.250943  6831 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:50.297834  6889 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:50.300141  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:50.302690  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 330) took 1997 ms. Trace:
I20260709 12:23:50.282531  5220 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:50.305362  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:50.306627  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 158529us
I20260709 12:23:50.307726  5115 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:50.310225  5115 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:50.303249  5076 rpcz_store.cc:276] 0709 12:23:48.304770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.490571 (+185801us) service_pool.cc:224] Handling call
0709 12:23:48.491064 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.006457 (+515393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.302324 (+1295867us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b540037f354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:50.302608 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1183333632}
I20260709 12:23:50.248672  6438 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:50.283630  6876 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:50.263947  7099 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:50.320027  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:50.319371  5115 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:50.302174  6889 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:50.301841  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 499) took 1725 ms (client timeout 1513 ms). Trace:
I20260709 12:23:50.315017  6654 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:50.346639  5939 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
W20260709 12:23:50.342533  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 184386us
I20260709 12:23:50.285938  7096 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:50.337590  6876 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:50.300416  7074 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:50.539363  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 235595us
W20260709 12:23:50.578279  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 231188us
I20260709 12:23:50.325557  7023 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:50.322043  7021 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:50.310969  7103 consensus_queue.cc:1059] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
W20260709 12:23:50.547047  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 230535us
I20260709 12:23:50.588218  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:50.557734  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:50.589439  5189 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:50.542515  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:50.563331  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 662) took 1341 ms. Trace:
W20260709 12:23:50.563330  7119 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.218s	user 0.001s	sys 0.009s
I20260709 12:23:50.591354  5221 rpcz_store.cc:276] 0709 12:23:49.222397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.549448 (+327051us) service_pool.cc:224] Handling call
0709 12:23:49.549882 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:49.565576 (+ 15694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:49.565609 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:49.566087 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:23:49.651239 (+ 85152us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:49.663963 (+ 12724us) log.cc:844] Serialized 14 byte log entry
0709 12:23:49.664209 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.887344 (+223135us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b5400163d54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:49.887399 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:50.296702 (+409303us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b18003d7260. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:23:50.563058 (+266356us) raft_consensus.cc:1685] finished
0709 12:23:50.563089 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:50.563263 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7377,"prepare.queue_time_us":55871,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":13792,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":9318,"spinlock_wait_cycles":22381184,"thread_start_us":82118,"threads_started":1,"wal-append.queue_time_us":146907}
I20260709 12:23:50.596726  5189 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:50.611032  5189 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:50.614989  7050 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:50.561503  7025 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:50.573980  7174 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:50.624406  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:50.568326  7006 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:50.452077  6831 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:50.577451  6889 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:50.577307  6966 consensus_meta.cc:287] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.564s	user 0.030s	sys 0.035s
I20260709 12:23:50.624218  5942 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:50.623540  7066 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:50.617209  7055 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:50.628583  7023 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:50.644488  7073 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:23:50.556622  5941 rpcz_store.cc:269] 0709 12:23:48.574647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.579460 (+  4813us) service_pool.cc:224] Handling call
0709 12:23:50.286973 (+1707513us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b5400240c54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:50.299708 (+ 12735us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52536704}
I20260709 12:23:50.650293  7025 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:50.679615  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 91519.5us
I20260709 12:23:50.634392  7050 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:23:50.654230  7175 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:50.685423  7073 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:50.641767  7174 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:50.666724  7006 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:50.676213  7023 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:50.687126  7066 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:23:50.720009  7175 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:50.744140  6953 raft_consensus.cc:3055] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:50.745776  7099 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:50.762630  7098 consensus_meta.cc:287] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.646s	user 0.009s	sys 0.002s
I20260709 12:23:50.681213  5115 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:50.729342  6438 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:50.838212  6953 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:50.732136  7006 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:50.864825  7023 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:50.753643  7096 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:50.801437  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:50.757848  7101 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.765s	user 0.014s	sys 0.000s
W20260709 12:23:50.744263  7170 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
W20260709 12:23:50.739797  7171 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:23:50.762900  7174 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:50.764952  7175 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:50.759764  7176 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 12:23:50.760453  7181 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 12:23:50.905030  6886 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.663s	user 0.011s	sys 0.017s
I20260709 12:23:50.877990  7074 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:50.751395  7073 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:50.879347  6649 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:50.911871  6649 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:50.912045  6876 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:50.881242  7098 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:50.890386  6953 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:50.900601  5073 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:50.890934  7101 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:50.905941  6886 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:50.915098  7099 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:23:50.917199  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 272299us
W20260709 12:23:50.917614  6974 consensus_meta.cc:287] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.697s	user 0.015s	sys 0.006s
I20260709 12:23:50.918002  6889 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.031793  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:51.033041  6889 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:50.921025  6969 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.966s	user 0.014s	sys 0.010s
I20260709 12:23:50.921226  7006 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:50.922269  6953 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:50.925037  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 861) took 1623 ms. Trace:
I20260709 12:23:51.034559  5070 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:51.034327  5218 rpcz_store.cc:276] 0709 12:23:49.301277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.760427 (+459150us) service_pool.cc:224] Handling call
0709 12:23:49.760878 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.817136 (+ 56258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.924384 (+1107248us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b54008502d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:50.924741 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112338688}
W20260709 12:23:51.035727  7169 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.699s	user 0.004s	sys 0.007s
I20260709 12:23:50.928092  6876 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:50.929667  7101 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:50.917037  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 235827us
I20260709 12:23:51.038415  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:51.040985  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:51.045650  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:51.057801  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:51.059751  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 516) took 1124 ms. Trace:
I20260709 12:23:51.060420  5939 rpcz_store.cc:276] 0709 12:23:49.935142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.980089 (+ 44947us) service_pool.cc:224] Handling call
0709 12:23:50.012672 (+ 32583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:50.568711 (+556039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.059049 (+490338us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b5400006254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:51.059446 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":767515904}
I20260709 12:23:51.062247  7101 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:51.031891  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:51.062839  7101 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:51.063411  7101 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:51.045817  7074 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:51.047168  7099 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:51.065759  7099 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:51.065786  5148 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:51.068073  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.069151  5148 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:51.051220  6995 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:23:51.051669  7162 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.706s	user 0.002s	sys 0.006s
I20260709 12:23:51.052457  6889 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:51.052278  6974 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:51.055702  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:51.064801  7074 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.082693  7190 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:51.085027  7190 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:51.086813  5071 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:23:51.081019  7191 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:51.096508  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 596) took 1119 ms. Trace:
I20260709 12:23:51.096200  7191 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:51.104413  5940 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:51.108346  6974 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:51.096987  5072 rpcz_store.cc:276] 0709 12:23:49.976702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.976917 (+   215us) service_pool.cc:224] Handling call
0709 12:23:50.009523 (+ 32606us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:50.014812 (+  5289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:50.014850 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:50.015422 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 12:23:50.191211 (+175789us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:50.191268 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.191350 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:51.035020 (+843670us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b240050bd18. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a52d49c4 00007fd8a52d2acb 00007fd8a548ddc9 00007fd8a548da55 00007fd8a548d09d 00007fd8a80bf10e 00007fd8a80d688c 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221
0709 12:23:51.070202 (+ 35182us) raft_consensus.cc:1685] finished
0709 12:23:51.070235 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:51.096231 (+ 25996us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4766,"raft.queue_time_us":5043,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":13318,"spinlock_wait_cycles":24859904,"thread_start_us":113794,"threads_started":1,"wal-append.queue_time_us":224375}
I20260709 12:23:51.111220  6995 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.090293  6969 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:51.117334  6969 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:51.095304  7190 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:51.126107  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 518) took 1190 ms. Trace:
I20260709 12:23:51.127308  6995 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:51.127504  7114 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:51.129721  7050 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.125123  6886 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:51.147856  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:51.149891  6876 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:51.150779  6876 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:51.153790  7114 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:51.154450  5067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 595) took 1181 ms. Trace:
I20260709 12:23:51.154911  5067 rpcz_store.cc:276] 0709 12:23:49.972908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.973075 (+   167us) service_pool.cc:224] Handling call
0709 12:23:49.973422 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:49.973836 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:49.973873 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:49.974361 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 12:23:50.181238 (+206877us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:50.181303 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.181424 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:51.050812 (+869388us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b240050bd18. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a52d49c4 00007fd8a52d2acb 00007fd8a548ddc9 00007fd8a548da55 00007fd8a548d09d 00007fd8a80bf10e 00007fd8a80d688c 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221
0709 12:23:51.131841 (+ 81029us) raft_consensus.cc:1685] finished
0709 12:23:51.131888 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:51.154199 (+ 22311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3551,"raft.queue_time_us":981,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":12268,"spinlock_wait_cycles":24618240,"thread_start_us":206269,"threads_started":1,"wal-append.queue_time_us":318061}
I20260709 12:23:51.153841  7050 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:51.132958  5946 rpcz_store.cc:276] 0709 12:23:49.935734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.054448 (+118714us) service_pool.cc:224] Handling call
0709 12:23:50.071128 (+ 16680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:50.071694 (+   566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:50.071739 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:50.072441 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 12:23:50.199501 (+127060us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:50.199551 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.199630 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:50.242629 (+ 42999us) raft_consensus.cc:1685] finished
0709 12:23:50.242655 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:51.077585 (+834930us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b5400006c50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:51.125825 (+ 48240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10485,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":37319424,"thread_start_us":126663,"threads_started":1,"wal-append.queue_time_us":162520}
I20260709 12:23:51.159403  7050 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:51.168004  7064 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:23:51.170882  5215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 247) took 1971 ms (client timeout 1877 ms). Trace:
W20260709 12:23:51.171470  5215 rpcz_store.cc:269] 0709 12:23:49.199211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.217203 (+ 17992us) service_pool.cc:224] Handling call
0709 12:23:51.170358 (+1953155us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400446bd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:51.170575 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393241984}
I20260709 12:23:51.152567  6876 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:51.180724  7098 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:51.181371  7074 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:51.182075  7074 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.181692  7064 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.182787  7074 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:51.186584  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:51.187831  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 669) took 1278 ms. Trace:
I20260709 12:23:51.188295  5216 rpcz_store.cc:276] 0709 12:23:49.909003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.922107 (+ 13104us) service_pool.cc:224] Handling call
0709 12:23:51.186242 (+1264135us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416eee 000000000036eb39 00007fd8ad48a1e0 00007fd8ad5530c4 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:23:51.187719 (+  1477us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9708416}
I20260709 12:23:51.106201  6889 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.198525  7191 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:51.199733  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:51.201169  7114 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:51.174876  7190 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:51.191679  7074 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:51.208877  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.209056  5944 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:51.209918  7190 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:51.207548  5190 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.522s (SENT) [suppressed 8 similar messages]
I20260709 12:23:51.212239  6889 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:51.212888  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.206266  5939 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:23:51.213914  5147 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:51.214197  5939 pending_rounds.cc:85] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 3
I20260709 12:23:51.214646  5939 pending_rounds.cc:107] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:23:51.214974  5939 raft_consensus.cc:2889] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:51.219914  5143 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:51.235725  5149 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:23:51.202430  7064 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:51.241657  5067 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []
W20260709 12:23:51.249660  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.243651  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:51.260200  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52938.1us
I20260709 12:23:51.279248  5146 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:51.257519  7101 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:51.297987  7106 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:51.299168  6927 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.299988  7106 raft_consensus.cc:3055] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:23:51.298054  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.300557  7106 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:51.300189  7114 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:51.303158  7114 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:51.303604  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:51.306303  5216 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:51.306200  7106 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:51.307852  7106 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:51.314060  5145 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:51.314764  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 253) took 2114 ms. Trace:
I20260709 12:23:51.315438  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.316915  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:51.323932  6969 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:51.326468  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.329846  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:51.331394  6799 raft_consensus.cc:3055] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:51.332023  6799 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:51.332511  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:51.336323  5147 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:51.324000  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.339063  5146 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:51.340361  5146 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:51.341643  7056 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:51.342454  7056 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.345105  7056 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:51.347604  7101 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:51.315253  5217 rpcz_store.cc:276] 0709 12:23:49.200598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.541485 (+340887us) service_pool.cc:224] Handling call
0709 12:23:49.581096 (+ 39611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.714820 (+133724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.314063 (+1599243us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b5400114254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:51.314530 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":305129216}
W20260709 12:23:51.305370  6702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.713s	user 0.021s	sys 0.014s
W20260709 12:23:51.359320  6702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.813s	user 0.021s	sys 0.015s
W20260709 12:23:51.360252  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.329375  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 350) took 2552 ms. Trace:
W20260709 12:23:51.370834  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 74061.1us
I20260709 12:23:51.363636  5940 rpcz_store.cc:276] 0709 12:23:48.777157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.056810 (+279653us) service_pool.cc:224] Handling call
0709 12:23:50.290053 (+1233243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:51.106704 (+816651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.196832 (+ 90128us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b5400240c54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:51.329039 (+132207us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400240c50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:51.329257 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4705025920}
I20260709 12:23:51.394672  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.397872  7088 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.398773  7088 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:51.402920  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:51.404783  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.407752  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.407778  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:51.408254  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:51.408855  5220 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:51.411180  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:51.414915  5146 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:51.416344  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.417881  5149 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:51.419708  7165 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.420594  7165 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:51.420634  5148 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:51.422586  5220 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:51.430296  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:51.393786  5143 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: [3.2-5.3]
W20260709 12:23:51.441138  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 54714.6us
I20260709 12:23:51.443293  7064 consensus_queue.cc:1059] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:51.444200  7064 consensus_queue.cc:1243] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:23:51.446408  7191 raft_consensus.cc:3055] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:51.446678  7050 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:51.447117  6831 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:51.446991  7191 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:51.450826  7191 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:51.452612  7172 log.cc:927] Time spent T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.800s	user 0.003s	sys 0.006s
I20260709 12:23:51.452800  7191 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:51.454367  7074 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:51.484995  5145 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:23:51.489179  5218 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:51.489925  5218 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:51.424983  7099 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:51.300196  6927 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:51.538156  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:51.364441  6799 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.545280  5150 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:51.543428  5221 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:51.558377  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.553340  5144 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:51.535537  7098 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:51.564796  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 172314us
I20260709 12:23:51.564960  5141 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:51.578125  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.590402  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:51.591324  7165 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:51.573992  6799 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:51.561199  6720 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:51.593096  6720 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:51.595108  6927 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:51.595922  6720 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:51.602017  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:51.603034  5219 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:51.576526  5218 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:51.567618  7088 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:51.616868  7055 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:23:51.620710  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 116215us
W20260709 12:23:51.621820  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.625085  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:51.571252  7098 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:51.630409  5069 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:51.631064  7114 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:51.625578  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:51.633427  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:51.618940  7055 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:51.619451  6927 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:51.639245  5218 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:51.643585  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:51.644208  5067 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:51.650260  5073 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:51.655246  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:51.660408  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.672569  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:51.676658  5146 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:51.693765  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 510) took 1760 ms. Trace:
I20260709 12:23:51.694420  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:51.694563  5943 rpcz_store.cc:276] 0709 12:23:49.933464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.971164 (+ 37700us) service_pool.cc:224] Handling call
0709 12:23:49.971528 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:50.176320 (+204792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.688257 (+1511937us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400008554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:51.693495 (+  5238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377400064}
I20260709 12:23:51.693922  7009 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:51.695948  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:51.701692  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:51.693765  4895 catalog_manager.cc:5697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 4 to 5, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 5 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:51.698252  5944 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:51.699107  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:51.711805  5942 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:51.702564  5946 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:51.696259  7009 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:51.707867  5074 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:51.707674  6921 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:51.668309  6927 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.736279  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:51.739869  5221 raft_consensus.cc:2410] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:23:51.740768  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:51.681025  7114 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:51.743505  6927 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:51.689846  7099 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:51.744282  7190 consensus_meta.cc:287] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.003s
I20260709 12:23:51.744949  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:51.755864  5141 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:51.746517  5142 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:51.747215  7009 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:51.741081  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:51.757834  5143 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:51.760514  5073 raft_consensus.cc:2410] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:23:51.750422  7165 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:51.779004  5150 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:51.775123  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:51.779942  5069 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:51.782737  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:51.784758  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 209437us
I20260709 12:23:51.787693  5146 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:23:51.790462  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:51.790462  5946 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:51.793285  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:51.795207  7190 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:51.795921  7190 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:51.797999  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.798792  5147 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:51.798996  7055 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.800912  7055 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:51.803241  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.771766  7074 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:51.803071  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:51.820815  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:51.828656  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:51.827766  5072 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:51.817951  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:51.805408  6649 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:51.842164  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:51.844965  7088 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:51.845750  7088 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.848186  7088 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:51.838642  7074 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.854554  5224 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:51.858723  7114 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:51.859243  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:51.859591  7114 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:51.859704  6953 consensus_queue.cc:1059] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:51.860113  5223 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:51.860553  7098 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:51.860992  6953 raft_consensus.cc:3055] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:23:51.861901  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.864118  5147 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:23:51.864560  5143 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:51.864725  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:51.865752  6927 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:51.865983  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.866799  7101 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:51.867305  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:51.865983  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.868319  5149 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:51.868856  7087 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:51.869261  7014 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:51.868340  6886 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:51.869483  7087 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:51.870129  6886 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:51.871101  6886 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:51.871677  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.872146  7087 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:51.872490  6886 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:51.873790  5899 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:51.865245  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:51.874620  5148 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:51.859988  5943 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:51.876461  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:51.881681  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 56936.4us
I20260709 12:23:51.793500  7165 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:51.861362  6953 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:51.887008  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:51.885608  5143 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:51.892999  5074 raft_consensus.cc:2393] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:51.898032  5145 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:51.894976  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:51.909149  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:51.968938  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 43694.4us
I20260709 12:23:51.912168  7190 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:51.972539  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 56389.3us
I20260709 12:23:51.977850  7165 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:51.982244  7088 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.982547  5188 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:51.983091  7088 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:51.990738  7088 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:51.983510  6962 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:23:51.998847  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 159731us
I20260709 12:23:52.005435  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:52.001034  5219 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:52.000319  5148 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:52.009647  5141 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
W20260709 12:23:52.013052  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 108349us
I20260709 12:23:52.013463  7087 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:52.013764  6964 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:52.016193  7198 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:52.000324  7165 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.016961  7198 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:52.017663  7198 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:52.017428  7165 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:52.006753  7056 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:52.020447  7056 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.020798  7056 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:52.022538  7165 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:51.991341  6953 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:52.034559  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:52.036785  6921 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:52.039124  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:52.016001  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:51.971539  7114 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:52.051285  5218 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
W20260709 12:23:52.053364  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 48276.9us
I20260709 12:23:52.060680  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.062052  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.062438  7087 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:52.063257  7087 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:52.065969  7087 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:52.051896  5218 pending_rounds.cc:85] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 2
I20260709 12:23:52.067687  5218 pending_rounds.cc:107] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:23:52.067971  5218 raft_consensus.cc:2889] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:52.068300  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.045929  7064 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:52.043828  6962 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:23:52.130328  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.016673  7099 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
W20260709 12:23:52.135574  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.134857  6877 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:23:52.146071  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.138787  7088 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.150560  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:52.167415  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 300122us
I20260709 12:23:52.141435  7161 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:23:52.171144  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 155265us
I20260709 12:23:52.165899  7014 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:52.171231  6877 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:52.183575  5947 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:52.186046  6877 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:52.188483  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.188547  7232 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:52.196069  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:52.196674  7161 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:52.196746  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:52.186338  7227 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:23:52.167587  7088 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:52.180961  7014 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:52.209199  7074 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:52.200245  7231 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:52.203380  7009 raft_consensus.cc:2764] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 12:23:52.222306  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:52.212702  7198 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:52.225395  7222 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.251s	user 0.002s	sys 0.000s
I20260709 12:23:52.229727  7198 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:52.240447  6831 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.241313  6831 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:52.244307  6831 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:52.245709  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.246563  6799 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.247498  6799 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:52.250625  7014 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:52.252461  6953 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.214363  6962 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:52.258318  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:52.262431  5074 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:52.266008  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 63004.3us
I20260709 12:23:52.270095  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:52.270330  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 89091.2us
I20260709 12:23:52.258286  6649 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.224303  7231 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:52.295508  7231 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:52.286417  6799 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:52.289171  6649 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:52.305577  7229 log.cc:927] Time spent T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.105s	user 0.000s	sys 0.002s
I20260709 12:23:52.307271  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.310649  5074 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:52.312408  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:52.315076  7088 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:52.316049  7233 log.cc:927] Time spent T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 12:23:52.319370  7226 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.328s	user 0.003s	sys 0.000s
I20260709 12:23:52.320375  7231 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:23:52.321755  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 51773.5us
I20260709 12:23:52.322449  7055 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:52.323045  7055 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:52.323943  7055 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:52.326562  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.327263  6649 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:52.327512  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:52.327873  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:52.328091  5943 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:23:52.328483  5943 pending_rounds.cc:85] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 3
I20260709 12:23:52.328706  5943 pending_rounds.cc:107] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:23:52.328979  5943 raft_consensus.cc:2889] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:52.329192  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:52.330295  6886 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:52.330734  6886 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:52.331678  6886 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:52.332966  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:52.333124  6886 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:52.335397  7232 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:52.335915  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.336632  5942 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:52.337983  6927 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:52.339677  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.340750  5943 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:52.341869  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.340200  5898 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:52.343633  7236 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
W20260709 12:23:52.346174  7237 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.160s	user 0.000s	sys 0.002s
I20260709 12:23:52.347676  5072 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:52.172312  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:52.351390  7056 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:52.353901  5218 raft_consensus.cc:2368] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6.
I20260709 12:23:52.361274  5222 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
W20260709 12:23:52.361285  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 63728.7us
I20260709 12:23:52.357107  5041 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:52.298112  7173 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:52.369580  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:52.362720  7101 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.386840  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:52.387939  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.388566  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.393124  5067 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:52.389174  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:52.394485  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:52.394485  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:52.368428  6649 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.370626  7232 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:52.398485  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:52.403972  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:52.371939  7198 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.412084  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.415536  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.408252  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.378504  7088 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:52.363735  7074 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:23:52.436067  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:52.441788  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:52.441987  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.443142  7241 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:52.443781  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.432461  6877 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:23:52.447417  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.449872  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:52.449884  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 65885.7us
I20260709 12:23:52.440163  5067 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:52.422549  7064 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:52.472906  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.456676  5190 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:52.473788  5221 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:52.475371  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.480482  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.431770  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:52.403654  7101 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:52.478883  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:52.418717  6649 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:52.483378  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:52.483378  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.484438  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.457783  7088 raft_consensus.cc:697] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:52.472124  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.511170  5219 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:52.426873  7198 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:52.468520  7056 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:52.471453  6831 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 12:23:52.515914  7239 consensus_queue.cc:1059] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
W20260709 12:23:52.517059  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 60539.1us
I20260709 12:23:52.518484  4895 catalog_manager.cc:5697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:52.519639  6921 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:52.520699  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.554411  5947 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:52.521080  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:52.523955  4899 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43222 (request call id 141) took 9277 ms. Trace:
I20260709 12:23:52.523619  7088 consensus_queue.cc:237] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:52.530467  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:52.531102  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.443811  7241 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:52.539381  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 55380.2us
I20260709 12:23:52.480633  7231 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:23:52.477478  7098 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 12:23:52.551296  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:52.555116  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 76293.1us
I20260709 12:23:52.564894  5222 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:52.553958  6921 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.568101  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:52.521615  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:52.543993  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:52.571996  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:52.562096  6953 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:52.602665  5115 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:52.605903  7242 raft_consensus.cc:3055] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:52.608630  7014 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:52.568003  7190 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:52.611367  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:52.608711  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.617158  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:52.620407  7173 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:52.621920  5115 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:52.658286  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.576340  5941 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:52.616245  7242 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:52.559779  7006 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:23:52.597676  6921 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:52.639142  6962 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:52.640615  5219 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:52.641172  7101 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:52.621816  4895 catalog_manager.cc:5697] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 5, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 5 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:52.584808  7165 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:52.666638  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.601682  7161 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
W20260709 12:23:52.682204  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.611152  6953 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.601964  6877 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:52.670220  5218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2725c773e7344020b5752861d00b493e"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:52.612046  7114 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:23:52.671980  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.568188  7238 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 12:23:52.654991  7014 consensus_queue.cc:1243] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer ceec344dd00f40c18ef58414ce147016 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 12:23:52.691277  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 80478.8us
I20260709 12:23:52.658639  7173 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:52.704643  4895 catalog_manager.cc:5697] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 3 to 4, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 4 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:52.720186  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.616813  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.693485  5898 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:52.669169  7098 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:52.653554  6649 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:52.680488  5942 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 4: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:52.714250  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:52.723137  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 111855us
I20260709 12:23:52.674947  6964 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.714772  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.721387  5945 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:52.679692  7198 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:52.732923  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:52.734720  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:52.694268  6962 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:52.693308  7099 raft_consensus.cc:3055] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:52.693220  7006 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:52.692936  6953 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:52.734989  7231 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:52.729874  4895 catalog_manager.cc:5697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:52.737995  5223 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:52.669503  7190 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:52.744319  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:52.726846  7173 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:52.700953  7165 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:52.771404  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.773614  4895 catalog_manager.cc:5697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:52.741127  5943 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
W20260709 12:23:52.779822  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 47033.1us
I20260709 12:23:52.780704  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:52.733943  6886 consensus_queue.cc:1059] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:23:52.779523  7050 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:52.780762  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:52.784490  7243 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 12:23:52.785766  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:52.786422  7055 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:52.787006  7190 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:52.785785  7247 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:23:52.752048  7099 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:52.809994  5948 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:52.559747  4899 rpcz_store.cc:276] 0709 12:23:43.246730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.256008 (+  9278us) service_pool.cc:224] Handling call
0709 12:23:43.480602 (+224594us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b54006a5754. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a5090 00007fd8a8543f3a 00007fd8ae2f9e9c 00007fd8ae3f804e 00007fd8ac8b92a4 00007fd8ac8b9232 00007fd8ac8b9161 00007fd8ac8b90dc 00007fd8ac8b8392 00007fd8a687648c 00007fd8a68758c6
0709 12:23:52.523675 (+9043073us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:45.764706 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:45.765607 (+   901us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:45.765660 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:23:45.765689 (+    29us) tablet.cc:662] Decoding operations
0709 12:23:45.767917 (+  2228us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:45.768035 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:45.768079 (+    44us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:23:45.774863 (+  6784us) tablet.cc:701] Row locks acquired
0709 12:23:45.774888 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:23:45.775107 (+   219us) write_op.cc:270] Start()
0709 12:23:45.775274 (+   167us) write_op.cc:276] Timestamp: P: 1783599825775040 usec, L: 0
0709 12:23:45.775301 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:23:45.776335 (+  1034us) log.cc:844] Serialized 7464 byte log entry
0709 12:23:46.609311 (+832976us) op_driver.cc:464] REPLICATION: finished
0709 12:23:46.748620 (+139309us) write_op.cc:301] APPLY: starting
0709 12:23:46.748755 (+   135us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:46.749682 (+   927us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:46.756929 (+  7247us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:46.824941 (+ 68012us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:46.825474 (+   533us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0709 12:23:46.825529 (+    55us) write_op.cc:312] APPLY: finished
0709 12:23:46.848252 (+ 22723us) log.cc:844] Serialized 131 byte log entry
0709 12:23:46.848477 (+   225us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:46.848696 (+   219us) write_op.cc:454] Released schema lock
0709 12:23:46.848762 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:46.849220 (+   458us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:46.849252 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":103971,"reported_tablets":35,"spinlock_wait_cycles":53587712,"child_traces":[["op",{"apply.queue_time_us":124565,"apply.run_cpu_time_us":7368,"apply.run_wall_time_us":100577,"num_ops":19,"prepare.queue_time_us":390200,"prepare.run_cpu_time_us":104516,"prepare.run_wall_time_us":174272,"raft.queue_time_us":392546,"raft.run_cpu_time_us":13825,"raft.run_wall_time_us":130030,"replication_time_us":826973,"thread_start_us":857516,"threads_started":4,"wal-append.queue_time_us":211372,"wal-append.run_cpu_time_us":33158,"wal-append.run_wall_time_us":1887908}]]}
W20260709 12:23:52.825593  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 90936.6us
W20260709 12:23:52.832283  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 210440us
I20260709 12:23:52.848248  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:52.848461  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:52.844031  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 58363us
I20260709 12:23:52.817126  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:52.749075  6964 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:52.773010  7074 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:23:52.844027  7231 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:52.867182  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:52.868635  7222 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.165s	user 0.002s	sys 0.000s
I20260709 12:23:52.837622  6889 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:23:52.872033  7251 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:23:52.873317  5115 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:52.875379  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 53485.2us
I20260709 12:23:52.843753  6962 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:52.876876  5898 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:52.880151  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.855966  7198 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:52.880978  5940 raft_consensus.cc:3055] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:52.881609  5940 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:52.882266  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.882984  5148 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:52.846411  7241 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:52.884702  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.885672  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:52.886366  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 69333.4us
I20260709 12:23:52.888782  5041 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:52.882464  5940 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:52.893321  5148 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:52.876857  6889 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.771982  7242 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:52.901386  5940 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:52.896313  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.877837  7101 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:52.904811  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.907987  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.878809  7161 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:52.922317  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.930408  5145 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:52.924136  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:52.925165  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.926110  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:52.932495  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 43786.7us
I20260709 12:23:52.934044  5147 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:23:52.934672  5041 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:52.936049  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.905781  5143 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:52.890269  7087 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:52.877686  7197 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 12:23:52.938817  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 62050.5us
I20260709 12:23:52.941149  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.941983  5143 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:52.930408  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:52.904593  6889 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:52.903745  7190 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:52.946183  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.908689  7114 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:52.944310  5146 raft_consensus.cc:2376] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6. Re-sending same reply.
I20260709 12:23:52.953079  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:52.953477  7252 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:23:52.954504  7264 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:52.955175  7264 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.955621  7264 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:52.961175  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:52.961982  5150 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:52.965068  5900 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:52.942842  6831 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:52.907135  7243 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:23:52.960606  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:52.938513  7087 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:52.944869  7241 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:52.950953  7114 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.976529  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:52.977620  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:52.972383  6831 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.973666  7190 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:52.927953  7099 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:52.978319  7087 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:53.007666  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:53.018152  5940 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:52.983909  7114 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:52.986744  6831 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:52.982381  7009 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:53.030376  7009 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:52.986099  7098 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:53.018162  7280 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:53.034634  7280 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:53.040130  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:53.041604  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:53.042776  7099 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:53.049976  7262 log.cc:927] Time spent T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.169s	user 0.002s	sys 0.000s
I20260709 12:23:53.052335  7277 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:53.042847  7275 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.058791  7275 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:53.063011  7190 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:53.064379  7056 raft_consensus.cc:3055] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:53.065059  7056 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:53.065989  6886 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:53.067392  6964 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:53.069187  7198 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:53.071908  7098 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:53.072425  7098 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.072785  7098 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:53.074244  7253 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:23:53.073096  6649 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:23:53.106662  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 65147us
I20260709 12:23:53.127828  5941 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:23:53.130146  5947 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:53.134756  7277 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:53.138484  7277 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.138885  7277 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:53.147027  7275 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:53.147207  6953 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:53.148921  5945 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:23:53.128746  7064 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:53.148665  7006 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:23:53.162521  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 293641us
I20260709 12:23:53.028086  7278 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.140748  6921 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:53.094569  7165 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:23:53.173213  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 208305us
I20260709 12:23:53.169521  5190 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:23:53.175396  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.176167  5900 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:53.183692  5218 raft_consensus.cc:606] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:23:53.201977  5218 raft_consensus.cc:614] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:53.183848  7165 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:53.205930  7264 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:53.211961  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 48246.5us
W20260709 12:23:53.226958  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 223585us
I20260709 12:23:53.236888  5190 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:53.175293  6649 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.246266  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:53.181103  7238 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:23:53.249841  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 119590us
I20260709 12:23:53.251152  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:53.210039  7267 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:53.222184  7197 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:23:53.275682  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 62798us
I20260709 12:23:53.171900  7278 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:53.260768  5943 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:23:53.232162  6889 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:53.263300  7197 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:53.304522  5190 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:53.255231  6649 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:53.313280  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.291803  7289 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:53.297467  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 96092.7us
I20260709 12:23:53.331588  7009 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:53.297245  7281 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 12:23:53.298177  7265 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 12:23:53.248883  7238 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:53.186070  7056 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:53.320298  7288 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:53.325062  6889 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:53.338475  5944 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:53.326100  6921 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:53.326807  6953 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:53.327600  7087 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:53.328431  6831 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:53.329566  7098 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:53.334676  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 78304.7us
I20260709 12:23:53.334800  5115 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:53.324923  7286 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:53.348537  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:53.365031  5940 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:53.312252  7197 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:53.263010  7267 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:53.386595  7267 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:53.387549  7238 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:53.388034  7291 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:53.388646  7290 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:53.388790  7291 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:53.395284  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:53.370767  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:53.370450  7292 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:23:53.397495  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:53.395800  7291 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:53.398867  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:53.371217  7286 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:53.401454  7286 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:53.402518  7006 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:53.403329  7006 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:53.376264  7087 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:53.377761  6889 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:53.378682  6831 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:53.396380  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:53.405501  5069 raft_consensus.cc:2368] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4.
I20260709 12:23:53.388399  5941 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:53.408399  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:53.411368  7197 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:53.415630  5940 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:53.396039  5946 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:53.375197  7278 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:53.380900  7287 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:53.348778  5216 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:53.382218  7264 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:53.384155  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:53.395462  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:53.380568  6649 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:53.404036  7006 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:23:53.363584  7173 consensus_meta.cc:287] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.577s	user 0.012s	sys 0.007s
I20260709 12:23:53.433735  5944 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:53.425694  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:53.430011  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:53.424108  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:53.460381  5142 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:53.380944  7098 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:53.462971  6649 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:53.560714  6649 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:53.560068  7197 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:53.562114  6649 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:53.422670  7114 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:53.568671  7114 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:53.571097  7114 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:53.575119  7248 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.575452  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:53.575852  7248 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:53.577370  7291 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:53.577909  7291 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:23:53.578512  5042 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.007s	sys 0.012s
I20260709 12:23:53.463744  7173 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:53.468132  5188 leader_election.cc:400] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4.
I20260709 12:23:53.585165  5188 leader_election.cc:403] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:53.468943  5115 leader_election.cc:400] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6.
I20260709 12:23:53.556972  7287 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.366297  6877 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.103s
W20260709 12:23:53.559072  7099 consensus_meta.cc:287] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.000s
I20260709 12:23:53.559501  5946 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:53.559798  7278 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:53.566140  7266 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:53.579196  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:53.579273  5042 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.007s	sys 0.012s
I20260709 12:23:53.565120  7267 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:53.594126  7287 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:53.614935  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.584455  6927 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:53.597795  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 139863us
W20260709 12:23:53.598769  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 178357us
I20260709 12:23:53.617326  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:53.618530  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:53.564857  7239 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.006s	sys 0.007s
W20260709 12:23:53.609956  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 723818us
W20260709 12:23:53.608676  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 47806.6us
I20260709 12:23:53.635277  5141 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:23:53.636544  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 57473.4us
W20260709 12:23:53.604156  7258 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.746s	user 0.000s	sys 0.004s
I20260709 12:23:53.645165  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:53.638474  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:53.639448  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:53.623595  7266 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:53.652294  7248 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:53.625510  7239 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.008s	user 0.006s	sys 0.008s
W20260709 12:23:53.654644  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.623634  6921 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:23:53.662441  5900 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:53.688900  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:53.689883  5223 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:53.661939  7265 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:53.700343  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:53.700479  6831 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:53.701645  7297 raft_consensus.cc:3055] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:53.705991  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:53.702175  7297 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:53.687472  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:53.653165  7098 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:53.702402  7276 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:53.729027  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 66703.7us
I20260709 12:23:53.633682  7087 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 12:23:53.633359  6927 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:53.623963  7278 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:53.630669  7277 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:53.744421  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:53.700405  7301 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:53.718039  7165 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:53.734927  7267 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:53.791738  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:53.734735  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 287) took 1595 ms. Trace:
I20260709 12:23:53.739948  7006 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:53.796578  5147 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:53.653165  7291 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:53.754565  7259 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.754173  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:53.822036  5215 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:53.754817  7239 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:53.765755  6831 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 12:23:53.785290  7098 raft_consensus.cc:697] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:53.779080  7099 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:53.790982  7165 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:53.812062  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:53.812403  5150 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:23:53.814802  7310 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:23:53.837491  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:53.853886  5147 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:53.784698  7301 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.818320  7173 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:53.819870  7280 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:53.808257  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:53.846084  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:53.837057  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:53.870894  5148 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:53.842579  5145 pending_rounds.cc:85] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 1
I20260709 12:23:53.871999  5145 pending_rounds.cc:107] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:23:53.872255  5145 raft_consensus.cc:2889] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:53.815394  7278 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:53.890111  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:53.869781  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:53.892477  5146 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:53.870949  5144 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:53.857486  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:53.890993  5150 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:53.793762  7161 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:53.898824  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:53.900175  5115 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:53.853834  7165 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:53.844555  7291 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:53.834107  7287 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:53.904629  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 204150us
I20260709 12:23:53.827929  6927 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:53.910470  5042 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:53.913976  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:53.922086  5148 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:53.873356  7301 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:53.810271  7297 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:53.935206  7297 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:53.852432  7006 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:53.937647  7303 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:53.938009  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:53.928730  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:53.922475  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:53.931313  5069 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:23:53.931526  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 135619us
I20260709 12:23:53.944087  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:53.946194  5148 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:53.866139  7099 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:53.866523  7239 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:53.950085  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:53.950796  7303 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:53.985441  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:53.956401  7242 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:53.994055  5146 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:53.994460  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:53.995193  5942 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
W20260709 12:23:53.997103  7295 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:23:53.836180  7259 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:54.000684  7259 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:53.906224  7278 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:54.002305  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:54.002797  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 387) took 1035 ms. Trace:
I20260709 12:23:54.003302  5939 rpcz_store.cc:276] 0709 12:23:52.967644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.968260 (+   616us) service_pool.cc:224] Handling call
0709 12:23:52.983640 (+ 15380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:53.166503 (+182863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:53.166542 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:53.174940 (+  8398us) log.cc:844] Serialized 26 byte log entry
0709 12:23:53.260944 (+ 86004us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:53.622674 (+361730us) log.cc:844] Serialized 14 byte log entry
0709 12:23:53.625142 (+  2468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.625336 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:54.002575 (+377239us) raft_consensus.cc:1685] finished
0709 12:23:54.002598 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:54.002734 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1822,"prepare.queue_time_us":548182,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":21110,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":57069,"thread_start_us":547683,"threads_started":3,"wal-append.queue_time_us":670401}
I20260709 12:23:53.959576  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:53.961642  7190 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:53.962759  7056 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:54.008183  7239 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:53.963366  7249 raft_consensus.cc:3055] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:53.897446  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:53.966094  7098 consensus_queue.cc:237] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:53.929347  6649 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:53.809345  7267 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:53.968516  7277 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:53.976315  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 53939.1us
I20260709 12:23:53.978247  5115 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:53.979980  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57004 (request call id 12) took 1302 ms. Trace:
W20260709 12:23:54.042289  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 64148.8us
I20260709 12:23:54.042140  5218 rpcz_store.cc:276] 0709 12:23:52.669502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.670069 (+   567us) service_pool.cc:224] Handling call
0709 12:23:53.971633 (+1301564us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b5400446bd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a8535853 00007fd8ad50360a 00007fd8a6a897d4 00007fd8a6a89762 00007fd8a6a89691 00007fd8a6a8960c 00007fd8a6a888c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:53.971794 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123942656}
I20260709 12:23:53.986197  5150 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:53.992906  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:53.993678  7064 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:54.019838  5144 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:54.034924  5147 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
W20260709 12:23:54.045037  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52256.2us
I20260709 12:23:54.046406  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:54.057415  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:54.057312  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:54.059235  5147 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:54.063694  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:54.048175  5190 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:54.067476  5190 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:54.058375  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:54.013446  7249 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:53.905400  7161 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:53.985831  7305 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:54.093596  7305 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:54.094120  7305 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:54.095876  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 464) took 1155 ms. Trace:
I20260709 12:23:54.096294  5943 rpcz_store.cc:276] 0709 12:23:52.940846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.951220 (+ 10374us) service_pool.cc:224] Handling call
0709 12:23:53.002510 (+ 51290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:53.300405 (+297895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:54.095583 (+795178us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54002e5254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:54.095811 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519071104}
I20260709 12:23:53.980937  7287 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 12:23:54.075798  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:54.074791  7296 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:54.035626  6649 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:54.065286  5148 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:54.081806  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:54.049053  5215 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:54.071709  7064 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:54.090081  7265 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:54.090794  7231 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:54.092752  7301 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:54.048409  7303 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:54.048156  7191 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:54.102715  5042 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:54.066890  7173 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:54.067196  7276 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.121131  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:54.101298  6649 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:54.126081  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:54.129442  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 47737.1us
I20260709 12:23:54.075071  7248 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:23:54.136248  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:54.111394  7296 raft_consensus.cc:697] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:54.117043  6927 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.117493  7303 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.121912  7173 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.120234  7006 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:54.122762  6799 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:54.134553  7277 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:54.134263  7312 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:54.132408  7314 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:54.170498  7311 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:54.152552  7173 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:54.148694  7303 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:54.123651  7276 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:54.213033  7311 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:54.154215  7265 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:54.170730  7114 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:54.161595  7277 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:54.165956  7006 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:54.227847  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 169558us
I20260709 12:23:54.151830  6927 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:54.174081  7312 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:54.218508  7314 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:23:54.239032  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 111174us
I20260709 12:23:54.240926  5042 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:53.796590  5217 rpcz_store.cc:276] 0709 12:23:52.132891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.133979 (+  1088us) service_pool.cc:224] Handling call
0709 12:23:52.141607 (+  7628us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:52.179146 (+ 37539us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:23:52.183060 (+  3914us) log.cc:844] Serialized 14 byte log entry
0709 12:23:52.352664 (+169604us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b180012ef80. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db23a 00007fd8a8545a2a
0709 12:23:52.680030 (+327366us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:52.753176 (+ 73146us) log.cc:844] Serialized 26 byte log entry
0709 12:23:52.767736 (+ 14560us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:52.772713 (+  4977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:52.782476 (+  9763us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:53.604567 (+822091us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b240035e598. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a52d49c4 00007fd8a52d2acb 00007fd8a548ddc9 00007fd8a548da55 00007fd8a548cfb0 00007fd8a8103488 00007fd8a8103be8 00007fd8a80db398 00007fd8a80d676a 00007fd8a80d5bb6
0709 12:23:53.688415 (+ 83848us) raft_consensus.cc:1685] finished
0709 12:23:53.688438 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:53.728109 (+ 39671us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75495,"prepare.queue_time_us":3860,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":61600,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":15643,"spinlock_wait_cycles":83064448,"thread_start_us":260554,"threads_started":1,"wal-append.queue_time_us":568720}
I20260709 12:23:54.223124  7278 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:54.243578  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:54.229847  7277 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:54.247396  7296 consensus_queue.cc:237] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:54.248983  7314 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:54.217576  7064 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:54.227453  7074 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:54.270939  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:54.249651  6799 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:54.242264  7312 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:54.294466  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.266707  7099 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:54.238166  7006 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:54.271713  7291 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:54.227625  7265 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:54.311704  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 47131.1us
W20260709 12:23:54.317766  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 85946.1us
I20260709 12:23:54.320259  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:54.307798  7318 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:54.312096  7267 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:54.318132  7114 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:54.322685  7302 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:54.339664  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.340814  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.341270  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.342489  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.347826  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:54.359884  5074 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
W20260709 12:23:54.349972  7322 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:23:54.351486  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.353824  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.354394  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.354925  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.356191  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.357223  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 465) took 1416 ms. Trace:
I20260709 12:23:54.369630  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:54.370477  5943 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:54.371829  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:54.366437  5945 rpcz_store.cc:276] 0709 12:23:52.941155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.947612 (+  6457us) service_pool.cc:224] Handling call
0709 12:23:52.972238 (+ 24626us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:53.149521 (+177283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:54.356954 (+1207433us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54002d78d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:54.357152 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405588736}
I20260709 12:23:54.373943  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:54.374890  5946 raft_consensus.cc:2368] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4.
I20260709 12:23:54.377573  5068 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:54.378471  6927 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:54.380052  7173 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:54.334414  7161 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:54.328996  7318 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:54.383302  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:54.360689  7280 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:54.386592  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.387540  5073 raft_consensus.cc:2410] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:23:54.388378  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:54.323762  7278 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:54.389431  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:54.366168  7301 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:54.408582  5189 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:54.414773  7324 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:23:54.414881  7050 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:54.415180  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.416622  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:54.422004  7260 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:54.422255  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 551) took 1257 ms. Trace:
I20260709 12:23:54.422463  7260 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:54.422708  5941 rpcz_store.cc:276] 0709 12:23:53.165221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.171084 (+  5863us) service_pool.cc:224] Handling call
0709 12:23:53.182255 (+ 11171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:53.406705 (+224450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:54.422002 (+1015297us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b5400003a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:54.422188 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473636608}
I20260709 12:23:54.423318  7260 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:54.425288  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.393306  7318 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:54.398824  7056 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:54.362437  7302 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:54.439632  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 119459us
I20260709 12:23:54.424568  7260 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:54.435309  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:54.441990  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:54.442600  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:54.443460  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:54.410818  7276 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:54.444103  7286 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:54.444310  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:54.440954  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:54.454773  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:54.455091  7311 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:54.455437  5217 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:54.455734  7305 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:54.444813  5943 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:54.448740  7299 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:54.446812  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:54.445086  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:54.451519  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:54.458349  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:54.445068  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 117340us
I20260709 12:23:54.430756  7050 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.457955  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:54.485163  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:54.470566  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:54.478147  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:54.486429  5945 raft_consensus.cc:2410] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:23:54.437505  7284 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:54.444871  7056 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:54.445745  7014 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.488288  7050 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:54.488909  7173 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:54.490528  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:54.488308  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:54.493791  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 58595.8us
I20260709 12:23:54.466161  7074 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:54.478839  7313 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.483224  7231 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:54.497109  7006 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:54.503121  7265 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:54.496954  6927 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:54.500826  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:54.505656  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:54.546490  5148 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:23:54.492429  7288 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:54.514065  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:54.538028  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:54.515471  5900 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:54.539183  5189 leader_election.cc:400] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 3 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4.
I20260709 12:23:54.539230  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:54.488961  7259 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:23:54.542956  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 51366.9us
I20260709 12:23:54.543474  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:54.514258  5150 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: []
W20260709 12:23:54.553067  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 82652.5us
I20260709 12:23:54.554368  5147 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
W20260709 12:23:54.555284  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 57985us
I20260709 12:23:54.555802  5146 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:23:54.561621  7259 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.514741  7074 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:54.566838  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:54.621860  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 51745.9us
I20260709 12:23:54.574628  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.627261  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:54.636916  5148 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:54.482172  7312 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:54.647814  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 132411us
I20260709 12:23:54.648957  5900 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:54.622850  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:54.641186  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 50613.6us
I20260709 12:23:54.624428  5072 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:54.652311  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:54.577785  7259 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:54.664779  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:54.665688  5219 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:54.622484  7006 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:54.504284  7014 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:54.660000  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 99991.6us
I20260709 12:23:54.661303  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.673113  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:54.673990  5142 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:54.675979  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:54.661514  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:54.662483  5146 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:54.504251  7284 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:54.658505  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:54.572444  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:54.663383  7288 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:54.670881  7333 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:54.678117  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:54.675213  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 52479.4us
I20260709 12:23:54.736971  5146 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:54.737709  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:54.660286  7239 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:54.550463  7313 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:54.743101  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:54.743474  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:54.656137  7302 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:54.733065  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 63363us
I20260709 12:23:54.752321  5042 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:54.671998  7006 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:54.640123  7299 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 12:23:54.675735  7340 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.746868  5114 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:54.676050  7232 raft_consensus.cc:3055] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:54.776291  5142 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:54.731307  5148 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:23:54.734355  7099 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:54.786191  5042 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:54.790092  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 223325us
I20260709 12:23:54.680774  7328 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:54.740859  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:54.793883  5042 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:54.796096  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:54.806022  7347 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:54.806327  7350 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:54.781985  7064 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:54.785455  7287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.057s	sys 0.077s
I20260709 12:23:54.659662  7303 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:54.747428  7231 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:54.800040  5219 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:54.792912  7278 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:54.804431  7239 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:54.748545  7173 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:54.812156  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:54.814365  5148 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:23:54.816301  5042 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:54.788168  7302 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:54.781721  7191 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:54.764593  7006 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:54.786288  7232 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:54.753368  7114 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
W20260709 12:23:54.814301  7165 consensus_meta.cc:287] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.881s	user 0.008s	sys 0.006s
I20260709 12:23:54.817006  7350 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:54.792735  7340 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:54.808955  6889 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:54.831494  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:54.818436  7287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.057s	sys 0.077s
I20260709 12:23:54.817693  5216 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:54.835235  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 186390us
I20260709 12:23:54.837759  5219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:54.818050  7305 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:54.843569  7056 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:54.843667  6717 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:23:54.844161  7338 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:23:54.827129  7239 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:54.834007  7277 consensus_meta.cc:287] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.000s
W20260709 12:23:54.890385  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 214419us
I20260709 12:23:54.836844  7301 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:54.896668  7301 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:54.897744  6717 raft_consensus.cc:697] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:54.897852  7301 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:54.884575  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 53161.7us
I20260709 12:23:54.898793  6717 consensus_queue.cc:237] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:54.900101  7239 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:54.823081  7064 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:54.885308  5900 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:54.886307  5222 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:54.825949  7288 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:54.886823  6889 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:54.838243  7088 consensus_queue.cc:1059] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:23:54.887562  7340 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:54.884529  5219 raft_consensus.cc:606] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:23:54.895535  7277 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:54.853006  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.839116  7191 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:54.839990  6886 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:54.909337  7299 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:54.909951  7305 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:54.910437  7344 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 12:23:54.910992  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:54.913633  7341 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:23:54.913981  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:54.914500  7291 consensus_meta.cc:287] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.603s	user 0.008s	sys 0.002s
W20260709 12:23:54.914829  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.914726  7342 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:23:54.916661  6649 consensus_meta.cc:287] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.775s	user 0.011s	sys 0.000s
W20260709 12:23:54.917595  7339 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
I20260709 12:23:54.890602  7087 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:54.902022  6927 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:54.943914  6921 consensus_meta.cc:287] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.232s	user 0.012s	sys 0.005s
I20260709 12:23:54.919230  5222 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:54.841894  7347 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:54.903184  7327 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:54.941509  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:54.942215  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:54.963608  5943 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:54.847239  7165 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:54.845054  7350 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:54.937779  6886 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:54.950572  7291 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:54.918408  7232 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:54.843616  7231 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:54.952824  7312 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:54.961135  7352 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:54.963883  6921 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 3. Current term is 4.
I20260709 12:23:54.944986  7305 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:54.970636  7014 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:54.962411  7327 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:54.972339  7284 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:54.987993  5067 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:54.948112  6649 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:54.987385  7358 raft_consensus.cc:3055] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:54.984009  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:54.959316  6927 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:54.979705  6886 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:54.983101  7352 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:54.998055  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:54.998656  5219 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:55.005770  7278 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.006383  7278 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.008483  7278 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.009406  5218 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:54.993598  7232 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:54.992218  7327 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:54.976769  6889 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:55.014173  7064 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:54.994678  7358 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:23:55.021689  7345 log.cc:927] Time spent T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:23:55.021881  6886 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.023478  5076 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:54.972362  7347 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.033139  7347 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:55.010648  7352 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:55.034811  5148 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:55.010452  7303 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.034034  7064 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:55.039425  6927 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:55.022903  7313 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:55.021276  7006 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:55.044126  7064 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.049073  7165 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:55.050863  5944 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:55.070209  7064 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.051343  6886 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:55.036262  7266 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:55.047640  7305 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:55.080507  6649 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:55.054600  7277 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:55.082521  7006 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:55.084064  7302 raft_consensus.cc:3055] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:55.098584  7302 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:55.099345  7239 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.099601  7302 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:55.101629  7302 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.101825  7239 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:55.087208  7303 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:55.105881  7303 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.054792  7191 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:55.108958  7303 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:55.080626  7318 consensus_meta.cc:287] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.000s
I20260709 12:23:55.097343  5216 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:55.109797  7318 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.117064  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:55.117959  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.118589  5941 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:55.119150  5940 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:55.118664  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.119096  5943 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:55.122943  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.123575  5940 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:23:55.124061  5189 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:55.128122  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:55.130990  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:55.078701  7358 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:55.136099  7290 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:55.136584  7290 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:55.136619  7358 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:55.137012  7290 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:55.109289  7074 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.149045  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:55.154084  5216 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.133467  7303 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:23:55.168346  7303 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:55.168745  7303 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:55.169732  7303 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.172389  5943 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:55.174953  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:55.134471  7191 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:55.174155  5189 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:55.180224  7064 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:55.182401  7064 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:55.183001  7318 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.183682  7239 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.179060  7191 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:55.184360  7191 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:55.184304  7239 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.187242  7239 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:55.187731  5223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:55.188423  5223 raft_consensus.cc:606] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:23:55.188735  5223 raft_consensus.cc:614] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:55.191316  5219 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:55.183360  7064 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.160215  7074 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.166183  7352 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:55.226039  7368 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:23:55.201715  7296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.008s	sys 0.008s
I20260709 12:23:55.234402  5216 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:55.200687  7303 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:55.193235  7327 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:55.240036  5148 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:55.242205  7327 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:55.235316  7296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.008s	sys 0.009s
I20260709 12:23:55.199730  7350 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:55.248101  7259 consensus_meta.cc:287] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.002s
I20260709 12:23:55.248627  7259 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.250733  7350 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:55.254465  7302 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:55.255105  7302 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:55.256381  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:55.257081  5223 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:55.259536  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:55.260154  5223 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:55.261242  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.262496  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.263173  5943 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:55.263352  5215 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:55.249931  5216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:55.264310  5216 raft_consensus.cc:606] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:55.264644  5216 raft_consensus.cc:614] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:55.266433  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:55.267452  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:55.270377  5146 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:55.273236  5943 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:23:55.273483  5141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 306) took 2366 ms (client timeout 1636 ms). Trace:
I20260709 12:23:55.273950  5115 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:55.277156  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:55.274039  5141 rpcz_store.cc:269] 0709 12:23:52.905720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.936684 (+ 30964us) service_pool.cc:224] Handling call
0709 12:23:55.269879 (+2333195us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b540019ee54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:55.270014 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1582667008}
I20260709 12:23:55.274937  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:55.280232  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:55.283145  5940 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:55.285976  7287 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:55.288471  5948 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:55.288573  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:55.290009  5941 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: []
W20260709 12:23:55.290158  7371 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 12:23:55.291483  7316 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.962s	user 0.003s	sys 0.003s
I20260709 12:23:55.292251  5075 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:55.294061  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 307) took 1505 ms. Trace:
I20260709 12:23:55.293843  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:55.294763  5939 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:55.293826  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:55.295853  7293 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:55.296228  5142 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:55.289848  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:55.297935  5946 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:23:55.294554  5149 rpcz_store.cc:276] 0709 12:23:53.788862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.795852 (+  6990us) service_pool.cc:224] Handling call
0709 12:23:53.812193 (+ 16341us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:53.812705 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:53.812738 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:53.813504 (+   766us) log.cc:844] Serialized 52 byte log entry
0709 12:23:54.134841 (+321337us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:54.254487 (+119646us) log.cc:844] Serialized 14 byte log entry
0709 12:23:54.301120 (+ 46633us) log.cc:844] Serialized 14 byte log entry
0709 12:23:54.359283 (+ 58163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:54.359381 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:54.415904 (+ 56523us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b180031c120. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:23:55.248898 (+832994us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:23:55.293793 (+ 44895us) raft_consensus.cc:1685] finished
0709 12:23:55.293816 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.293969 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38898,"prepare.queue_time_us":236484,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":143745,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":22433,"spinlock_wait_cycles":18585600,"thread_start_us":490270,"threads_started":4,"wal-append.queue_time_us":440064}
W20260709 12:23:55.300864  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.257098  7302 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:55.273728  7296 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.313503  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:55.311093  7287 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:55.307111  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:55.275710  7161 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:55.273276  7267 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:55.303557  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:55.278368  7280 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:55.297875  7333 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:55.317852  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:55.326254  5144 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:55.317951  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:55.283629  7350 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.322144  5223 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:55.333637  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:55.334244  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:55.334559  5149 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:55.296465  7293 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.317093  5943 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:55.276933  7259 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:55.284711  7297 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.328894  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:55.339141  5142 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:55.339385  5219 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:55.331459  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:55.340993  5216 raft_consensus.cc:2393] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:55.327909  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:55.335400  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:55.324385  5948 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:55.331023  6799 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:23:55.345153  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:55.346633  5076 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:55.325917  7099 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.358670  5223 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:55.357085  5149 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:55.325902  7333 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.369204  5041 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:55.350826  7297 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.376145  7297 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.336679  7301 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:23:55.336683  7350 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:55.368430  7333 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.379041  7350 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:55.324234  6649 consensus_queue.cc:1059] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:23:55.321992  7296 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.390317  5216 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:23:55.393239  7376 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:23:55.325717  6831 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:23:55.350425  7291 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:23:55.398169  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 317) took 1560 ms. Trace:
I20260709 12:23:55.324924  7287 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.389338  7293 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.360021  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:55.420639  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 51587.8us
I20260709 12:23:55.423103  5939 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
W20260709 12:23:55.413255  7255 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 1.095s	user 0.004s	sys 0.000s
I20260709 12:23:55.412249  6649 raft_consensus.cc:3055] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:55.350078  7352 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:55.385268  7331 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:23:55.414485  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 110239us
I20260709 12:23:55.430608  5041 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:55.419842  7291 consensus_queue.cc:1243] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:23:55.416795  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:55.434305  5216 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:23:55.436218  5216 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:23:55.418749  5219 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:23:55.424417  5223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:23:55.403045  5943 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:23:55.419220  5145 rpcz_store.cc:276] 0709 12:23:53.838089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.841709 (+  3620us) service_pool.cc:224] Handling call
0709 12:23:53.842258 (+   549us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:53.939915 (+ 97657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:53.939983 (+    68us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:53.940768 (+   785us) log.cc:844] Serialized 52 byte log entry
0709 12:23:54.099324 (+158556us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:54.230853 (+131529us) log.cc:844] Serialized 14 byte log entry
0709 12:23:55.366537 (+1135684us) spinlock_profiling.cc:243] Waited 3.79 ms on lock 0x7b18000cdea0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a80e34bc 00007fd8a543148b 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8a8533060 00007fd8a8545716 00007fd8a8556287
0709 12:23:55.366967 (+   430us) log.cc:844] Serialized 14 byte log entry
0709 12:23:55.367233 (+   266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.367342 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:55.397766 (+ 30424us) raft_consensus.cc:1685] finished
0709 12:23:55.397815 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.398076 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6264,"prepare.queue_time_us":179669,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":99,"raft.queue_time_us":91761,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":41824,"spinlock_wait_cycles":55413760,"thread_start_us":265283,"threads_started":3,"wal-append.queue_time_us":161376}
I20260709 12:23:55.368345  7099 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.415144  7280 raft_consensus.cc:697] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:55.425222  6717 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:55.432651  5190 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:55.418182  7267 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:55.441270  5223 raft_consensus.cc:606] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:55.426304  6649 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:55.442456  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:55.444561  5223 raft_consensus.cc:614] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:55.444902  5145 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:55.443647  5190 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:55.436254  7296 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.450196  5145 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:55.450538  7099 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.451351  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:55.451990  5145 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:23:55.447150  7267 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.454203  7297 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:55.454589  7197 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:55.454820  7297 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.455106  7197 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:55.455284  7297 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:55.456264  7267 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.456324  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 683) took 3049 ms. Trace:
I20260709 12:23:55.456185  7197 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.456954  5220 rpcz_store.cc:276] 0709 12:23:52.416356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.445075 (+ 28719us) service_pool.cc:224] Handling call
0709 12:23:52.451634 (+  6559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:52.479862 (+ 28228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:52.479895 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:52.505348 (+ 25453us) log.cc:844] Serialized 26 byte log entry
0709 12:23:52.693750 (+188402us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:52.728586 (+ 34836us) log.cc:844] Serialized 14 byte log entry
0709 12:23:52.747642 (+ 19056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:52.747756 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:54.293633 (+1545877us) spinlock_profiling.cc:243] Waited 7.73 ms on lock 0x7b18003d71a0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:23:55.412431 (+1118798us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:23:55.456077 (+ 43646us) raft_consensus.cc:1685] finished
0709 12:23:55.456101 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.456229 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27304,"prepare.queue_time_us":34575,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":37624,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":28164,"spinlock_wait_cycles":22060032,"thread_start_us":125010,"threads_started":1,"wal-append.queue_time_us":212496}
I20260709 12:23:55.445633  6831 consensus_queue.cc:1059] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:23:55.459717  6717 raft_consensus.cc:3055] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:55.460141  6717 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:55.460310  7050 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:55.461030  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 383) took 1114 ms. Trace:
I20260709 12:23:55.461073  6717 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:55.462699  6717 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.445763  7379 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:55.448644  7293 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:23:55.465893  7293 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:55.466320  7293 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:55.467890  7114 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:55.468369  7114 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:55.468698  5143 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:23:55.469103  6649 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:55.467409  7289 raft_consensus.cc:993] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: : Instructing follower ceec344dd00f40c18ef58414ce147016 to start an election
I20260709 12:23:55.469173  7114 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:55.470902  6649 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.471534  7114 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:55.472628  7287 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.444408  7301 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.461448  5069 rpcz_store.cc:276] 0709 12:23:54.346594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.425255 (+ 78661us) service_pool.cc:224] Handling call
0709 12:23:54.425521 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.052759 (+627238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:55.052790 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:55.052830 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:55.052858 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.460778 (+407920us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b5400328a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:55.460832 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.460964 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1441931136}
I20260709 12:23:55.470929  7289 raft_consensus.cc:1081] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Signalling peer ceec344dd00f40c18ef58414ce147016 to start an election
I20260709 12:23:55.450040  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.481858  7290 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.467381  7293 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.484706  5141 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138"
dest_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:56156
I20260709 12:23:55.485602  5141 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:55.485718  7293 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:55.486050  5141 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:55.490190  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:55.490527  5190 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:55.447680  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:55.491823  5143 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
W20260709 12:23:55.491779  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 44261.4us
I20260709 12:23:55.499167  7379 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:55.499686  7379 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.500171  7379 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:55.500499  7248 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:55.502239  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:55.502758  7291 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:55.503420  7291 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:55.503822  7303 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:55.504295  7303 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.504720  7303 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:55.504539  7291 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.507438  5939 pending_rounds.cc:85] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 2
I20260709 12:23:55.507841  5939 pending_rounds.cc:107] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:23:55.508097  5939 raft_consensus.cc:2889] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:55.508764  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:55.509053  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:55.509467  5147 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:55.510001  5146 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:55.510449  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:55.510898  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:55.517975  5146 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:55.498514  5141 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.488508  7290 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:55.513034  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:55.511087  5147 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:55.495079  5041 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:55.530719  5220 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:55.531878  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:55.524646  7352 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.534057  5141 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.538465  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:55.535147  7088 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:55.541092  6831 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:55.541433  7088 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:55.536361  7280 consensus_queue.cc:237] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:55.542006  7088 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:55.539661  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:55.537133  7358 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:23:55.548151  5147 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:55.533910  7277 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:55.533874  7352 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.534682  7248 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:23:55.551642  7248 raft_consensus.cc:3055] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:55.551792  7352 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:55.548209  7312 raft_consensus.cc:3055] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:23:55.553278  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.549468  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:55.456701  7191 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.555058  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:55.553314  5900 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:55.556298  5143 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:55.552074  7248 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:55.552685  7312 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:55.552016  7379 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.572592  7197 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:55.573007  7379 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.560812  7277 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:55.573837  7277 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.563861  5146 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:55.564185  7249 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.566152  7248 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.556403  7259 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:55.568630  7312 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:55.569036  5900 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:55.569756  7343 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:55.578076  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:55.599763  7343 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.600283  7343 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:55.580440  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.585493  7088 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:55.592377  7340 consensus_queue.cc:1059] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:55.594693  5114 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:55.595775  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 325) took 1060 ms. Trace:
W20260709 12:23:55.597692  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:55.605042  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.576218  7277 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.606912  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:55.607703  5943 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:55.604760  5148 rpcz_store.cc:276] 0709 12:23:54.534982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.036784 (+501802us) service_pool.cc:224] Handling call
0709 12:23:55.036987 (+   203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.240623 (+203636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.595464 (+354841us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b5400239454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:55.595698 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466284800}
I20260709 12:23:55.609079  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.610433  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.611101  5948 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 3.
I20260709 12:23:55.616743  5945 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:55.617892  5042 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:55.621507  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:55.622324  5948 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:23:55.620573  7249 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.624581  6799 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:55.618979  7290 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:55.625687  7302 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:55.626129  7290 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.607621  7334 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:55.621456  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.604367  7301 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:55.629429  5943 raft_consensus.cc:2393] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 8: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:55.614140  7231 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:55.618314  7114 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.622216  7312 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:55.618569  7379 raft_consensus.cc:3055] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:55.621111  7277 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:55.641474  7379 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:55.641965  7277 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.642566  7379 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.643600  7333 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.644124  7379 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:55.644559  7277 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.645903  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.629879  7340 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:23:55.618990  7358 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:55.631949  7248 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.644972  5147 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:55.617219  7259 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:55.653476  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.621232  6831 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:55.654227  5942 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:55.628540  7290 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.649703  7114 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:55.649115  7358 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:55.666057  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.668881  7088 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:55.670293  5068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 627) took 1321 ms (client timeout 1695 ms). Trace:
I20260709 12:23:55.671801  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:55.670737  5068 rpcz_store.cc:269] 0709 12:23:54.341743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.341873 (+   130us) service_pool.cc:224] Handling call
0709 12:23:55.662959 (+1321086us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b5400328054. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:55.663106 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26204032}
I20260709 12:23:55.680881  5948 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:55.664498  7231 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:55.687525  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:55.688520  5945 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:55.688122  7231 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.698117  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 317) took 1256 ms. Trace:
I20260709 12:23:55.698736  5221 rpcz_store.cc:276] 0709 12:23:54.441866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.442388 (+   522us) service_pool.cc:224] Handling call
0709 12:23:54.442703 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:54.539933 (+ 97230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:54.540001 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:54.540077 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:54.540124 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.697873 (+1157749us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b54002b39d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:55.697926 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.698049 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222964096}
I20260709 12:23:55.662372  7286 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:23:55.705631  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 82738.8us
I20260709 12:23:55.709429  5069 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:55.711010  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:55.711845  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:55.711489  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.713781  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:55.704548  7286 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:55.719087  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:55.722673  5939 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:55.723830  7114 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:55.724087  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:55.734783  7191 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:55.738078  5218 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:23:55.705092  7333 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.743186  5189 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:55.746240  5069 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:55.750643  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:55.752986  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:55.720427  7259 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:55.754213  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:55.754405  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:55.755069  5218 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:55.756170  7358 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:55.756095  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:55.724153  7302 raft_consensus.cc:3055] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:55.758845  7302 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:55.759917  7302 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:55.761843  5222 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:55.816176  7286 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.813392  7329 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:55.817621  7286 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:55.817960  7329 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:55.818961  7329 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:55.820119  7329 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.766506  7302 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:55.828398  7297 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:55.832636  7334 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:55.834146  5221 raft_consensus.cc:2368] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:55.803902  7056 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:55.834692  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:55.838867  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:55.804386  7249 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:55.841045  5067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 622) took 1501 ms. Trace:
I20260709 12:23:55.847460  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:55.848003  5067 rpcz_store.cc:276] 0709 12:23:54.339323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.339443 (+   120us) service_pool.cc:224] Handling call
0709 12:23:55.840844 (+1501401us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b5400328554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:55.840973 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1446670080}
I20260709 12:23:55.846629  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:55.841842  5072 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:55.852200  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:55.855599  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:55.856395  5070 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:55.857366  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:55.857669  7356 consensus_queue.cc:1059] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:55.858661  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:55.861773  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.803639  7340 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:23:55.823098  7333 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.906075  7333 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:55.870283  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.870702  5943 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:23:55.876888  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.864476  7231 raft_consensus.cc:2764] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:23:55.908378  7333 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:55.886772  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:55.863837  7249 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:55.887753  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:55.888499  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:55.911180  5070 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:55.912101  7249 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:55.914750  5900 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:55.916111  7285 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:55.918581  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:23:55.919167  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.919237  5943 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.920692  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.858706  7356 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:55.922325  7356 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:55.922534  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 380) took 1577 ms. Trace:
I20260709 12:23:55.923370  7356 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.924681  7356 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.895601  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:55.890627  5947 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:55.837173  7297 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:55.926419  7114 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:55.927037  7114 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:55.899816  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:55.928980  5218 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:55.930655  6889 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:55.931092  6889 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:55.932015  6889 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:55.936843  6649 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:55.921756  7285 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:55.902400  6877 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:55.921591  7291 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:23:55.922957  5074 rpcz_store.cc:276] 0709 12:23:54.345638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.423597 (+ 77959us) service_pool.cc:224] Handling call
0709 12:23:55.663431 (+1239834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.663730 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:55.663772 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:55.663847 (+    75us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:55.663899 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.663984 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.922307 (+258323us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400328050. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:55.922464 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2850495232}
I20260709 12:23:55.921523  5946 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:55.873299  7303 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:55.900156  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.957867  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:55.958079  7056 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:55.959275  5945 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:55.959172  7056 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:55.959879  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:55.944983  5218 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:55.933302  6889 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:55.951179  7087 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:23:55.970060  5217 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
W20260709 12:23:55.988263  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.007773  5943 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
W20260709 12:23:56.021732  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 60002.2us
I20260709 12:23:56.022810  7114 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:55.966630  7333 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:56.024479  7297 raft_consensus.cc:3055] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:56.024953  7297 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:56.025990  7297 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:56.027182  7297 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:56.038789  5218 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:56.040086  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:56.040800  5218 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
W20260709 12:23:56.041087  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.042196  5074 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:56.042395  5222 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:23:56.046602  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.047385  5945 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:55.963579  7087 raft_consensus.cc:3055] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:56.075450  5068 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:56.095878  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:56.094084  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:56.093331  7388 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:23:56.075742  5216 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
W20260709 12:23:56.097316  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 44065.9us
I20260709 12:23:56.099920  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:55.970090  7278 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:56.106699  5220 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:55.954293  7343 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:56.112957  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.119576  5188 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:56.109181  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:56.120862  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:56.120990  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:56.113941  7299 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:56.114792  7173 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:56.127027  7173 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.127473  7173 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:56.112155  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:56.020942  6717 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:56.122332  7197 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:56.124348  6877 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:56.094178  7333 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:56.094676  7087 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:23:56.139449  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.147392  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:56.147305  7323 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:56.153581  5944 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 8: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:56.155387  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:56.111243  7278 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:56.157125  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:56.143083  7331 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:56.133140  5070 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:56.131685  7050 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:56.148380  6649 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:56.171957  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:56.162737  7290 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:56.133176  7299 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:56.176970  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:56.192277  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:56.174072  7312 consensus_meta.cc:287] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 12:23:56.148121  6717 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:56.195132  5188 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:56.154244  7386 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:23:56.202490  7333 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:56.151841  7197 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:56.291230  7197 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:56.180392  7340 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:56.294574  7299 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:56.194481  6877 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:56.194145  7265 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.296030  7265 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:56.296778  7289 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:56.297386  7289 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:56.298384  7265 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:56.122960  7285 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:56.299336  7394 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:56.301898  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:56.206020  7087 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:56.160313  7395 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:23:56.181771  7331 raft_consensus.cc:697] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:56.167910  7278 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:56.225400  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:56.236240  6717 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:56.240283  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:56.241026  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 110981us
I20260709 12:23:56.223758  7343 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:56.252259  7267 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.255616  7303 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:56.266451  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:56.329946  5942 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 8: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:56.266987  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:56.331337  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 64447.2us
W20260709 12:23:56.274778  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 79733.9us
I20260709 12:23:56.276351  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 390) took 1776 ms. Trace:
I20260709 12:23:56.333771  7197 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:23:56.259122  7391 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:23:56.302110  5074 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:23:56.335985  5074 pending_rounds.cc:85] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 3
I20260709 12:23:56.336251  5074 pending_rounds.cc:107] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:23:56.336493  5074 raft_consensus.cc:2889] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:56.337787  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:56.337936  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:56.338464  5074 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:56.158437  7277 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:23:56.339362  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:56.340073  7277 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:56.173242  6649 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:56.299358  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:56.340399  5215 raft_consensus.cc:2410] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:23:56.340659  6649 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:56.341076  5076 raft_consensus.cc:2410] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:23:56.304857  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:56.342656  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:56.342741  7352 consensus_queue.cc:1059] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:56.343897  7352 raft_consensus.cc:3055] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:56.311334  5041 leader_election.cc:400] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:56.343957  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:56.344470  5041 leader_election.cc:403] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:23:56.344904  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:56.345369  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:56.345734  5220 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:23:56.347009  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:56.347463  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.348773  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:56.333793  5072 rpcz_store.cc:276] 0709 12:23:54.500122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.787222 (+287100us) service_pool.cc:224] Handling call
0709 12:23:55.841655 (+1054433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.852400 (+ 10745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.276054 (+423654us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400328550. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:56.276251 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2424301440}
I20260709 12:23:56.345762  5073 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:23:56.354422  5073 pending_rounds.cc:85] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 2
I20260709 12:23:56.354688  5073 pending_rounds.cc:107] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:23:56.354102  7331 consensus_queue.cc:237] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:56.355103  5073 raft_consensus.cc:2889] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:56.356885  5073 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:56.357167  5072 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:56.328146  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:56.374536  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 46510us
I20260709 12:23:56.330103  7346 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:56.376770  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:56.377525  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:56.305946  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:56.344367  7352 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:56.339377  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:56.387962  5074 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:56.341351  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:56.359323  5223 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:56.311177  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:56.318554  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:56.375207  5070 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:56.375865  7323 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:56.333232  7173 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:56.376389  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:56.341933  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:56.310561  7302 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:56.378576  5075 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:23:56.348033  7114 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5.
I20260709 12:23:56.396855  5219 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:56.386229  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:56.401059  5899 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:56.402591  5068 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:56.415098  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:56.415405  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:56.416348  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:56.410351  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:56.417754  5899 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:56.340452  7277 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:56.366360  7293 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:56.429126  5899 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:56.429196  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:56.421787  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:56.422943  5224 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:23:56.384367  7087 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:56.380528  7290 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.319388  6799 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 12:23:56.298353  7289 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:56.424331  5074 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:56.377583  7332 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:56.350278  7267 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:56.427632  7242 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:56.424319  7348 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:56.424141  7356 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:56.446363  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:56.435166  7293 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:23:56.454494  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 126950us
I20260709 12:23:56.402371  7302 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:56.456302  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.455309  5899 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:56.461804  5898 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:56.445796  7314 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:23:56.453508  7329 consensus_meta.cc:287] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.004s
I20260709 12:23:56.444559  7395 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:23:56.460191  7293 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:56.456360  7284 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:56.446019  7290 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:56.450193  7332 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:56.470152  7343 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:56.490864  6717 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.468708  7348 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:23:56.482905  7241 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:56.494733  7303 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.513248  5216 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:56.506126  5215 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:56.481581  7329 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:56.510000  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 121793us
I20260709 12:23:56.522675  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:56.524176  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.460664  7352 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:56.494536  7343 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.467165  7277 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:56.546640  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 228156us
W20260709 12:23:56.548532  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 80507.5us
I20260709 12:23:56.547760  5114 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:56.554250  5147 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:23:56.554459  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 88992.4us
I20260709 12:23:56.558038  5216 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:56.559449  7314 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:56.560118  7290 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:56.562294  5146 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:56.562822  7409 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:56.563351  7302 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:56.535351  7400 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:56.508769  6717 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:56.579051  5898 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:56.579447  5149 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:56.580691  5142 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:56.581032  5898 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:23:56.538486  7343 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:56.569701  5147 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:56.585830  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:56.550354  7297 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.523s	user 0.012s	sys 0.004s
I20260709 12:23:56.567082  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:56.587015  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:56.572877  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:56.553045  7352 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:56.594476  5146 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:56.595111  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.595261  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:56.596266  5141 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:23:56.600219  5150 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:56.601055  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:56.601893  5141 raft_consensus.cc:2393] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:56.532097  7303 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:56.606072  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:23:56.605351  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 57730.9us
I20260709 12:23:56.608764  5149 raft_consensus.cc:2393] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 4: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:56.610342  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:56.611143  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:56.610692  5147 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:56.612159  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:56.612818  5141 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:56.614704  5149 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:23:56.566152  7267 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:56.618240  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:56.618495  5144 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:56.619114  5941 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:56.621765  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:56.621832  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:56.624750  5144 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:56.634809  5146 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:23:56.597918  7404 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:56.634673  5218 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:56.641281  5146 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:56.635970  5148 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:56.612040  7173 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:56.647445  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:56.658638  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.628909  7331 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:56.588660  7314 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:23:56.637177  7259 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:56.701793  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:56.702749  5147 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:56.685326  7331 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.673573  7284 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:56.698892  7341 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:23:56.751673  5899 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.68:43449 timed out after 1.659s (SENT) [suppressed 14 similar messages]
I20260709 12:23:56.686096  7404 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:56.752199  5941 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:56.755503  5144 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:56.760033  5147 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:56.760357  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:56.750998  7331 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:56.711930  7259 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:23:56.789949  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.782083  7173 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:56.799108  7406 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 12:23:56.777326  7259 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:56.806525  5150 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:56.808004  6717 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:56.809446  7303 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:56.789927  7410 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.768040  7284 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:56.758419  7332 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:56.852533  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:56.852824  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:56.853443  5141 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:23:56.833261  7410 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:56.854275  7231 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:56.855122  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:56.858490  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:56.859184  5147 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:56.862720  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:56.864365  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:56.867131  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:56.803192  7173 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:56.848376  7404 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:56.874595  6717 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:56.880653  6921 raft_consensus.cc:3055] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:56.881198  6921 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:56.885911  5145 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:56.882139  6921 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:56.892041  6921 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:56.884346  7419 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:56.872623  7087 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:23:56.885391  7331 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:56.887089  7411 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
W20260709 12:23:56.957942  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 80643.9us
W20260709 12:23:56.957130  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 205485us
I20260709 12:23:56.949144  7335 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:56.963675  7242 consensus_meta.cc:287] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.002s
I20260709 12:23:56.952595  7296 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:56.971855  5249 heartbeater.cc:646] Failed to heartbeat to 127.1.123.126:38285 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.1.123.126:38285 timed out after 15.000s (SENT)
I20260709 12:23:56.964026  7314 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:23:56.966975  7289 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:23:56.978917  7242 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:57.008970  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:57.010278  5900 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:57.015986  5042 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:23:56.996996  7415 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:23:57.018932  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:57.018961  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.975185  7335 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:57.025827  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:57.027532  5149 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 8: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:57.026434  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.029657  7297 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:57.029940  7423 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 12:23:57.031939  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.032138  7332 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.032712  7332 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:57.025902  6877 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:23:57.035986  5224 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:23:57.036134  7432 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:57.034934  7332 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:57.037058  7432 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:57.037438  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:23:57.038290  7420 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:23:57.038476  5215 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:56.979454  7296 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:57.040680  5220 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:23:57.041163  5220 pending_rounds.cc:85] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 4
I20260709 12:23:57.041435  5220 pending_rounds.cc:107] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:23:57.041719  5220 raft_consensus.cc:2889] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:57.042088  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.029981  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:57.047468  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.048468  5222 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:57.037735  7432 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:23:57.050160  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.013901  7056 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:57.059772  5146 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:23:57.061069  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:57.062168  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:57.020170  7413 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:23:57.063498  5222 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:56.988420  7424 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 12:23:57.019598  7050 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.067317  5215 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:23:57.068817  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.020828  7418 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:57.025784  7411 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:57.022284  7334 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:57.083026  5149 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:57.076288  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:57.030148  7267 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:57.064105  7293 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.569s	user 0.014s	sys 0.000s
I20260709 12:23:57.080123  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:57.077565  7387 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
W20260709 12:23:57.091643  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 69945.8us
I20260709 12:23:57.102452  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:57.097189  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.070411  7297 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:57.023803  7303 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:23:57.071770  7395 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:57.094257  7439 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.103798  5071 raft_consensus.cc:2410] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:23:57.098969  5067 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:57.108570  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:57.074972  7332 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:57.084251  7231 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:57.063565  7056 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:57.087662  6877 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:23:57.126083  7442 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:23:57.099845  7284 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:57.107070  7289 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:57.101380  7411 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:57.108007  7297 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.089818  7197 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:23:57.081249  7050 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:57.132143  7231 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.138857  7440 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:57.139580  7296 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:57.153182  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 44714.8us
I20260709 12:23:57.106726  7293 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:57.126668  7352 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.001s
W20260709 12:23:57.122956  7343 consensus_meta.cc:287] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.515s	user 0.013s	sys 0.000s
I20260709 12:23:57.096493  7334 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:57.151224  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 71251.5us
W20260709 12:23:57.147063  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 70907.4us
W20260709 12:23:57.151467  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 49157.7us
I20260709 12:23:57.140110  7335 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:57.104414  7267 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:57.118055  7314 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:57.107782  7409 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
W20260709 12:23:57.156806  7278 consensus_meta.cc:287] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.830s	user 0.014s	sys 0.000s
I20260709 12:23:57.206118  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:57.132206  7431 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:23:57.149713  7297 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:57.081878  7418 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:57.181002  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.183013  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.185783  5224 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:23:57.149937  7414 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:57.194191  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:23:57.158931  6649 consensus_meta.cc:287] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.818s	user 0.014s	sys 0.000s
I20260709 12:23:57.158908  7050 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:57.230731  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.232378  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:57.205364  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:57.209685  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:57.228001  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:57.233139  5224 raft_consensus.cc:2376] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5. Re-sending same reply.
I20260709 12:23:57.215677  5145 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:23:57.233498  5149 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:57.233898  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:57.223191  5223 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:57.200253  7314 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.128191  7439 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:57.232478  5070 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:57.225298  7293 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:57.231513  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.236644  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:57.244004  5067 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:23:57.238297  7431 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:57.238481  5899 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:57.196192  7440 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:23:57.244954  7414 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:57.257300  7414 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:57.259070  7267 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:57.259605  7418 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:57.160939  7411 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:57.214301  7278 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:57.264505  7334 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:57.265230  5143 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:57.265735  7431 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.266896  7439 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:57.194065  7343 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.269335  7343 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:57.200281  7387 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:23:57.270926  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.264691  7278 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:57.271631  5067 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:57.272246  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:57.291234  5070 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:57.243541  7314 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:57.273612  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:57.201680  7332 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:57.225373  7056 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:57.230173  7296 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:57.240701  6649 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:57.151985  7432 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:57.310756  5070 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:57.292384  7343 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:23:57.336743  7393 log.cc:927] Time spent T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 1.218s	user 0.022s	sys 0.025s
I20260709 12:23:57.339087  5069 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:57.347188  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:57.347728  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.348963  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.349692  5942 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:57.350764  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:57.353654  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.289824  7259 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.354455  5070 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:57.354575  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.367197  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:57.306104  7231 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:57.355127  5217 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:57.368947  5068 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:23:57.369414  5068 pending_rounds.cc:85] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 2
I20260709 12:23:57.369689  5068 pending_rounds.cc:107] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:23:57.369940  5068 raft_consensus.cc:2889] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:57.367372  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.371595  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.374573  5943 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:57.375146  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.376111  5941 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:57.377034  7259 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:57.373121  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.382299  5070 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:57.310312  7444 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:57.385113  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.392218  5217 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:57.384135  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:57.393355  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:57.394361  5114 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:57.395042  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.395996  5217 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:57.396543  7278 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:57.397174  7278 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.397661  7278 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:57.398624  7442 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:57.391269  5218 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:57.340233  7343 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:57.355002  7267 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.339202  6649 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:57.414575  5067 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:57.312284  7296 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:57.384920  7409 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:57.381098  7431 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:57.421433  5943 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:57.418883  7332 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:57.435374  7296 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:57.468797  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:57.470175  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.471446  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:57.472460  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:57.474113  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:57.475822  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:57.478251  5070 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:57.480613  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.419534  7267 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:57.480278  7291 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:57.483299  7267 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:57.422817  7446 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 12:23:57.469869  7296 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.490345  7296 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:57.492184  7452 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:23:57.492926  7296 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:57.494102  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.494926  5070 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:57.499214  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:23:57.481375  7453 log.cc:927] Time spent T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.091s	user 0.009s	sys 0.011s
I20260709 12:23:57.505256  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 406) took 1208 ms. Trace:
I20260709 12:23:57.509009  7289 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.511696  5069 rpcz_store.cc:276] 0709 12:23:56.296771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.302604 (+  5833us) service_pool.cc:224] Handling call
0709 12:23:56.302909 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.339768 (+1036859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.504978 (+165210us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400327b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:57.505166 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2382378880}
I20260709 12:23:57.480432  7387 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:57.505688  7446 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:57.518049  7352 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:57.518378  7446 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.518986  7446 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:57.518774  7352 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:57.523874  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.524709  5215 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:57.526768  7352 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:57.515985  7289 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:57.480113  7343 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:57.528038  7387 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:57.515654  7291 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:23:57.546042  7387 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:57.497573  7418 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:23:57.529325  5899 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:57.556329  7249 raft_consensus.cc:3055] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:57.556878  7249 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:23:57.558216  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 336) took 1178 ms. Trace:
I20260709 12:23:57.558688  5216 rpcz_store.cc:276] 0709 12:23:56.380366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.385953 (+  5587us) service_pool.cc:224] Handling call
0709 12:23:57.558041 (+1172088us) spinlock_profiling.cc:243] Waited 98.9 ms on lock 0x7b54003caad4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:57.558155 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227397504}
I20260709 12:23:57.573088  5216 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:57.576561  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.578022  5069 raft_consensus.cc:2410] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:23:57.580508  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 64293.6us
I20260709 12:23:57.513892  7296 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:57.585862  7249 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:57.592730  7386 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:57.593230  7386 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:57.594303  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.594240  7386 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:57.594869  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.595450  7386 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:57.595829  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.596279  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:57.597702  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:57.598469  5945 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:57.598527  7387 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.600124  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.603232  7421 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:57.603911  7421 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.595563  5942 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 3.
I20260709 12:23:57.612048  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:57.621831  7284 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:23:57.624712  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 95510us
I20260709 12:23:57.625406  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:57.625810  7277 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:57.626992  7191 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:57.627914  7394 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:57.628605  7424 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:57.629060  7424 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:57.607960  7421 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:57.629675  7424 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:57.630376  5945 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:23:57.629081  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:57.622589  7284 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:57.587330  7333 raft_consensus.cc:476] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Not starting pre-election -- already a leader
I20260709 12:23:57.636360  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:57.586370  7289 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:57.638964  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:57.646869  5898 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:57.627431  7314 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:57.588318  7296 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:57.638360  7285 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:23:57.650710  7241 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:57.645979  5188 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:57.644829  7248 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:57.659906  7284 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:57.663817  5943 raft_consensus.cc:2368] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8.
W20260709 12:23:57.664500  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.664924  7314 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:57.645979  7411 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:57.651224  7297 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:57.665647  7411 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:57.666150  7411 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:57.667310  7297 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:57.658448  7421 consensus_queue.cc:1059] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:57.667995  7312 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:57.668399  5224 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:57.668557  7312 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.670928  7312 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:57.672873  7173 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:57.673906  7410 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:57.653636  7428 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:23:57.648869  7431 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:57.676857  7431 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:57.677285  7431 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:57.678162  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 344) took 1124 ms. Trace:
I20260709 12:23:57.678598  5143 rpcz_store.cc:276] 0709 12:23:56.554172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.556226 (+  2054us) service_pool.cc:224] Handling call
0709 12:23:56.556504 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.265917 (+709413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.677922 (+412005us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b5400837554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:57.678100 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629396352}
I20260709 12:23:57.679787  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 341) took 1297 ms. Trace:
I20260709 12:23:57.659288  7241 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:23:57.659646  7383 log.cc:927] Time spent T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 1.353s	user 0.013s	sys 0.000s
I20260709 12:23:57.680222  5223 rpcz_store.cc:276] 0709 12:23:56.381925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.418170 (+ 36245us) service_pool.cc:224] Handling call
0709 12:23:57.203849 (+785679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.234755 (+ 30906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.679592 (+444837us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b54003b7c50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:57.679725 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1806159232}
I20260709 12:23:57.684660  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.653170  7446 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:57.696273  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.678807  7296 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:57.680691  6877 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:57.739846  7265 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.831775  7265 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:57.832387  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:57.833066  5217 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:57.696601  7284 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:23:57.834030  7284 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:57.834257  7265 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:57.834443  7284 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:57.757223  7446 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:57.757223  7387 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:57.644671  7349 consensus_queue.cc:1059] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
W20260709 12:23:57.758910  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 67769.6us
I20260709 12:23:57.759732  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.760942  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:57.842778  5070 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:57.250797  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:57.668514  7421 raft_consensus.cc:3055] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:57.705984  7443 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:23:57.681846  7428 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.802067  5190 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:57.803784  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.804116  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.804608  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:57.803793  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:57.849786  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.850673  5215 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
W20260709 12:23:57.807657  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.808871  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.812148  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.814783  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:57.817783  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.823437  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.823879  5948 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:57.854768  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:57.820851  7087 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:23:57.827989  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:57.869590  5221 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:57.828487  7241 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.819020  7278 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.830749  6877 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:57.797256  7343 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.838351  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 850) took 1262 ms. Trace:
I20260709 12:23:57.921357  5150 rpcz_store.cc:276] 0709 12:23:56.575625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.601060 (+ 25435us) service_pool.cc:224] Handling call
0709 12:23:56.614529 (+ 13469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.807242 (+192713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.838075 (+1030833us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400239954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:57.838285 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440856064}
I20260709 12:23:57.835414  7284 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:57.846220  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.923852  7343 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:57.925146  7454 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.500s	user 0.004s	sys 0.012s
I20260709 12:23:57.856025  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:57.925927  5224 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:57.868199  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:57.882910  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 80926.2us
I20260709 12:23:57.883183  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:57.890877  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:57.928067  5944 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:57.928465  5068 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:57.886996  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:57.822010  7424 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:57.896209  7278 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:57.845849  7421 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:57.932222  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:57.932732  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:57.925220  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:57.904011  7460 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.321s	user 0.003s	sys 0.000s
I20260709 12:23:57.908004  7056 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:57.901122  7087 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:57.870640  7356 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:23:57.910984  5069 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:23:57.913750  7395 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.915125  7291 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:57.821518  7411 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.934626  7446 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:23:57.919903  7453 log.cc:927] Time spent T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:23:57.927271  5073 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:23:57.928090  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 60001.6us
I20260709 12:23:57.867283  7329 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:57.916878  7431 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.871417  7333 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:57.939283  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:58.028474  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 89294.7us
I20260709 12:23:58.029282  5041 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:58.030544  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:57.942283  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:57.906845  6877 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:58.031986  5076 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:58.032446  7412 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:57.956769  5940 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:58.034293  6717 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:23:58.034849  7412 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:58.035130  6717 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:58.035341  7412 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:58.035596  6717 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:57.926486  7343 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:58.035992  6717 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:57.961329  5224 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:23:57.919073  5070 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:23:57.940537  7056 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.992134  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:57.867684  7428 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:57.965379  7296 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:58.000289  5068 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:23:58.009820  5217 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:23:57.944051  7087 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:58.020114  5115 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:58.022576  7463 log.cc:927] Time spent T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:23:57.936420  7277 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:23:58.022742  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:58.065130  5939 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:57.964587  7395 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:58.026062  7331 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:57.944064  6889 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:58.050792  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45003.6us
I20260709 12:23:58.063552  5041 leader_election.cc:400] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8.
I20260709 12:23:58.069957  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:58.037804  7056 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:58.061587  7422 log.cc:927] Time spent T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:23:57.976377  7291 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:23:58.056988  7435 log.cc:927] Time spent T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:23:58.059540  7302 raft_consensus.cc:2764] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
W20260709 12:23:58.079943  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 60018us
I20260709 12:23:58.121026  7332 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:58.127432  7284 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:58.128453  7428 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:58.073341  7329 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:58.129498  7329 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:58.131794  7329 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:58.134084  7291 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:58.137468  7411 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:58.140343  7481 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.141186  7481 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:58.145087  7481 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:58.086686  7395 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:58.152674  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:58.087316  7446 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:58.154309  7446 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:58.156450  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.157347  5147 raft_consensus.cc:2368] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:58.157452  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:58.089062  7410 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:58.089926  7400 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:58.037823  7346 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:23:58.104897  7056 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:58.105654  5948 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:58.113737  7431 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:58.171840  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.172120  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.172705  5149 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:23:58.173190  5147 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:58.176790  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:58.150616  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.133339  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:58.181288  5073 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:58.061487  7323 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:58.156629  7446 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:58.192343  7446 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:58.192229  7323 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:58.193071  7446 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:58.080035  7432 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.158073  5146 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:58.195353  7323 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:58.195708  7432 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:58.159446  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 350) took 1096 ms. Trace:
I20260709 12:23:58.198182  5220 rpcz_store.cc:276] 0709 12:23:57.062768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.062932 (+   164us) service_pool.cc:224] Handling call
0709 12:23:57.081169 (+ 18237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.758439 (+677270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.758476 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.758522 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:57.758568 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:58.157378 (+398810us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54004830d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:58.157431 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:58.159382 (+  1951us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449337216}
I20260709 12:23:58.159806  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.199084  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:58.162825  5076 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:58.200248  5072 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:23:58.158223  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:58.186065  7400 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:58.187680  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:58.202381  5147 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:58.203588  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:58.204515  5147 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:58.203701  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:58.046916  7421 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:58.076295  7333 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 12:23:58.215826  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.205564  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.218163  5147 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:23:58.218964  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.220575  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:58.220599  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.210434  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:58.195391  7446 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:58.222904  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.225574  5146 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:58.151628  7191 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:23:58.230100  5146 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:58.230636  7421 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:58.234441  7323 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:58.235278  7323 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:58.238106  7346 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:58.238898  7346 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:58.239490  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:58.239907  6877 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:58.208613  7197 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.243501  7197 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:58.221868  7331 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:58.246052  7331 raft_consensus.cc:697] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:58.247334  7331 consensus_queue.cc:237] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:58.253189  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.254253  5149 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:58.197842  6799 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:58.265882  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 191405us
I20260709 12:23:58.275100  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:58.279577  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:58.281494  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:58.282083  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:58.283686  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:58.284632  5070 raft_consensus.cc:2368] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:58.284924  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:58.285787  5067 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:58.290673  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:58.296172  5898 leader_election.cc:400] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:58.296561  5898 leader_election.cc:403] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:23:58.287514  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:58.297380  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:58.271986  7346 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:58.280414  5223 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:58.223629  7087 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:58.306310  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:23:58.322163  7477 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:23:58.275475  7323 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:58.271261  7446 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:58.305995  7312 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:58.328966  6877 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:58.302836  7242 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:58.559867  7056 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:58.353621  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:58.270139  6799 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:58.319972  7197 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:58.542425  5220 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:23:58.396085  7411 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:58.391381  7428 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:58.481365  7346 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:58.579164  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 281688us
I20260709 12:23:58.569144  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:58.520298  7278 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.568248  7242 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:58.539325  7050 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.306560  7400 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:58.577742  7484 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.582899  7428 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.590361  7323 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:58.639720  7329 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:58.613780  7087 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:58.564047  7446 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:58.604652  7278 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:58.677736  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.592324  7346 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:58.713847  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.657156  7428 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:58.632761  7291 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:58.722337  5147 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:23:58.614738  7050 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:58.734633  7482 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:23:58.742471  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.743286  5143 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:58.744204  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:58.744756  5143 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:58.748025  7050 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:58.749557  7412 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.714s	user 0.015s	sys 0.000s
W20260709 12:23:58.739588  7410 consensus_meta.cc:287] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.570s	user 0.016s	sys 0.004s
I20260709 12:23:58.752157  7410 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:58.752032  7412 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:58.728104  7242 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:58.750396  7346 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:58.746011  7411 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 4. Current term is 5.
I20260709 12:23:58.756220  7412 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:58.757501  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:58.758597  7476 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.571s	user 0.005s	sys 0.000s
I20260709 12:23:58.759084  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:58.762202  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:58.763823  5148 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:58.765902  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:58.766410  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 435) took 1077 ms. Trace:
I20260709 12:23:58.766924  5069 rpcz_store.cc:276] 0709 12:23:57.689368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.759433 (+ 70065us) service_pool.cc:224] Handling call
0709 12:23:58.766176 (+1006743us) spinlock_profiling.cc:243] Waited 68.2 ms on lock 0x7b5400347454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:58.766320 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156857216}
I20260709 12:23:58.769807  7346 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.770414  7346 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:58.771390  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:58.772137  5071 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:58.772724  7346 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:58.738075  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:58.774811  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:58.775990  7191 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:58.776412  7191 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.776650  5147 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:58.776770  7191 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:58.758478  5947 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:58.783911  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 932) took 1439 ms. Trace:
I20260709 12:23:58.753283  5142 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:23:58.794526  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.784351  5219 rpcz_store.cc:276] 0709 12:23:57.344630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.367039 (+ 22409us) service_pool.cc:224] Handling call
0709 12:23:57.403956 (+ 36917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.468925 (+ 64969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.468973 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.469027 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:57.469079 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:58.783645 (+1314566us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b540067ff54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:58.783695 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:58.783846 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54646656}
I20260709 12:23:58.663946  7442 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:23:58.801949  5899 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:58.806124  7483 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.813351  7444 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:58.813975  7444 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.814332  7444 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:58.814771  7446 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:58.656692  7484 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:58.821591  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:58.782859  7488 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:23:58.822976  5189 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:58.674324  7323 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:58.825938  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:58.826795  5220 raft_consensus.cc:2368] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7.
I20260709 12:23:58.829110  6877 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:58.832181  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 230339us
I20260709 12:23:58.832935  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 604) took 1028 ms. Trace:
I20260709 12:23:58.835093  5940 rpcz_store.cc:276] 0709 12:23:57.804843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.805052 (+   209us) service_pool.cc:224] Handling call
0709 12:23:58.832656 (+1027604us) spinlock_profiling.cc:243] Waited 87.6 ms on lock 0x7b540000a854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:58.832845 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201476352}
I20260709 12:23:58.837930  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:58.849188  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:58.801518  7289 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:58.851054  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:58.859186  5220 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:58.852656  5899 leader_election.cc:400] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7.
I20260709 12:23:58.871855  5899 leader_election.cc:403] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:23:58.829955  6877 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:58.867010  7231 raft_consensus.cc:3055] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:23:58.857658  7284 consensus_meta.cc:287] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.729s	user 0.012s	sys 0.004s
I20260709 12:23:58.803296  7291 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:58.888381  5943 raft_consensus.cc:2410] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:23:58.875092  6877 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.894905  6877 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:58.898723  6799 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:58.899781  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.904295  7424 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:58.905673  6889 consensus_meta.cc:287] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.830s	user 0.009s	sys 0.004s
I20260709 12:23:58.906193  6889 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:58.907316  5068 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:58.910403  7191 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:58.865151  7289 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:58.831532  7483 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:58.917580  7446 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:58.918982  7484 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:58.873346  7346 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:58.884996  7430 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:58.921944  7318 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:58.922367  7318 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:23:58.923202  7318 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:58.946951  5947 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:58.947198  5223 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:23:58.898593  7491 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.177s	user 0.001s	sys 0.001s
I20260709 12:23:58.872709  7231 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:23:58.967147  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:58.968251  5071 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:58.973848  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:58.897344  7432 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:58.979092  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:58.980603  5940 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:23:58.954505  7249 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:58.952188  6889 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:58.957991  7050 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:58.996205  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:23:58.986292  6717 consensus_meta.cc:287] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.950s	user 0.015s	sys 0.000s
I20260709 12:23:58.959522  7387 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:58.958942  7312 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:58.987098  7278 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:59.024071  5898 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:59.001359  7302 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:23:59.029368  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 407389us
W20260709 12:23:59.022492  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 58574.4us
I20260709 12:23:59.017035  7191 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:58.981047  7485 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:23:59.032780  6921 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:59.022559  6717 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:59.040769  7302 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.011322  6889 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:59.032102  7412 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:23:59.014639  7050 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:59.041352  5143 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:23:59.049614  5898 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:59.032986  7387 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:59.053753  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:59.055449  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:59.041069  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:23:59.066517  5072 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
W20260709 12:23:59.086269  7486 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:23:59.092667  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 611) took 1160 ms. Trace:
I20260709 12:23:59.032699  7231 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:59.067206  6921 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:59.099020  7323 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:59.098230  7291 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:59.050990  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.106297  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:59.093433  5943 rpcz_store.cc:276] 0709 12:23:57.931798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.931955 (+   157us) service_pool.cc:224] Handling call
0709 12:23:59.092338 (+1160383us) spinlock_profiling.cc:243] Waited 955 ms on lock 0x7b5400008f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:23:59.092542 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2196393856}
I20260709 12:23:59.113509  5220 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:23:59.092855  7444 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:59.100839  6921 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:59.051508  7424 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:59.082870  7404 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:59.072544  7302 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:59.104180  6889 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:23:59.139010  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.146652  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.124512  7197 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:23:59.148908  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 297949us
W20260709 12:23:59.121332  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 80395us
I20260709 12:23:59.122812  7087 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:23:59.137902  7424 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:59.302325  7424 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:59.302892  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 860) took 1126 ms. Trace:
I20260709 12:23:59.303339  5149 rpcz_store.cc:276] 0709 12:23:58.176800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.255145 (+ 78345us) service_pool.cc:224] Handling call
0709 12:23:58.262683 (+  7538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:58.272641 (+  9958us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:58.272677 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:58.296178 (+ 23501us) log.cc:844] Serialized 26 byte log entry
0709 12:23:58.554198 (+258020us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:58.875474 (+321276us) log.cc:844] Serialized 14 byte log entry
0709 12:23:59.302521 (+427047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.302609 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:59.302656 (+    47us) raft_consensus.cc:1685] finished
0709 12:23:59.302672 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:59.302827 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40826,"prepare.queue_time_us":190407,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":43623,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":6085,"thread_start_us":692011,"threads_started":3,"wal-append.queue_time_us":423471}
W20260709 12:23:59.304982  6649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.045s	sys 0.071s
W20260709 12:23:59.305423  6649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.045s	sys 0.072s
I20260709 12:23:59.309425  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:59.309945  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:59.311594  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:23:59.161657  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.314033  5144 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:59.164983  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:23:59.147702  7430 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7.
I20260709 12:23:59.172012  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:59.172959  5223 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:59.152649  7197 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.177888  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:23:59.154464  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 329142us
I20260709 12:23:59.182060  5944 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:23:59.327622  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.329679  7302 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:59.330785  7428 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.541s	user 0.012s	sys 0.001s
I20260709 12:23:59.331471  7428 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:59.332403  7302 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:59.334532  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.336413  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.191442  7387 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:59.162648  7484 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.194914  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.349108  5946 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:59.350224  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:59.350863  5946 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:59.325277  7273 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:59.352782  7273 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:23:59.353199  7273 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:59.333505  7197 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:59.205359  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.354156  5216 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:23:59.205107  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.348184  7430 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:59.208114  7444 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:59.211512  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.212347  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.358142  7430 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:59.213106  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.359362  5942 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:23:59.359591  7484 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:59.219915  7483 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:59.221907  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.362587  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.222934  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.363991  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.364384  5948 raft_consensus.cc:2368] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
I20260709 12:23:59.231384  7388 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:59.365715  7388 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:59.366730  7388 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:59.231755  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.367514  5946 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:23:59.367867  7388 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:59.368533  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.369442  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:59.371896  5946 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:23:59.372957  5945 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:59.243917  4899 catalog_manager.cc:5697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 2 to 6, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 6 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.252275  7231 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:59.254622  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.385963  5215 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:23:59.256065  7289 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:59.365382  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.263238  7050 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:59.264304  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.365243  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.267336  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.268016  7432 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.276600  7318 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:59.134667  7312 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:59.283811  4895 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43234 (request call id 124) took 17857 ms (client timeout 15000 ms). Trace:
I20260709 12:23:59.285318  7505 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.288380  7400 consensus_queue.cc:1059] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:59.487402  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.288281  6877 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:59.494419  6877 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:59.290442  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:59.299985  5076 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.007s	sys 0.008s
W20260709 12:23:59.495994  5076 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.008s	sys 0.008s
I20260709 12:23:59.496950  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 902) took 1216 ms. Trace:
W20260709 12:23:59.316095  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 151236us
I20260709 12:23:59.498342  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:59.499222  7430 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:23:59.337551  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.501672  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:59.502816  6649 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:59.503230  7428 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:59.503417  6649 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:59.483517  6921 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:59.339376  5947 raft_consensus.cc:2368] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8.
I20260709 12:23:59.505291  6921 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:59.505920  6649 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:59.507970  6921 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:59.391659  5220 raft_consensus.cc:2368] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
I20260709 12:23:59.397926  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:23:59.508388  7483 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.508787  5144 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:23:59.198953  6717 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:59.509012  7483 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:59.401822  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:23:59.393642  7242 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.007s	sys 0.003s
W20260709 12:23:59.510602  7242 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.013s	sys 0.006s
I20260709 12:23:59.489686  7248 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.511992  7248 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:59.389012  4899 catalog_manager.cc:5697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
W20260709 12:23:59.514027  7249 consensus_meta.cc:287] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.007s
I20260709 12:23:59.509291  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:59.515064  4899 catalog_manager.cc:5697] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 3 to 4, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 4 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:59.473908  5940 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:59.474349  5946 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:59.518801  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.491583  7512 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.499750  7191 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:59.507690  7424 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:59.511418  7483 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:59.467410  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 60091us
I20260709 12:23:59.558488  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:59.516508  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.416512  7432 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:59.520898  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:23:59.490756  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.471119  5943 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:23:59.563369  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:59.535193  5218 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:59.533411  7506 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:59.534423  7248 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:59.516952  4899 catalog_manager.cc:5697] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 2 to 5, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 5 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:23:59.585915  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:59.585717  4899 catalog_manager.cc:5697] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 4 to 5, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 5 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.586800  5220 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:59.472393  4895 rpcz_store.cc:269] 0709 12:23:41.426451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.426931 (+   480us) service_pool.cc:224] Handling call
0709 12:23:59.283668 (+17856737us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:52.946321 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:53.075169 (+128848us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:53.075228 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:23:53.075239 (+    11us) tablet.cc:662] Decoding operations
0709 12:23:53.160748 (+ 85509us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:53.170685 (+  9937us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:53.172400 (+  1715us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:23:53.298401 (+126001us) tablet.cc:701] Row locks acquired
0709 12:23:53.298449 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:23:53.298640 (+   191us) write_op.cc:270] Start()
0709 12:23:53.298804 (+   164us) write_op.cc:276] Timestamp: P: 1783599833298571 usec, L: 0
0709 12:23:53.298826 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:23:53.299653 (+   827us) log.cc:844] Serialized 2459 byte log entry
0709 12:23:54.227879 (+928226us) op_driver.cc:464] REPLICATION: finished
0709 12:23:54.444366 (+216487us) write_op.cc:301] APPLY: starting
0709 12:23:54.444479 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:54.462771 (+ 18292us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:54.551882 (+ 89111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:54.560508 (+  8626us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:54.560785 (+   277us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:23:54.560827 (+    42us) write_op.cc:312] APPLY: finished
0709 12:23:54.561140 (+   313us) log.cc:844] Serialized 52 byte log entry
0709 12:23:54.561328 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:54.561450 (+   122us) write_op.cc:454] Released schema lock
0709 12:23:54.561511 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:54.561903 (+   392us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:54.561942 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1438491,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":205453,"apply.run_cpu_time_us":3366,"apply.run_wall_time_us":117544,"num_ops":6,"prepare.queue_time_us":90878,"prepare.run_cpu_time_us":15235,"prepare.run_wall_time_us":697514,"raft.queue_time_us":199865,"raft.run_cpu_time_us":7228,"raft.run_wall_time_us":271278,"replication_time_us":918638,"thread_start_us":617066,"threads_started":4,"wal-append.queue_time_us":668544,"wal-append.run_cpu_time_us":19571,"wal-append.run_wall_time_us":1769735}]]}
I20260709 12:23:59.541013  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:23:59.548259  5144 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:23:59.564821  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 928) took 1349 ms. Trace:
I20260709 12:23:59.564684  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:59.587764  4899 catalog_manager.cc:5697] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 4 to 5, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 5 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.566347  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:59.497332  5076 rpcz_store.cc:276] 0709 12:23:58.280955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.282697 (+  1742us) service_pool.cc:224] Handling call
0709 12:23:58.283120 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:58.283564 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:58.283605 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:58.750479 (+466874us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b2c001a9d48. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:23:58.751011 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:23:59.496466 (+745455us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:59.496517 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.496595 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:59.496644 (+    49us) raft_consensus.cc:1685] finished
0709 12:23:59.496656 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:59.496886 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":456,"raft.queue_time_us":3359,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":8503,"spinlock_wait_cycles":25130752,"thread_start_us":744999,"threads_started":1,"wal-append.queue_time_us":543011}
I20260709 12:23:59.569069  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:59.592051  4899 catalog_manager.cc:5697] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 3 to 4, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 4 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:23:59.593163  5069 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:23:59.593313  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:59.570021  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:59.573369  5221 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:23:59.594945  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:59.595008  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:23:59.595844  5221 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:59.573074  5947 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:59.596227  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:23:59.574580  5941 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:59.575857  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:23:59.597172  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:59.522842  7387 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:59.589304  5940 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:59.565809  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:59.598943  5068 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:59.600143  5221 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:59.590508  5148 rpcz_store.cc:276] 0709 12:23:58.217277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.764481 (+547204us) service_pool.cc:224] Handling call
0709 12:23:58.764784 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:58.781791 (+ 17007us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:23:58.782532 (+   741us) log.cc:844] Serialized 14 byte log entry
0709 12:23:59.128087 (+345555us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:59.136149 (+  8062us) log.cc:844] Serialized 26 byte log entry
0709 12:23:59.136328 (+   179us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:59.136369 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.136456 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:59.564509 (+428053us) raft_consensus.cc:1685] finished
0709 12:23:59.564541 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:59.564737 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6018,"prepare.queue_time_us":179,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":3473,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":9846,"thread_start_us":344827,"threads_started":1,"wal-append.queue_time_us":683075}
I20260709 12:23:59.603101  5189 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:23:59.603746  5115 leader_election.cc:400] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
I20260709 12:23:59.604076  5115 leader_election.cc:403] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:59.602502  4899 catalog_manager.cc:5697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 5, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:23:59.607779  5221 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:59.609865  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.610041  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:59.612052  4899 catalog_manager.cc:5697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 2 to 4, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 4 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.613826  4899 catalog_manager.cc:5697] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:23:59.615355  4899 catalog_manager.cc:5697] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:23:59.616572  4899 catalog_manager.cc:5697] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
W20260709 12:23:59.617771  5219 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.019s	sys 0.038s
W20260709 12:23:59.618844  5219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.019s	sys 0.038s
W20260709 12:23:59.573740  7241 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.043s	sys 0.090s
I20260709 12:23:59.624509  5188 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:59.625870  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:59.582065  7297 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.625000  5068 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:59.625293  5115 leader_election.cc:400] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
I20260709 12:23:59.628178  4899 catalog_manager.cc:5697] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 3 to 6, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 6 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:59.629426  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.633071  7444 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.633761  7332 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:59.571319  7428 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:59.653708  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:23:59.607091  7323 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:59.589589  7249 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.654798  4899 catalog_manager.cc:5697] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:23:59.584800  7329 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 12:23:59.516465  7505 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:59.663993  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 940) took 1548 ms. Trace:
I20260709 12:23:59.672883  5220 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:23:59.653978  6921 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:59.674154  7442 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.674744  7444 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:59.601881  7400 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:59.602025  7499 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:59.650550  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:59.537915  7512 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:59.617895  7248 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:59.678488  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:23:59.605106  7173 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:59.692863  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.623696  7050 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.727432  4899 catalog_manager.cc:5697] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 1 to 2, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.588446  7191 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.605607  7515 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:59.622159  7056 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.637832  7278 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
W20260709 12:23:59.685559  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 59780.5us
I20260709 12:23:59.780676  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:59.642093  7241 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.043s	sys 0.091s
I20260709 12:23:59.581874  7430 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:23:59.676772  7428 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.641237  7483 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:59.730644  5042 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.049s	sys 0.078s
W20260709 12:23:59.808142  5042 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.050s	sys 0.078s
W20260709 12:23:59.808586  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 664480us
I20260709 12:23:59.664846  7297 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:59.728600  7333 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:59.658053  7346 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.761520  7323 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:59.735915  7249 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:23:59.761297  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 82944.6us
I20260709 12:23:59.830384  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:23:59.753386  5075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.005s	sys 0.023s
W20260709 12:23:59.831511  5075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.005s	sys 0.023s
I20260709 12:23:59.738727  5218 rpcz_store.cc:276] 0709 12:23:58.115982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.116215 (+   233us) service_pool.cc:224] Handling call
0709 12:23:58.116601 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.575257 (+1458656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.663629 (+ 88372us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400483ad4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:23:59.663911 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3236928384}
I20260709 12:23:59.730325  7331 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.650825  7387 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:23:59.733685  5947 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:23:59.771560  4899 catalog_manager.cc:5697] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:23:59.843961  4899 catalog_manager.cc:5697] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:23:59.589063  7506 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:23:59.750586  7446 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.010s	sys 0.019s
W20260709 12:23:59.779342  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.775920  7515 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:59.802640  5900 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.008s	sys 0.030s
W20260709 12:23:59.856829  5900 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.008s	sys 0.031s
W20260709 12:23:59.787348  5939 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.019s	sys 0.053s
I20260709 12:23:59.758075  7248 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.779616  7050 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:59.786408  7056 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:59.752990  7442 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:59.745100  7400 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.835157  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:23:59.880676  5218 raft_consensus.cc:2393] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:23:59.807826  6649 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:59.839089  5115 leader_election.cc:400] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8.
I20260709 12:23:59.777108  7499 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:23:59.813997  7428 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:59.847090  5189 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:23:59.906076  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 59132.3us
I20260709 12:23:59.819305  7333 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:59.852731  5221 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:23:59.779378  7191 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:23:59.854319  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 356053us
I20260709 12:23:59.830389  7346 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:59.862151  7323 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:23:59.857385  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 717592us
I20260709 12:23:59.858037  5072 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:23:59.862514  5069 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:59.860901  7500 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:23:59.820195  7278 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:23:59.842001  7329 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:59.893488  7424 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:59.912691  7432 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:23:59.895035  5115 leader_election.cc:403] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:23:59.928791  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 89862us
W20260709 12:23:59.874188  7513 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
I20260709 12:23:59.909152  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:23:59.894989  6649 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:23:59.857785  5939 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.019s	sys 0.057s
I20260709 12:23:59.843439  7388 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:23:59.862337  7515 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:59.867123  7248 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:59.729175  6717 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:59.873171  7302 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.894963  7400 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:59.843302  7331 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:59.945362  6649 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:59.850661  7446 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.010s	sys 0.019s
W20260709 12:23:59.919289  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.931169  5073 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:59.910405  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:59.968968  5216 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.013s	sys 0.031s
I20260709 12:23:59.911456  7333 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:59.924096  5042 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:23:59.920408  7323 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:23:59.939400  7278 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:23:59.866201  7508 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:23:59.945044  7483 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:59.957154  7386 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:59.969434  5216 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.013s	sys 0.031s
W20260709 12:23:59.970026  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 59691.2us
I20260709 12:24:00.035240  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:23:59.972643  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 48645us
I20260709 12:23:59.908895  6921 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:59.908836  7499 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:23:59.977360  7512 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:23:59.977744  7432 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:59.981765  7444 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:23:59.983317  7050 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:59.960951  7302 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:23:59.984576  7506 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:00.037667  7505 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:23:59.990159  5900 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:00.081733  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 91672.8us
I20260709 12:23:59.994614  7442 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:23:59.998418  7517 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:23:59.942195  7410 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:23:59.941792  7173 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:23:59.951010  7329 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:00.006100  7346 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:00.007771  7191 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:00.014102  7520 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:23:59.927264  7500 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:23:59.928853  7387 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:00.038293  6921 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:00.048888  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 398435us
I20260709 12:24:00.123413  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:00.080559  7512 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:00.093152  7430 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:23:59.987190  7273 consensus_meta.cc:287] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.627s	user 0.005s	sys 0.005s
I20260709 12:24:00.088050  5900 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:00.038344  7386 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:00.035526  7508 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:00.076514  5042 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:00.096099  7241 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:24:00.080431  7388 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
W20260709 12:24:00.141127  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 53174.6us
I20260709 12:24:00.164915  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:00.165637  5216 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 3.
W20260709 12:24:00.147267  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 70898.1us
I20260709 12:24:00.141271  5073 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:00.153242  7241 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:00.153884  7508 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:00.157648  7273 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:00.158937  7302 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.129119  6921 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:00.192077  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:00.193041  5070 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:00.161732  7512 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:00.164098  7386 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.111459  7191 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:00.172550  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:00.172822  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:00.175040  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:00.210587  5216 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:00.177994  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 367) took 1369 ms. Trace:
I20260709 12:24:00.210981  5219 rpcz_store.cc:276] 0709 12:23:58.808678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.810165 (+  1487us) service_pool.cc:224] Handling call
0709 12:23:58.810504 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:58.810908 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:58.810938 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:59.089810 (+278872us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b2c000af528. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:23:59.090431 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 12:23:59.619381 (+528950us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:59.619430 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.619740 (+   310us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:00.146525 (+526785us) raft_consensus.cc:1685] finished
0709 12:24:00.146548 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:00.177922 (+ 31374us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9560,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":46170496,"thread_start_us":528464,"threads_started":1,"wal-append.queue_time_us":735069}
I20260709 12:24:00.122128  7249 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.183789  5215 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:00.108421  7329 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.199061  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:00.176290  7506 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:00.153055  7499 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.208527  7410 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:00.226866  6649 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:00.183848  7527 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:00.229863  5076 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:00.214917  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:00.233570  6649 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.235543  5072 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:00.220531  7484 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:00.227331  7521 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
W20260709 12:24:00.239322  7197 consensus_meta.cc:287] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.885s	user 0.007s	sys 0.010s
I20260709 12:24:00.286464  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 739) took 1020 ms. Trace:
I20260709 12:24:00.233180  7346 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.294458  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:00.295728  5148 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:00.296124  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:00.297377  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:00.299045  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:00.300359  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:00.301041  5147 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:24:00.245582  7331 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:00.314086  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:00.311484  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:00.226101  7515 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.317009  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:00.233466  7499 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:24:00.315032  5147 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:00.307946  7505 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:00.327404  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:00.327601  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:00.326426  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 291305us
I20260709 12:24:00.332037  5148 raft_consensus.cc:2410] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:24:00.336062  5075 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 676) took 2092 ms. Trace:
I20260709 12:24:00.332145  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:00.337087  5147 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:24:00.299430  7197 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:00.336537  5075 rpcz_store.cc:276] 0709 12:23:58.243396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.243546 (+   150us) service_pool.cc:224] Handling call
0709 12:23:58.243861 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:59.028589 (+784728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:59.028642 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:59.221783 (+193141us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b2c001a9d48. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:23:59.222414 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 12:23:59.832019 (+609605us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:59.832084 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.204989 (+372905us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b5400219b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:00.205072 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:00.335813 (+130741us) raft_consensus.cc:1685] finished
0709 12:24:00.335845 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:00.335991 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18186,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":9965,"spinlock_wait_cycles":788343040,"thread_start_us":1063501,"threads_started":2,"wal-append.queue_time_us":685655}
I20260709 12:24:00.340413  5222 raft_consensus.cc:2368] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6.
I20260709 12:24:00.306339  7442 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:00.316095  7483 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:00.394912  7483 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:00.396971  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 428) took 1220 ms. Trace:
I20260709 12:24:00.354192  7442 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:00.343887  7512 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:24:00.409375  7231 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.421037  7428 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.422197  7231 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:00.424197  7428 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:00.424547  7231 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:00.336428  7505 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:00.436546  7333 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.438131  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:00.439533  7333 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.446964  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:00.358165  7532 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:00.359901  7329 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.366528  7514 raft_consensus.cc:3055] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:00.452448  7506 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:00.492133  7514 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:00.492313  7506 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:00.492967  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:00.493683  5218 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:00.493475  7514 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:00.494968  7506 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.495385  7514 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:00.368003  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:00.370154  7516 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.471s	user 0.000s	sys 0.004s
I20260709 12:24:00.496855  5220 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:24:00.371416  5222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 732) took 1168 ms. Trace:
I20260709 12:24:00.498185  5222 rpcz_store.cc:276] 0709 12:23:59.203494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.204238 (+   744us) service_pool.cc:224] Handling call
0709 12:24:00.371126 (+1166888us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b540023b9d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:00.371325 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2258709248}
I20260709 12:24:00.500839  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 945) took 1016 ms. Trace:
I20260709 12:24:00.501606  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:00.502393  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:00.381065  7318 consensus_meta.cc:287] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.965s	user 0.009s	sys 0.002s
I20260709 12:24:00.330250  7323 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:00.507126  5145 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:00.387471  7515 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.392124  7297 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:00.392657  7500 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:00.238721  7527 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:00.393463  7539 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:00.350459  7050 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:00.527875  7050 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.530588  7050 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.353641  7197 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:00.307358  7346 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:00.287232  5223 rpcz_store.cc:276] 0709 12:23:59.266108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.390123 (+124015us) service_pool.cc:224] Handling call
0709 12:23:59.390592 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.390871 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:23:59.896093 (+505222us) log.cc:844] Serialized 14 byte log entry
0709 12:24:00.285658 (+389565us) log.cc:844] Serialized 14 byte log entry
0709 12:24:00.285958 (+   300us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:00.286021 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:00.286100 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.286201 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:00.286378 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12288,"prepare.queue_time_us":675114,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":111,"raft.queue_time_us":8512,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":9303,"spinlock_wait_cycles":44206720,"thread_start_us":782787,"threads_started":2}
I20260709 12:24:00.532266  5144 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:00.533466  7346 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:00.533501  7500 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:00.402998  5939 rpcz_store.cc:276] 0709 12:23:59.176840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.176976 (+   136us) service_pool.cc:224] Handling call
0709 12:23:59.178884 (+  1908us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:59.179663 (+   779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:59.179693 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:59.180278 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 12:23:59.261983 (+ 81705us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b18000571c0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db0d1 00007fd8a84f6a0f
0709 12:24:00.396364 (+1134381us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b18000571c0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a80e34bc 00007fd8a543148b 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db0d1
0709 12:24:00.396558 (+   194us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:00.396600 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.396685 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:00.396730 (+    45us) raft_consensus.cc:1685] finished
0709 12:24:00.396743 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:00.396905 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":331,"raft.queue_time_us":443,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":133,"spinlock_wait_cycles":53003392,"thread_start_us":679038,"threads_started":1,"wal-append.queue_time_us":757592}
W20260709 12:24:00.402271  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 219274us
I20260709 12:24:00.354876  7248 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.454459  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.456748  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.458475  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.542217  5943 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:24:00.460242  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.433154  7505 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:00.392221  7400 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.491334  7442 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:00.498198  7497 raft_consensus.cc:3055] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:00.534425  7500 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:00.378822  7273 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:00.501314  5215 rpcz_store.cc:276] 0709 12:23:59.484435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.484562 (+   127us) service_pool.cc:224] Handling call
0709 12:23:59.491916 (+  7354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:00.213451 (+721535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.500506 (+287055us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54008502d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:00.500765 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590490880}
I20260709 12:24:00.504946  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:00.513676  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 152453us
W20260709 12:24:00.513711  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 347746us
I20260709 12:24:00.517247  7446 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:00.520053  5190 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:00.526975  7056 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.550390  5899 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:00.533835  7527 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:00.502935  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:00.625690  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 122871us
I20260709 12:24:00.508723  7506 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:00.568449  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 148839us
I20260709 12:24:00.370193  6921 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:00.630318  5221 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:24:00.555330  7497 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:00.635080  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.605116  7442 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.637670  7547 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:00.633989  7508 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.630563  7430 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:24:00.637735  7056 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:00.637761  7538 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:00.663306  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:00.630707  7506 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:00.638816  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 119004us
I20260709 12:24:00.641422  6921 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:00.645781  7386 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:00.648347  7432 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:00.654933  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 47018us
I20260709 12:24:00.686815  5188 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:00.654950  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.657086  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.708129  5946 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:00.708256  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:00.708318  5115 leader_election.cc:400] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6.
I20260709 12:24:00.708998  5115 leader_election.cc:403] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:00.709369  5943 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:00.666031  5947 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:00.613991  7329 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
W20260709 12:24:00.633249  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 82928.9us
I20260709 12:24:00.682039  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:00.714041  5217 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:00.660686  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.656054  7547 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:00.721618  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 55627.4us
I20260709 12:24:00.726294  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:00.722602  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:00.627136  7446 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.754642  5190 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:00.757072  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:00.739277  6921 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:00.741951  7548 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:00.788517  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:00.790000  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:00.706957  7484 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:00.742522  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:00.795631  5942 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:00.743996  7497 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:00.729976  7547 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:00.748629  7248 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:00.669051  7538 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.808151  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:00.730175  7056 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:00.751029  7500 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:00.658088  7508 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:00.761466  5190 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:00.762930  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:00.838549  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 77189.8us
I20260709 12:24:00.839354  5190 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:00.839473  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:00.840725  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.672387  7506 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:00.767350  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:00.841663  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.841861  5947 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:00.842828  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.843454  5947 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:00.844337  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:00.774055  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.706802  7432 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.791407  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.849043  5946 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:00.850056  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:00.763808  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:00.792510  5941 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:00.851527  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.852401  5941 raft_consensus.cc:2410] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:00.793136  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.793870  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.721685  7387 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.794580  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.716358  7386 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:00.798569  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:00.803784  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:00.809643  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:00.868695  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:00.658640  7442 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:00.814771  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:00.821931  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:00.829224  5217 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:00.721621  7331 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 12:24:00.762794  7544 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:00.790818  7512 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:00.751240  7278 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.718s	user 0.014s	sys 0.000s
I20260709 12:24:00.859608  7273 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:00.875922  5071 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
W20260709 12:24:00.893429  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.907716  7543 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:00.908413  7442 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:00.909230  7273 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:00.910264  7273 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.847116  7410 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.863951  7248 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:00.943642  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.864776  7432 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:00.938817  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 150970us
I20260709 12:24:00.866528  7333 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:00.970165  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:00.866817  7323 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:00.810497  7484 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.881351  7318 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:00.981849  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:00.826225  7505 consensus_queue.cc:1175] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "90e5220a62ed42fe83c82d0302d18fac" responder_term: 6 status { last_received { term: 6 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 6 index: 3 } }
I20260709 12:24:00.917951  7278 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8.
W20260709 12:24:01.023161  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 155692us
I20260709 12:24:01.044111  7278 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.044775  7484 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.044646  7278 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.045065  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:01.036186  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 54444.6us
I20260709 12:24:01.046840  7278 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.047912  5041 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:01.049178  7560 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:01.043216  7500 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:01.049772  6649 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:01.042412  7318 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:00.961818  6921 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:00.886497  7387 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:01.006057  7400 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:01.061218  7552 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
I20260709 12:24:01.061508  5142 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:00.971690  7410 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:00.908466  7543 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.086185  7537 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:01.087522  6921 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:01.022200  7333 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.010357  7248 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:01.049437  7278 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 12:24:01.030435  7323 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.130650  7428 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:01.135993  7563 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:01.028014  7056 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.168332  7530 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:01.207365  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 403681us
I20260709 12:24:01.172811  7537 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.173585  7572 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:01.224020  7572 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.216498  7446 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.222424  7537 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:01.210419  7428 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.244606  7572 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:01.249493  7550 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:01.220551  7530 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:01.267567  7546 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
W20260709 12:24:01.268112  7558 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
W20260709 12:24:01.277230  7557 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.307s	user 0.006s	sys 0.000s
I20260709 12:24:01.215914  7563 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.278796  7538 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.280428  7563 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.280140  7543 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:01.281471  7539 raft_consensus.cc:3055] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:01.282958  7539 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:01.284497  7554 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.257s	user 0.000s	sys 0.003s
I20260709 12:24:01.284034  7539 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:01.285521  7539 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:01.287220  7530 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:01.272418  7550 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:01.290081  7550 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.291491  7547 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.294395  7548 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6.
I20260709 12:24:01.344730  5073 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:01.345844  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 300579us
I20260709 12:24:01.348368  5115 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:01.367164  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.368053  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:01.368971  5147 raft_consensus.cc:2376] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7. Re-sending same reply.
I20260709 12:24:01.369343  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.367214  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.370527  5144 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:01.367704  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:01.373013  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:01.380771  7278 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:01.381592  7278 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:01.382164  7278 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:01.382398  5222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 961) took 1019 ms. Trace:
I20260709 12:24:01.382988  5222 rpcz_store.cc:276] 0709 12:24:00.362505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.184095 (+821590us) service_pool.cc:224] Handling call
0709 12:24:01.184989 (+   894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.185250 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.185285 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:01.185349 (+    64us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:01.185406 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.185478 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.382020 (+196542us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:01.373589  5149 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:01.390559  5940 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:01.390691  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:01.395670  7318 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.398185  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 955) took 1138 ms. Trace:
I20260709 12:24:01.248855  7386 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.401996  7386 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.203428  7248 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:01.403048  7386 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:01.403630  7333 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.407330  7387 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.274492  7508 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.402051  7410 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.396118  6649 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:01.401052  7323 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:01.410723  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 192530us
I20260709 12:24:01.398820  5218 rpcz_store.cc:276] 0709 12:24:00.259505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.494496 (+234991us) service_pool.cc:224] Handling call
0709 12:24:00.494869 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:00.495315 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:00.495348 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:00.495927 (+   579us) log.cc:844] Serialized 26 byte log entry
0709 12:24:00.715936 (+220009us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:00.716005 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.716115 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:01.376001 (+659886us) raft_consensus.cc:1685] finished
0709 12:24:01.376048 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.397906 (+ 21858us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10819,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":5821,"thread_start_us":219513,"threads_started":1,"wal-append.queue_time_us":312740}
I20260709 12:24:01.421586  7302 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.434975  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.435398  7302 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:01.438573  7386 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.439898  7547 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:01.445729  7538 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.446336  7560 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.446718  7538 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:01.447211  7560 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.447926  7428 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:01.445179  5899 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:01.450955  7581 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:01.451812  7581 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:01.453822  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.454159  7386 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.455210  7577 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:01.454991  7386 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:01.455838  7577 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:01.456447  7577 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:01.458636  7386 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:01.459394  7563 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:01.460182  7428 consensus_queue.cc:1059] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:01.460826  7563 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:01.462286  7563 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:01.421576  7248 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:01.466231  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.467414  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 954) took 1208 ms. Trace:
I20260709 12:24:01.471472  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:01.473843  7508 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.474185  7323 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:01.439200  7572 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.474702  7508 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.477638  7572 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.477918  7508 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:01.475323  5142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 385) took 1180 ms (client timeout 1508 ms). Trace:
I20260709 12:24:01.478873  5143 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
W20260709 12:24:01.479045  5142 rpcz_store.cc:269] 0709 12:24:00.295211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.295882 (+   671us) service_pool.cc:224] Handling call
0709 12:24:01.461257 (+1165375us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b5400582854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:01.475068 (+ 13811us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1718114176}
I20260709 12:24:01.475271  7581 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:01.464407  7560 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.453956  7302 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:01.461609  7547 raft_consensus.cc:3055] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:01.485934  6649 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:01.487264  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.467962  5224 rpcz_store.cc:276] 0709 12:24:00.259043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.259192 (+   149us) service_pool.cc:224] Handling call
0709 12:24:00.259558 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:00.534162 (+274604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:00.534210 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:00.534255 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:00.534285 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.466773 (+932488us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b540067ff54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:01.466847 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.467172 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631006464}
I20260709 12:24:01.486486  7547 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:01.492234  7547 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.496315  7547 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:01.503166  7302 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:01.503904  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:01.504859  7508 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:01.504900  5069 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:01.506081  7508 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:01.507550  7508 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:01.515801  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.518826  7538 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:01.521122  7497 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:01.536264  7556 log.cc:927] Time spent T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: Append to log took a long time: real 0.513s	user 0.005s	sys 0.000s
W20260709 12:24:01.533663  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 474084us
I20260709 12:24:01.539148  6649 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.547114  5073 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:01.546309  5148 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:01.460362  7056 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.558868  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:01.557888  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:01.560081  5072 raft_consensus.cc:2393] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:01.560107  5069 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:01.565228  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:01.592098  7506 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.705s	user 0.001s	sys 0.017s
I20260709 12:24:01.594877  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.596891  7297 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.597668  7297 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.598153  5150 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:01.600193  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.601111  5147 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:24:01.603716  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.604761  5141 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:01.605213  7508 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:01.607566  5940 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:24:01.609966  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.618657  7410 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.619449  7410 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.622383  7410 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.623860  7297 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.627133  7550 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:01.649266  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.592954  7506 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.656792  7577 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:01.658066  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 51118.4us
I20260709 12:24:01.612365  7056 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.666936  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.665314  7428 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.668661  7428 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.670883  5939 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:01.672048  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:01.675220  7506 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.675246  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.676245  7506 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:01.681697  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.682104  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.683864  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.684644  5948 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:01.691919  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.676978  7506 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:01.677820  7428 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:01.696501  7577 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.696666  7197 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.694130  5220 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:01.684963  7056 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:01.705749  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:01.652906  6921 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.713119  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:01.713650  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.714221  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:01.723786  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 110325us
I20260709 12:24:01.710368  7197 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.733626  5941 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:01.735543  7197 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:01.732986  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.731792  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.738453  7534 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.739215  7534 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.742722  7318 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.743304  7329 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:01.743460  7318 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:01.744045  7329 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:01.703111  7278 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:01.750047  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:01.766124  5222 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:01.748101  7056 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:01.771605  5188 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:01.773572  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.765261  7508 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.775820  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.766744  7506 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:01.781577  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:01.788321  7527 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:01.788944  7527 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.789393  7527 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:01.789232  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 60690.3us
I20260709 12:24:01.771392  7056 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:01.779156  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:01.798435  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 47582.4us
I20260709 12:24:01.793968  7056 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:01.805176  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:01.801936  7197 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:01.824375  7329 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.807083  7318 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.829813  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:01.804865  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.821496  7563 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.799317  7508 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:01.835878  5221 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:01.838254  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.839295  5947 raft_consensus.cc:2393] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:01.841938  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:01.843140  5945 raft_consensus.cc:2368] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9.
I20260709 12:24:01.843003  6649 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.843963  6649 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:01.845149  7508 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:01.846654  5944 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:01.847316  6649 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:01.847551  5221 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:01.848186  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.849058  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.849330  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:01.849653  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.849968  7508 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:01.850170  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.850484  6649 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.859855  7508 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:01.860411  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.850390  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.861291  5220 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 3.
I20260709 12:24:01.839133  7278 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:01.851334  5224 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:01.851266  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.852025  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.855131  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.865000  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:01.852734  7539 consensus_meta.cc:287] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.567s	user 0.005s	sys 0.013s
I20260709 12:24:01.852890  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.853327  7506 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.872507  7506 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:01.875058  7506 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.853727  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:01.875909  5943 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:01.853929  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 391) took 1113 ms. Trace:
I20260709 12:24:01.854036  7537 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:01.854081  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:01.856383  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.857079  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.857592  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 925) took 1111 ms. Trace:
I20260709 12:24:01.859014  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.860296  6649 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:01.850957  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.862752  5042 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:01.867233  5940 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:01.820642  7197 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:01.889549  5073 rpcz_store.cc:276] 0709 12:24:00.722056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.722291 (+   235us) service_pool.cc:224] Handling call
0709 12:24:01.807129 (+1084838us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b540017fa54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:01.833806 (+ 26677us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1417911552}
I20260709 12:24:01.871389  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.838287  7563 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:01.896389  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 82618.7us
I20260709 12:24:01.866617  7508 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.900113  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.903129  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.904843  5221 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:01.905014  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:01.905867  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.907131  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.908907  6649 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:01.907055  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.894770  7197 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:01.910775  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.886771  5216 rpcz_store.cc:276] 0709 12:24:00.740750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.744077 (+  3327us) service_pool.cc:224] Handling call
0709 12:24:00.744458 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:00.793722 (+ 49264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:00.793792 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:00.798247 (+  4455us) log.cc:844] Serialized 26 byte log entry
0709 12:24:00.960730 (+162483us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:00.960787 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.800843 (+840056us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b5400553a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:01.803617 (+  2774us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:01.830304 (+ 26687us) raft_consensus.cc:1685] finished
0709 12:24:01.830326 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.853726 (+ 23400us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56866048,"thread_start_us":162053,"threads_started":1,"wal-append.queue_time_us":392724}
I20260709 12:24:01.943953  5940 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:01.946168  5941 raft_consensus.cc:2410] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:01.912915  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.951030  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.913130  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:01.956017  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.956856  5149 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
W20260709 12:24:01.957343  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 52457.9us
I20260709 12:24:01.957854  5150 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:01.958020  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.958536  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:01.959007  5145 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:01.959838  7428 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:01.960727  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:01.960565  7428 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:01.961771  7278 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:01.962811  7278 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:24:01.963661  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.965831  7534 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:01.965905  7428 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:01.966910  6921 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:01.967621  7318 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:01.958721  5144 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:01.966768  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:01.904886  7302 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:01.978479  5221 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:01.982165  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:01.990983  7569 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:01.991923  7569 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:01.992533  7569 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:01.971747  5041 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.65:37253 timed out after 1.601s (SENT) [suppressed 11 similar messages]
W20260709 12:24:01.954289  7432 consensus_meta.cc:287] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.771s	user 0.017s	sys 0.004s
I20260709 12:24:01.994442  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:01.974032  7547 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:02.007225  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.009312  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:02.022204  5943 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:01.997350  7297 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:02.034121  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 65363.3us
I20260709 12:24:02.051882  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:02.053596  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:02.037957  7500 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:02.033861  7297 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:02.056330  7597 log.cc:927] Time spent T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.193s	user 0.000s	sys 0.002s
W20260709 12:24:02.056598  7598 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 12:24:02.057018  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 197632us
I20260709 12:24:02.016098  7432 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:02.059623  5943 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:02.060227  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:02.030879  7547 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:02.072165  5146 raft_consensus.cc:2410] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:02.074462  7241 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:02.075139  5940 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:02.075397  7241 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:02.076794  5042 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:02.079344  7302 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:02.079806  7302 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:02.080251  7302 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:02.074561  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:02.082821  7537 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:02.083354  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 213845us
I20260709 12:24:02.087996  7444 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:02.091441  7527 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:02.092357  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:02.095280  7241 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:02.096477  7432 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:02.085779  5042 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:02.106851  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:02.119536  5221 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:02.121745  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:02.132401  7506 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:02.067929  7563 consensus_queue.cc:1059] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 12:24:02.145694  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 60036.2us
I20260709 12:24:02.146674  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:24:02.154044  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 52504.9us
W20260709 12:24:02.161491  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 273885us
I20260709 12:24:02.147125  7248 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:02.162863  7527 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:02.215818  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:02.219677  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:02.209038  7428 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:02.232923  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:02.240242  5947 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:02.251472  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:02.258544  5948 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
W20260709 12:24:02.260382  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.265659  7584 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.266724  7584 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:02.265936  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:02.265758  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 202756us
I20260709 12:24:02.270247  5221 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:02.271170  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:02.272696  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 130926us
I20260709 12:24:02.272205  7318 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:02.283178  7318 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:02.283782  7318 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:02.285756  7297 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:02.288435  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:02.289786  7241 raft_consensus.cc:3055] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:02.283231  7584 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:02.290424  7241 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:02.291607  7241 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:02.293359  7278 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:02.312963  7278 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.272372  7577 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.279901  7428 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:02.304131  7432 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:02.335108  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.338121  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 167960us
I20260709 12:24:02.319811  7278 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:02.343255  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 221746us
I20260709 12:24:02.293449  7241 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:02.343345  5189 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:02.298554  7197 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:02.365841  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 154798us
I20260709 12:24:02.370270  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:02.371014  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:02.372022  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:02.373284  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:02.373955  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:02.376233  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:02.376724  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:02.373204  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:02.376886  5145 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:02.378943  7601 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:24:02.382622  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:02.342490  7547 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:02.385070  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.365422  7581 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
W20260709 12:24:02.387007  7602 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:24:02.390748  7428 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:02.400522  5144 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:02.388902  6877 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:02.384929  7585 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:02.405421  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:02.406996  7241 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:02.341005  7537 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:02.331292  7577 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:02.412563  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:02.419668  5145 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:02.415469  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:02.425674  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:02.431422  5145 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:02.436985  5221 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
W20260709 12:24:02.439787  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44920.1us
I20260709 12:24:02.442341  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:02.445966  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.452776  7547 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:02.461980  6649 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:02.462531  6649 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:02.463006  6649 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:02.463634  7534 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:02.470374  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:02.471618  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:02.443205  7432 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.475765  5141 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:02.460052  7248 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:02.484975  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 102141us
I20260709 12:24:02.461483  7547 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:02.495301  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 76654.9us
I20260709 12:24:02.493939  7547 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:02.500778  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:02.501871  5145 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:02.502628  5115 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:02.503376  7386 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:02.504964  5141 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:02.506692  5148 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:02.454034  7537 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:02.511876  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 109597us
W20260709 12:24:02.512898  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 218506us
I20260709 12:24:02.518076  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:02.519138  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:02.522202  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:02.524370  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:02.524657  5189 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:02.525955  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:02.528738  5947 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:02.514951  7584 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:02.482160  7432 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:02.545517  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:02.546977  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:02.554468  5939 raft_consensus.cc:2368] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5.
I20260709 12:24:02.546633  5942 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:02.556216  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:02.556769  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:02.557926  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:02.563580  5948 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:02.570461  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:02.577445  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:02.579599  5216 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:24:02.574889  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 56804us
I20260709 12:24:02.582010  5189 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:02.577580  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:02.551841  7550 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:02.552994  7297 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:02.583086  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:02.588739  5218 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:02.590266  5943 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:02.542507  6877 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:24:02.572899  7197 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:02.601399  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 131485us
I20260709 12:24:02.606030  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:02.592785  7550 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.624071  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:02.596868  7248 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:02.632908  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 138815us
I20260709 12:24:02.601341  7534 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:02.639183  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 98923.4us
I20260709 12:24:02.642060  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:02.643059  5219 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:02.642278  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 59323.2us
I20260709 12:24:02.642069  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:02.631345  7550 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:02.637863  7302 consensus_meta.cc:287] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.557s	user 0.015s	sys 0.000s
I20260709 12:24:02.634680  7248 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:02.644762  7603 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.259s	user 0.000s	sys 0.002s
I20260709 12:24:02.639086  7534 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:02.645678  7241 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:02.669080  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:02.672451  7607 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:24:02.665062  7241 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:02.673142  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 134929us
I20260709 12:24:02.658668  7248 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:02.663013  7534 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:02.672665  7616 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:02.692698  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:02.656435  7302 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:02.674738  7614 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:02.685405  7241 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:02.714169  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:02.690615  7537 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:02.720197  5073 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:24:02.720986  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 54766.6us
I20260709 12:24:02.718343  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:02.723361  5071 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:02.705485  7614 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:02.726063  7617 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:02.727269  7605 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:24:02.728216  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:02.732142  5076 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:02.732385  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:02.732807  5899 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:02.743170  5070 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:02.714658  7432 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:02.745369  5115 leader_election.cc:400] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9.
I20260709 12:24:02.746174  5115 leader_election.cc:403] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:24:02.773217  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:02.773969  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:02.774072  7617 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:02.738144  7614 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:02.790131  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 51886.5us
I20260709 12:24:02.791374  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:02.791930  7609 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.151s	user 0.002s	sys 0.000s
I20260709 12:24:02.854696  6649 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:02.836151  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 113903us
I20260709 12:24:02.724159  7537 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.773433  7619 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:02.847244  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:02.847316  5072 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:24:02.861366  5219 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:02.860316  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:02.871379  6649 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:02.854960  7618 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:02.869472  7506 consensus_meta.cc:287] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.732s	user 0.014s	sys 0.009s
W20260709 12:24:02.875962  7318 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.588s	user 0.015s	sys 0.007s
I20260709 12:24:02.866420  7584 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.874030  7547 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:02.890201  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 237991us
I20260709 12:24:02.887849  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:02.898376  5070 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:02.904904  5076 raft_consensus.cc:2368] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5.
I20260709 12:24:02.907421  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:02.889742  7616 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:02.908147  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:02.866732  7537 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:02.915349  7624 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:02.915539  7537 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:02.916007  7624 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:02.916767  7537 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:02.883093  7506 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:02.888517  6877 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.919158  6877 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:02.920305  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.908411  5074 raft_consensus.cc:2410] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:02.923082  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:02.925423  5075 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:02.906457  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:02.928896  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.930222  5941 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:02.927663  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:02.952430  5070 raft_consensus.cc:2410] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:24:02.956631  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 45721.9us
W20260709 12:24:02.957049  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 109917us
I20260709 12:24:02.921445  6877 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:02.915620  7617 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:02.959438  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:02.957739  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:02.917017  7624 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:02.888447  7318 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:02.932119  7616 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:02.901121  7584 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:02.967031  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:02.969448  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:02.973663  7612 log.cc:927] Time spent T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.167s	user 0.004s	sys 0.000s
I20260709 12:24:02.956655  7302 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:02.978008  7318 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:02.979662  7056 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.114s	user 0.009s	sys 0.009s
I20260709 12:24:02.942911  7241 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:02.980189  7056 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:02.981354  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:02.981408  5215 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:02.956106  7539 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:02.983704  7056 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:02.985193  7628 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:02.977051  7534 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:02.991242  7534 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:03.026465  7611 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.382s	user 0.000s	sys 0.003s
I20260709 12:24:03.030652  6921 raft_consensus.cc:3055] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:24:03.036202  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 69111.5us
I20260709 12:24:02.967180  7616 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.045445  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:03.046612  5219 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
W20260709 12:24:03.045647  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 183357us
W20260709 12:24:03.039914  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 55476.8us
I20260709 12:24:03.050676  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.054234  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:03.056813  7500 consensus_meta.cc:287] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.986s	user 0.011s	sys 0.006s
I20260709 12:24:03.057286  7500 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:03.056370  5899 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:03.044534  6921 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:03.064601  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.049942  7616 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:03.066924  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:03.058116  7569 consensus_meta.cc:287] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.063s	user 0.019s	sys 0.000s
W20260709 12:24:03.062058  7278 consensus_meta.cc:287] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.663s	user 0.017s	sys 0.000s
I20260709 12:24:03.070602  7537 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:03.073012  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:03.077189  5222 raft_consensus.cc:1240] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:03.080674  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.080672  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.079911  5144 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:03.081900  5148 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:03.081846  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.084452  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:03.083158  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:03.135172  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:03.137106  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:03.137778  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 64974.7us
I20260709 12:24:03.063714  7539 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.085304  5146 raft_consensus.cc:2376] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8. Re-sending same reply.
I20260709 12:24:03.144143  5144 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:03.095598  7550 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.148874  7550 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:03.149816  7611 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:03.150987  7631 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:03.151700  7584 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:03.152391  7623 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.080230  7537 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.152973  7623 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:03.087812  7056 consensus_queue.cc:1059] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 12:24:03.152973  7537 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:03.155475  7623 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:03.078989  7569 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.156459  7637 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:03.157155  5223 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:03.158620  7569 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:03.061570  7248 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:03.161839  6921 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.163384  6921 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:03.163902  7248 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:03.166054  7633 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:24:03.167444  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.168002  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.168867  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.169269  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.175284  5069 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:03.142040  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:03.173933  5143 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:03.090104  7278 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.134709  7500 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:03.183337  7618 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9.
I20260709 12:24:03.183055  7626 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:03.195897  7626 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:03.196004  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.196467  7626 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:03.196899  5075 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:03.198329  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.199088  5074 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
W20260709 12:24:03.200482  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 58527us
I20260709 12:24:03.181232  5041 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:03.201905  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:03.202312  7527 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:03.202844  7527 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:03.203233  7527 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:03.205648  7641 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:03.206751  7641 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:03.207882  5150 raft_consensus.cc:2368] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7.
I20260709 12:24:03.157179  7637 raft_consensus.cc:697] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:03.180285  7631 raft_consensus.cc:697] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:03.205816  6877 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:03.194185  7500 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.180143  7539 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.221813  7428 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.252300  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.220315  7248 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:24:03.245229  7500 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:03.270663  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.271895  5074 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:03.274444  7637 consensus_queue.cc:237] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:03.273927  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:03.277825  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:03.278379  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:03.278993  5147 raft_consensus.cc:2376] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6. Re-sending same reply.
I20260709 12:24:03.279300  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.279886  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.280529  5142 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:03.280735  5147 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:03.281010  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:03.281770  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:03.282970  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.286769  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:03.251165  7278 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:03.289074  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:03.289742  5222 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:03.289899  7626 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:03.292665  7626 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:03.227272  7641 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:03.303198  5149 raft_consensus.cc:2410] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:03.248826  7534 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.301271  5115 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:03.316020  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:03.318526  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 227654us
I20260709 12:24:03.319514  5899 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:03.321484  7659 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:03.322221  7659 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:03.336344  5115 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:03.338464  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:03.339233  5943 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:03.340559  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:03.342973  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 139560us
I20260709 12:24:03.324671  7618 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:03.344206  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.344201  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.317562  7632 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:03.334786  7297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.008s	sys 0.004s
I20260709 12:24:03.349766  7631 consensus_queue.cc:237] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:03.350914  7539 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:03.358039  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:03.354035  5942 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:03.372503  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.373477  5943 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:03.267414  7428 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:03.370858  7622 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:03.346566  7547 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:03.323860  7534 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:03.376911  7622 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:03.379242  7534 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:03.379309  7622 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:03.375026  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:03.343271  7584 consensus_queue.cc:1059] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:24:03.381889  5947 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
W20260709 12:24:03.360272  7297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.008s	sys 0.004s
I20260709 12:24:03.387209  5074 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:03.348439  7618 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.384085  5943 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:03.343748  7550 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:03.381995  5115 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:03.348784  7660 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
W20260709 12:24:03.389727  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 45644.9us
I20260709 12:24:03.394364  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:03.341300  7614 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:03.418915  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.418399  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:03.370697  7632 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.395172  7659 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:03.437294  7537 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:03.440073  7537 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:03.394407  7584 raft_consensus.cc:3055] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:03.441812  7584 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:03.375562  7656 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:24:03.442816  7584 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:03.434924  7329 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:03.444182  7584 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:03.445191  7614 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:03.424168  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:03.448812  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:03.449569  5224 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:03.452816  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:03.453739  5068 raft_consensus.cc:2393] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 4: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:03.430415  5141 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:03.392232  7618 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:03.422739  7278 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:03.423574  7625 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:03.462848  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 54722.1us
I20260709 12:24:03.444214  7329 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.392802  7550 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.387006  7297 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:03.465512  7539 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:03.476064  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.460031  7318 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:03.474184  7329 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:03.468622  7278 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:03.477336  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 82483.3us
I20260709 12:24:03.479440  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:03.480737  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:03.494365  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:03.494971  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.496203  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.496996  5942 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:03.474499  7550 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:03.498473  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:03.494527  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.499159  5943 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:03.499800  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:03.500583  5074 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:03.496260  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.501811  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.502466  5069 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:03.485945  7278 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:03.498039  7539 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:03.501358  7625 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.497735  7652 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:03.514760  7652 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:03.533691  7428 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:03.533835  7652 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.510802  7539 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.536936  7563 raft_consensus.cc:3055] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:03.537395  7563 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:03.511296  7625 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:03.538447  7563 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.539518  7563 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:03.540486  7625 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:03.521158  7442 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.541952  7442 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:03.543138  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:03.544019  5945 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:03.544538  7442 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:03.546751  7632 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:03.564085  5944 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
W20260709 12:24:03.568276  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 88917us
I20260709 12:24:03.569752  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:03.572537  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:03.577348  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.578282  5115 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:03.578274  5898 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.508s (SENT)
I20260709 12:24:03.580710  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:03.477486  7527 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:03.505487  7616 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.592015  7616 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:03.594501  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:03.596436  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:03.497184  7297 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.598354  7386 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:03.598809  7386 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.599185  7386 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:03.597254  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:03.602675  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.512742  7577 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:03.603026  5069 raft_consensus.cc:2376] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8. Re-sending same reply.
I20260709 12:24:03.607808  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.609344  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.606626  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:03.576869  7656 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:03.619165  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:03.620347  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.586279  7442 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:03.623529  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.624413  5150 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:24:03.582696  7657 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:03.609239  7534 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:03.632280  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.632580  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.633247  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:03.632341  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.638571  5148 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:03.638862  5147 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:03.633468  5142 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:24:03.638931  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 66537.4us
I20260709 12:24:03.600658  7581 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:03.635236  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:03.635848  5042 leader_election.cc:336] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.503s (SENT)
I20260709 12:24:03.640439  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:03.643689  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:03.643755  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.646009  5069 raft_consensus.cc:2393] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:03.650184  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:03.650390  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.651275  7584 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:03.651424  5067 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:03.652359  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:03.652552  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.653797  7534 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:03.654497  7584 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:03.654881  7527 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:03.633208  7442 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.656049  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.654986  7534 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:03.660597  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.661970  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.663429  5143 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:24:03.657963  7442 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:03.666049  5076 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:03.667616  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.668998  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.669586  5071 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:03.670332  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.670558  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.671126  5142 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:03.671422  7569 raft_consensus.cc:3055] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:03.672664  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:03.674081  5114 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:03.677932  5150 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:03.677772  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:03.678442  5145 raft_consensus.cc:2376] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8. Re-sending same reply.
I20260709 12:24:03.679527  5114 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:03.680687  5942 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:03.671818  7569 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:03.682150  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.684100  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.687675  5067 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:03.683380  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:03.688144  7627 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.728s	user 0.000s	sys 0.006s
I20260709 12:24:03.691619  5145 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:24:03.677246  7581 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:03.692956  7581 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:03.695327  7581 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:03.681773  7616 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:03.696374  7241 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:03.699535  5114 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:03.682840  7641 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:03.642160  7500 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:03.703037  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 73598.3us
I20260709 12:24:03.684006  7539 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:03.658327  7550 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:03.706323  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:03.706625  5215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35624 (request call id 966) took 1859 ms (client timeout 1905 ms). Trace:
I20260709 12:24:03.706559  7329 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.707185  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:03.708554  5145 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:03.709478  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:03.710929  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 693) took 1034 ms. Trace:
I20260709 12:24:03.781522  5073 rpcz_store.cc:276] 0709 12:24:02.676593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.683139 (+  6546us) service_pool.cc:224] Handling call
0709 12:24:03.710702 (+1027563us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b540047ca54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:03.710858 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48295680}
I20260709 12:24:03.711486  5075 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:03.712409  7278 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:03.713876  6877 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:03.718226  7569 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.722530  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 660) took 1186 ms. Trace:
I20260709 12:24:03.729357  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 401) took 1254 ms. Trace:
I20260709 12:24:03.730818  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:03.743090  7655 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:03.744356  5189 leader_election.cc:400] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5.
I20260709 12:24:03.786355  5941 rpcz_store.cc:276] 0709 12:24:02.536017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.536156 (+   139us) service_pool.cc:224] Handling call
0709 12:24:02.544765 (+  8609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.932352 (+387587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.722221 (+789869us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b540023ffd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:03.722437 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":885915136}
W20260709 12:24:03.789114  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 68407.5us
I20260709 12:24:03.789453  5189 leader_election.cc:403] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:03.783864  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.744589  7618 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:03.744817  7248 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:24:03.745751  7638 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.592s	user 0.004s	sys 0.000s
I20260709 12:24:03.796845  5150 raft_consensus.cc:2368] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8.
I20260709 12:24:03.758426  5224 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:03.798493  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:03.761482  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:03.750658  7657 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:03.804724  5072 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:03.805018  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.765579  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:03.806581  5218 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:03.766856  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:03.807653  5216 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:03.807789  7500 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:03.767939  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:03.813876  5219 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: []
W20260709 12:24:03.777778  5215 rpcz_store.cc:269] 0709 12:24:01.847099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.849741 (+  2642us) service_pool.cc:224] Handling call
0709 12:24:03.706432 (+1856691us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400041ad4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:03.706544 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2579023744}
I20260709 12:24:03.814518  5942 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:03.765266  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:03.784621  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:03.787900  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 57201.3us
I20260709 12:24:03.790601  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:03.776073  5042 connection.cc:256] RPC call timeout handler was delayed by -1.7253025140003047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 12:24:03.816167  5042 connection.cc:256] RPC call timeout handler was delayed by -1.6893025140002464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
W20260709 12:24:03.816541  5042 connection.cc:256] RPC call timeout handler was delayed by -1.6695025140002144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
I20260709 12:24:03.787284  5223 rpcz_store.cc:276] 0709 12:24:02.475108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.475321 (+   213us) service_pool.cc:224] Handling call
0709 12:24:02.492789 (+ 17468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.157675 (+664886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.729086 (+571411us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54003fe8d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:03.729279 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1527390592}
W20260709 12:24:03.816847  5042 connection.cc:256] RPC call timeout handler was delayed by -1.6110025140001198s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2026s for a response
W20260709 12:24:03.795017  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 50876.1us
I20260709 12:24:03.790393  7614 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:03.796030  7197 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:03.796667  7585 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5.
I20260709 12:24:03.779556  5143 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
W20260709 12:24:03.791846  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 92393.9us
I20260709 12:24:03.792747  5946 raft_consensus.cc:2368] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:03.779229  7625 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:03.822254  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 112870us
I20260709 12:24:03.821398  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:03.826818  5190 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:03.828318  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:03.789039  7655 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:03.829388  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:03.830195  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:03.830673  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:03.831894  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:03.824556  5114 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:03.838593  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 659) took 1303 ms. Trace:
I20260709 12:24:03.841265  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.843349  5042 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:03.813300  7569 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:03.852815  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:03.861023  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:03.865686  5223 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:03.865717  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.866128  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:03.870821  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:03.871505  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:03.848398  7641 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:03.831015  7297 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:03.882722  7329 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:03.878103  7241 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:03.883960  7241 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:03.884034  7618 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:03.884477  7241 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:03.882767  7499 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:03.887795  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.888698  5942 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:03.870287  7444 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:24:03.890425  7444 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:03.890796  7444 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:03.891664  7278 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:03.891758  7444 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:03.892624  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 68158.5us
I20260709 12:24:03.894574  7641 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:03.895841  7641 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:03.898204  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:03.899196  5939 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:03.870359  7537 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:03.839236  5940 rpcz_store.cc:276] 0709 12:24:02.535557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.541819 (+  6262us) service_pool.cc:224] Handling call
0709 12:24:02.562056 (+ 20237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:02.646632 (+ 84576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:02.646670 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:02.659110 (+ 12440us) log.cc:844] Serialized 26 byte log entry
0709 12:24:03.052781 (+393671us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:03.052824 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.052903 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:03.744986 (+692083us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:03.825614 (+ 80628us) raft_consensus.cc:1685] finished
0709 12:24:03.825642 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.838515 (+ 12873us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5989,"raft.queue_time_us":25645,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":19063,"spinlock_wait_cycles":35611520,"thread_start_us":425073,"threads_started":2,"wal-append.queue_time_us":457865}
I20260709 12:24:03.877272  7637 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:03.900847  7637 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:03.903928  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:03.904819  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.893314  7444 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:03.906948  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:03.907094  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:03.907645  5216 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:03.907954  5072 raft_consensus.cc:2393] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:03.911294  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:03.912184  5220 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:24:03.910698  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:03.914727  5222 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:03.915544  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:03.876010  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:03.877692  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 50957.1us
I20260709 12:24:03.879837  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:03.856859  7584 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:24:03.879859  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 54691us
I20260709 12:24:03.917692  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:03.868759  7626 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:03.864928  7550 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:03.901321  7637 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:03.924647  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 121240us
I20260709 12:24:03.928052  5942 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:03.928653  5223 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:03.936003  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:03.945102  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:03.946635  5218 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:03.942474  7630 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:03.957942  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:03.947510  7318 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:03.958189  7630 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:03.958818  5223 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:03.959096  7318 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:03.960316  7318 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:03.969820  5900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.007s	sys 0.013s
W20260709 12:24:03.970314  5900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.007s	sys 0.013s
W20260709 12:24:03.970606  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 656302us
I20260709 12:24:03.971421  5900 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:03.973899  7617 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:03.974761  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:03.975492  7619 consensus_queue.cc:1059] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:03.977234  7619 raft_consensus.cc:3055] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:03.977787  7619 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:03.975531  5223 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:03.935617  7534 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:03.989931  7248 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:03.990486  5072 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:03.941663  7550 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:03.992417  5188 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:03.997054  7656 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:03.997632  7656 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:03.998153  7656 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:04.001474  7534 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:04.002279  7534 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:03.990944  7619 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:04.013649  7619 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:04.040861  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 147375us
W20260709 12:24:04.041267  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 69959.1us
W20260709 12:24:04.040861  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 48547.6us
I20260709 12:24:04.042171  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:04.042672  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 68348.1us
W20260709 12:24:04.042886  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.045801  5939 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:04.048179  5115 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:04.048704  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:04.050359  5900 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:24:04.051313  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 95888.4us
I20260709 12:24:04.056481  5188 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:04.063704  5114 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:04.064988  5188 leader_election.cc:400] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5.
I20260709 12:24:04.065999  5188 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:04.070084  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 144863us
I20260709 12:24:04.070902  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:04.071883  7657 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.075986  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:04.075493  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:04.123246  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 206276us
I20260709 12:24:04.059955  7652 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:04.131109  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:24:04.132411  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.132524  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 82256.3us
I20260709 12:24:04.074805  7604 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:04.115999  7442 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:04.138023  7442 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:04.143724  7655 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:04.145602  7655 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:04.148152  7241 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:04.151330  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 75958.4us
I20260709 12:24:04.154913  5042 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:04.155982  5220 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:04.047715  7499 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.060117  7660 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:04.126974  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:04.068207  7248 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:24:04.175588  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50538.3us
I20260709 12:24:04.176695  7634 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:04.177446  7634 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:04.178633  7634 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:04.116356  7626 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:04.172245  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:04.121315  7616 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:04.063861  7584 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:04.181293  5188 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:04.181905  5946 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:04.186254  5042 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:04.180076  7386 consensus_meta.cc:287] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.580s	user 0.014s	sys 0.008s
I20260709 12:24:04.168953  7539 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:04.234176  7386 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:04.234340  7539 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:04.236552  5076 raft_consensus.cc:2368] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8.
W20260709 12:24:04.238699  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 57487.4us
W20260709 12:24:04.240723  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 109705us
W20260709 12:24:04.244987  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 72864.1us
I20260709 12:24:04.252416  5218 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:04.174224  7577 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:04.171705  7537 raft_consensus.cc:2764] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:04.220407  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:04.220836  5222 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:04.228085  7630 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:04.229166  7635 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:04.232286  7585 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:04.311787  7506 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:04.311617  7652 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:04.312393  7506 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:04.312834  7506 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:04.213961  7616 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:04.214780  7614 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:04.208629  7626 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:04.315696  5190 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:04.180316  7442 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.218765  7248 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:04.315238  5900 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:04.317134  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.174435  7631 consensus_queue.cc:1059] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.122s
I20260709 12:24:04.209510  7527 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 12:24:04.334826  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:04.356673  5222 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:04.337354  5223 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:04.295315  7613 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:04.336661  5115 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:04.360148  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:04.360594  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:04.256484  7577 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:04.181859  7499 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:04.376628  5216 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:04.328969  7604 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:04.382686  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:04.335609  7626 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:04.389745  5115 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:04.386269  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:04.347899  7616 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:04.328429  7585 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:24:04.402908  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:04.403985  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:04.409041  5141 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:24:04.363962  7613 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:04.410099  5041 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:04.317134  7197 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 12:24:04.339067  7442 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:04.387964  7563 raft_consensus.cc:2764] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:24:04.421468  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:04.378515  7297 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:04.416585  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:04.377874  7635 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:04.421552  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:04.422647  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:04.420811  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:04.428812  5149 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:04.429075  5147 raft_consensus.cc:2376] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9. Re-sending same reply.
I20260709 12:24:04.423347  5189 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:04.419600  5143 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:04.443634  5148 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:04.415675  7432 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:04.415933  7613 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:04.458837  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 238488us
I20260709 12:24:04.459988  5042 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:04.423034  7641 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:04.426342  7297 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:04.493892  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 250883us
I20260709 12:24:04.382835  5223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:04.495945  5223 raft_consensus.cc:606] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:24:04.496513  5223 raft_consensus.cc:614] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:04.474110  7563 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:04.474519  7623 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:04.443269  7635 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:04.487802  7297 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:04.492916  7584 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:04.474749  7641 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.497579  7681 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:04.491143  7539 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:04.554628  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 221821us
I20260709 12:24:04.559490  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:04.501884  7432 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:04.498822  7318 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:04.535571  7637 consensus_meta.cc:287] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.010s
I20260709 12:24:04.577564  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:04.579413  5946 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:04.584213  5946 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:24:04.472899  7248 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:04.561326  7641 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:04.582683  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 122812us
I20260709 12:24:04.594100  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:04.558364  7681 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:04.549748  7624 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:04.565783  7623 raft_consensus.cc:697] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:04.555270  7614 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:24:04.623647  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:04.624190  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:04.583459  7689 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:04.647617  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:04.648557  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 89171.4us
I20260709 12:24:04.590710  7318 raft_consensus.cc:697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:04.655987  5190 leader_election.cc:400] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8.
W20260709 12:24:04.718842  5898 leader_election.cc:336] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.760s (SENT)
I20260709 12:24:04.605348  7539 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.722218  5042 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:04.717374  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:04.734001  5941 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:04.600203  7681 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:04.735374  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:04.738044  5188 leader_election.cc:400] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8.
I20260709 12:24:04.730785  5944 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:04.500375  7604 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:04.605881  7432 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:04.731856  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 72872.7us
I20260709 12:24:04.732189  5190 leader_election.cc:403] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 12:24:04.732698  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 309408us
I20260709 12:24:04.593966  7527 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:24:04.740921  5076 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:04.584841  7637 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:04.745978  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 92631us
I20260709 12:24:04.750419  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:04.751171  5220 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:04.751513  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:04.754685  5189 leader_election.cc:400] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:04.755080  5189 leader_election.cc:403] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:24:04.649900  7577 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:04.766124  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:04.767429  5223 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:24:04.769815  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 51042.1us
I20260709 12:24:04.773169  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:04.774387  5216 raft_consensus.cc:2410] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:04.646405  7500 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.111s
I20260709 12:24:04.779548  5946 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:04.736640  7698 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:04.740736  6921 raft_consensus.cc:3055] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:04.745476  7432 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:04.802371  7318 consensus_queue.cc:237] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:04.806181  7619 consensus_meta.cc:287] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.792s	user 0.008s	sys 0.006s
I20260709 12:24:04.796749  7697 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:04.807016  7637 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:04.779002  7577 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:04.808226  7577 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:04.683660  7624 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:04.761343  7689 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:04.829332  7623 consensus_queue.cc:237] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:04.847318  7691 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:04.848060  7329 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:04.848658  7329 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:04.810087  7432 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:04.810513  7563 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.790934  6921 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:04.790586  7499 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:04.760696  7539 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:04.865224  5900 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:04.755455  7584 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:04.860744  7655 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.715s	user 0.012s	sys 0.002s
I20260709 12:24:04.867800  7698 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:04.868592  7689 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:04.870869  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:04.869791  7617 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.895s	user 0.018s	sys 0.000s
I20260709 12:24:04.877763  7444 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:04.896409  7684 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
W20260709 12:24:04.896700  7678 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.317s	user 0.000s	sys 0.001s
I20260709 12:24:04.870434  7707 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:04.892411  7683 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
W20260709 12:24:04.887682  7534 consensus_meta.cc:287] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.885s	user 0.012s	sys 0.007s
I20260709 12:24:04.882912  7624 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:04.867663  7506 consensus_meta.cc:287] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.002s
W20260709 12:24:04.914270  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 43522us
I20260709 12:24:04.896970  7616 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:04.929387  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 176311us
I20260709 12:24:04.882510  7386 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:04.878232  7635 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:04.872575  7241 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:04.910465  7657 consensus_meta.cc:287] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.838s	user 0.014s	sys 0.000s
W20260709 12:24:04.902040  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 459726us
I20260709 12:24:04.883194  7539 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:04.917944  7550 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:04.916898  7631 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
W20260709 12:24:04.961755  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 103404us
I20260709 12:24:04.965591  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:04.994004  5946 raft_consensus.cc:2410] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:04.979413  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:04.980594  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:04.919970  7444 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:04.922931  7617 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:04.883772  7563 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:04.917028  7711 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8.
W20260709 12:24:04.986676  5898 leader_election.cc:336] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 2.097s (SENT)
I20260709 12:24:05.004386  5900 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:04.958981  7577 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:04.992632  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:04.999109  5144 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:04.999650  5141 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:04.873225  6921 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:04.991137  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.006563  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:05.004426  5898 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:04.998257  7444 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:05.003799  7539 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:05.013599  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:05.014520  7329 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:05.014657  5075 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:05.038901  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:05.047722  5939 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:04.989678  7657 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:04.959826  7534 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:04.961090  7506 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:05.051070  7617 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:05.027189  7616 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:05.020066  7563 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:05.054762  7656 consensus_meta.cc:287] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.056s	user 0.003s	sys 0.014s
I20260709 12:24:05.021780  7666 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:04.999403  7550 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.058156  7569 raft_consensus.cc:3055] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:05.090819  7550 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:05.095378  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 113997us
I20260709 12:24:05.096851  7534 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:05.063217  7241 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 12:24:05.098457  5899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.007s	sys 0.046s
I20260709 12:24:05.058218  7577 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:05.064018  7329 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:05.106600  5899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.008s	sys 0.046s
I20260709 12:24:05.020779  7660 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:24:05.097740  7569 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:05.119438  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 367969us
I20260709 12:24:05.066079  7297 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:05.074961  7655 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:05.017526  7432 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:05.091918  7634 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:05.137902  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.098291  7623 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.141887  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:05.155299  5075 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:05.156900  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.090894  7506 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:05.150031  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:05.058969  7539 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:05.111114  7714 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:05.158205  5074 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:05.165441  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:05.166355  5069 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:05.159991  5941 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:24:05.119114  7550 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.158919  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:05.168989  5072 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:05.168829  7550 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:05.172384  7550 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:05.091426  7624 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 12:24:05.174823  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 176962us
I20260709 12:24:05.177198  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.177989  5070 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:05.180501  5114 leader_election.cc:336] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.567s (SENT)
I20260709 12:24:05.184742  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:05.166002  7626 consensus_meta.cc:287] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.758s	user 0.006s	sys 0.009s
I20260709 12:24:05.151700  7634 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:05.097990  7656 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.174185  7714 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.194113  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:05.139771  7500 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:24:05.194072  7690 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 12:24:05.130306  7329 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.198746  7569 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:05.206636  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 459722us
I20260709 12:24:05.234086  5189 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:05.206888  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 109723us
I20260709 12:24:05.212450  5074 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
W20260709 12:24:05.235554  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.217732  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 222810us
I20260709 12:24:05.225404  5145 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:05.225512  5143 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:05.228437  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.228749  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.238458  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:05.188741  7506 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:24:05.298159  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 64181.7us
I20260709 12:24:05.327615  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.333180  5074 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:05.151827  7700 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.284s
I20260709 12:24:05.341079  5898 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:05.334115  5147 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:24:05.217634  7714 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:05.291374  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:05.327637  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:05.292620  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:05.328562  5943 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:05.165661  7623 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.234108  7550 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:05.297084  5216 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:05.301484  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:05.338907  5217 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:05.371415  5070 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:05.341181  5189 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:05.342690  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:05.373207  5221 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:05.289283  7718 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.343744  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.344446  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:05.346417  7726 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:05.347747  7634 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.348337  5150 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:05.348773  7432 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:05.351984  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:05.352105  7442 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:05.339027  7689 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:05.355551  7527 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:05.356627  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:05.290100  7723 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:05.297367  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:05.327054  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 103870us
I20260709 12:24:05.385692  5144 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:05.363160  7569 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:05.366472  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.364718  7506 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:05.377880  7613 consensus_meta.cc:287] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.899s	user 0.015s	sys 0.000s
I20260709 12:24:05.367540  5942 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:05.342988  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:05.377686  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:05.379869  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:05.214668  7626 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:05.380987  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:05.357319  5074 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:05.333367  7637 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:05.379859  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:05.296840  5223 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:05.394129  5147 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:24:05.398435  7506 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.400341  7506 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:05.376760  7500 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:05.401708  7719 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:05.401911  7500 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:05.383150  7718 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:05.388878  7689 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.405515  5075 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:05.390792  7634 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:05.407652  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 55830.4us
I20260709 12:24:05.406673  7689 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:05.429560  5221 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:05.429955  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:05.407176  7432 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:05.432476  7706 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.433322  7735 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.433185  7706 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.433907  7735 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:05.435812  7706 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:05.436240  7735 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:05.391940  7723 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.440074  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:05.441167  7666 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.444434  7666 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.441752  7652 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:05.442314  7734 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:05.447758  7734 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:05.443260  7687 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:05.443354  7617 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:05.451272  7617 raft_consensus.cc:3055] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:05.451681  7617 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:05.452600  7617 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.454682  5219 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:05.448887  7734 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.440296  7723 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:05.456884  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:05.458304  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:05.471975  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.472060  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:05.430914  5145 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:05.473090  5074 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:05.483857  5946 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:05.472756  5223 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:05.485647  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:05.490147  5943 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:05.493224  5941 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:05.427997  7534 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:05.393050  7577 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 12:24:05.534947  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:05.536207  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:05.455019  7666 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:05.496903  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.430738  7584 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:05.389225  7527 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.499789  5898 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:05.500917  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:05.504801  5042 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:05.554194  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:05.555603  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:05.556715  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:05.557106  7715 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.414s	user 0.002s	sys 0.000s
I20260709 12:24:05.505131  5223 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:05.496893  5114 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:05.473417  7652 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:24:05.510061  7709 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:24:05.512830  5068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 951) took 1708 ms (client timeout 1684 ms). Trace:
I20260709 12:24:05.513656  7694 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:05.514581  7703 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:24:05.516685  7725 log.cc:927] Time spent T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:24:05.518332  7698 raft_consensus.cc:3055] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:05.518194  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.520840  7718 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:05.523365  7624 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:05.473975  7687 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:24:05.524138  7660 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.527549  5224 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:05.544261  7297 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:05.500368  7719 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:05.529160  7710 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:05.530005  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:05.530650  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:05.531104  7563 raft_consensus.cc:3055] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:24:05.531955  7733 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:24:05.489578  7655 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.580925  7666 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:05.540680  5070 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:05.621476  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:05.546923  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.622396  5222 raft_consensus.cc:2393] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:05.548317  5942 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:05.542138  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.551409  7712 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:05.627983  7712 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:05.630879  7728 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.481s	user 0.002s	sys 0.001s
I20260709 12:24:05.631072  7624 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:05.631165  7329 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:05.551841  7442 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:05.632444  7442 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.593219  7698 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:05.632862  7442 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:05.635231  7197 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.635816  7197 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.612602  7719 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.636849  7719 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:05.557432  5948 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:05.620488  7710 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.585378  7694 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.504702  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.642027  5069 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:05.642940  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.557835  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:05.559170  7729 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:24:05.560173  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:05.562587  5223 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:05.572106  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:05.575711  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.576360  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 79534.9us
I20260709 12:24:05.654870  5070 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:05.654244  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.576808  7635 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:05.659256  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 87311.6us
W20260709 12:24:05.577052  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 72343.5us
W20260709 12:24:05.578351  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 78654us
I20260709 12:24:05.397094  7613 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:05.568480  5068 rpcz_store.cc:269] 0709 12:24:03.804220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.822354 (+ 18134us) service_pool.cc:224] Handling call
0709 12:24:05.391011 (+1568657us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b5400791420. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a84b2dc0 00007fd8a853ee18 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:05.512432 (+121421us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b540037f354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:05.512661 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1291609856}
I20260709 12:24:05.537796  7723 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:05.673861  7723 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:05.674463  7723 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:05.585659  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:05.679162  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:05.680014  5941 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 4.
I20260709 12:24:05.545361  7617 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:05.677685  5222 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:05.684484  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.588132  5215 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:05.689901  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.690608  5215 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:05.691758  5215 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:05.465121  7241 consensus_queue.cc:1059] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.115s
I20260709 12:24:05.694351  7241 raft_consensus.cc:3055] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:05.694761  7241 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:05.695700  7241 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.697032  7241 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.482853  7619 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:05.697875  7619 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:05.681104  7527 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:05.698853  7619 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.700300  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:05.605628  7721 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:05.609846  7623 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:05.543540  7534 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.611848  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.580600  7499 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:05.533658  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:05.611013  7656 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:05.623432  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:05.595543  7577 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:05.635076  5217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:05.647161  5944 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:05.656095  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.731851  5217 raft_consensus.cc:606] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:24:05.732419  5217 raft_consensus.cc:614] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:05.654245  7703 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:05.667223  7329 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:05.667299  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:05.668063  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.669564  7613 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:05.680042  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:05.477839  7444 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.749024  7444 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:05.754761  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.687767  7710 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:05.687912  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:05.584336  7709 consensus_queue.cc:1243] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer ceec344dd00f40c18ef58414ce147016 log is divergent from this leader: its last log entry 7.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:05.833010  5218 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:05.617887  7563 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:05.643271  7681 consensus_meta.cc:287] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.887s	user 0.014s	sys 0.000s
I20260709 12:24:05.702451  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:05.681257  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:05.714663  5940 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:05.715049  5075 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:05.617668  7660 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.662559  5898 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:05.746332  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:05.746515  7197 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:05.477041  7689 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 12:24:05.790400  5216 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:24:05.773000  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:05.752503  7613 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:05.756604  7585 raft_consensus.cc:3055] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:05.686568  7694 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:05.798413  7712 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:05.728266  7499 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:05.728027  7577 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:05.716779  7721 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:05.786289  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.902493  5070 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 6: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:05.854858  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:05.625912  7655 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:05.720395  7735 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 12:24:05.909962  7386 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:05.909670  7563 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.910385  7386 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:05.911396  7563 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:05.766013  7703 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:05.631659  7624 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:05.797395  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:05.763120  7500 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:05.721406  7550 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:05.812197  5224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:05.945981  5224 raft_consensus.cc:606] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:05.831241  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:05.834435  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 146609us
I20260709 12:24:05.947721  5042 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:05.841393  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 139011us
I20260709 12:24:05.809417  6921 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:05.845892  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 183426us
I20260709 12:24:05.951684  5217 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:05.952476  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:05.846475  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 73592.6us
I20260709 12:24:05.953514  5190 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:05.715912  7698 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:05.806607  7539 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:05.836619  7635 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:05.840955  7248 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.009s	sys 0.002s
I20260709 12:24:05.904884  5946 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:05.851862  7585 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:05.897696  7499 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:05.777328  7714 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.042337  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:05.847678  7329 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:05.916464  7735 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:05.809345  7708 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 12:24:05.847221  7637 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:05.936730  5068 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:05.959961  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:05.852713  7444 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:06.023401  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.066311  5217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
W20260709 12:24:06.067018  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.067164  5217 raft_consensus.cc:606] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:24:06.067667  5217 raft_consensus.cc:614] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:05.847661  7690 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 12:24:06.052866  7329 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:05.961789  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:06.025502  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:06.024904  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 55578.6us
W20260709 12:24:06.077916  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.919530  7666 raft_consensus.cc:3055] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:06.078949  7666 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:06.019651  7248 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.210s	user 0.009s	sys 0.007s
I20260709 12:24:06.044662  5146 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:05.965829  7506 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 12:24:05.845420  7681 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.045303  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:05.952816  7534 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:05.960908  7500 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:06.094244  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 49529.3us
I20260709 12:24:06.106503  5948 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:06.051005  5067 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:05.853019  7707 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.176s
I20260709 12:24:05.915805  7442 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.056002  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:06.114257  5142 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:06.021173  7318 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.021625  7729 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:05.903579  7712 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.119524  5067 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:06.119840  7712 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.121069  5067 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:06.124399  7712 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:06.038342  7197 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.126884  7197 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.119524  5943 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:06.127923  7748 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:24:06.077798  7693 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:06.081111  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 121438us
I20260709 12:24:06.130329  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:06.132076  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:06.045280  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:06.133024  7729 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:06.133265  5144 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:06.134334  7500 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:06.078967  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:05.917434  7703 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:06.051894  7691 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.052731  7655 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:06.090629  7746 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:06.095062  7702 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:06.064636  7735 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:06.099371  7634 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.693s	user 0.012s	sys 0.000s
I20260709 12:24:06.159587  5150 raft_consensus.cc:2368] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8.
I20260709 12:24:06.056936  7721 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:06.086094  7641 consensus_meta.cc:287] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 1.471s	user 0.018s	sys 0.000s
I20260709 12:24:06.075941  7626 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:06.105953  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:06.082518  7329 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:06.153404  7749 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:24:06.081705  5069 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:06.041826  7698 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:06.139125  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:06.028748  7577 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:06.095149  7666 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:06.154471  5068 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:06.199164  5070 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:06.199983  5068 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:06.157104  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:06.168691  5220 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:06.202291  5068 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:06.208781  5068 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:24:06.209380  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 52375.5us
W20260709 12:24:06.173089  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 67357.2us
I20260709 12:24:06.176729  5217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c86395378750473e80c76902ad29fbe7"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:06.207263  7655 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:06.182484  5223 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.217612  5223 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:06.136400  7691 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.149724  7746 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:06.152061  7702 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:06.189301  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50340.8us
I20260709 12:24:06.139273  7585 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.183774  7630 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:06.125312  5076 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:06.211400  7660 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:06.180500  7442 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:06.144104  7318 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:06.226312  7619 consensus_queue.cc:1059] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:06.227744  7637 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:06.229868  7624 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:06.171314  7641 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:06.235075  7735 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:06.203430  7539 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:06.197844  7721 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:06.282262  5114 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:06.276329  7577 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:06.211879  5217 raft_consensus.cc:606] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:24:06.350939  5217 raft_consensus.cc:614] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:06.264633  7703 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:06.344421  5075 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 724) took 1189 ms (client timeout 1545 ms). Trace:
I20260709 12:24:06.263644  7655 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:06.287132  7702 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:06.391510  7241 consensus_meta.cc:287] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.694s	user 0.013s	sys 0.005s
I20260709 12:24:06.276880  7746 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:06.288848  7714 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:06.347150  7585 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:06.359805  5223 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 12:24:06.364440  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:06.409896  5223 pending_rounds.cc:85] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 2
I20260709 12:24:06.364974  7537 raft_consensus.cc:3055] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:24:06.365476  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.366003  7641 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:06.350646  7539 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:06.369983  5072 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:06.370553  7717 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:06.341873  7660 raft_consensus.cc:3055] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:06.377652  5146 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:06.378129  7735 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:06.379640  7617 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:06.380308  7681 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:06.294950  7630 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.474936  7630 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:06.476153  7753 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.397s	user 0.000s	sys 0.012s
W20260709 12:24:06.454897  7723 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.780s	user 0.010s	sys 0.006s
I20260709 12:24:06.446127  7537 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:06.385197  5075 rpcz_store.cc:269] 0709 12:24:05.155160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.156510 (+  1350us) service_pool.cc:224] Handling call
0709 12:24:06.344122 (+1187612us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b540058a554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:06.344313 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559750400}
W20260709 12:24:06.340750  7719 consensus_meta.cc:287] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.703s	user 0.013s	sys 0.000s
I20260709 12:24:06.390961  7655 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:06.319977  7666 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:06.396845  5145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.061s	sys 0.066s
W20260709 12:24:06.481040  5145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.062s	sys 0.066s
W20260709 12:24:06.353437  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 71305.7us
I20260709 12:24:06.412123  4898 catalog_manager.cc:5697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 4 to 5, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:06.392782  7703 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.439659  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:06.484467  5948 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:06.441394  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:06.442307  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:06.443769  5946 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:06.444633  5217 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:06.449677  5216 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:06.395927  7329 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.453981  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:06.455220  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:06.490944  5221 raft_consensus.cc:2393] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:06.454398  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:06.490837  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:06.470780  7624 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.473400  7722 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:06.474344  7617 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:06.287555  7197 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:06.479187  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:06.482436  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.410715  5223 pending_rounds.cc:107] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 12:24:06.489121  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:06.463809  7660 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:06.455509  7539 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:06.498927  5223 raft_consensus.cc:2889] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:06.506322  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 804) took 1002 ms. Trace:
I20260709 12:24:06.506716  5223 rpcz_store.cc:276] 0709 12:24:05.504356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.504547 (+   191us) service_pool.cc:224] Handling call
0709 12:24:05.504962 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.180525 (+675563us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b540033cfa0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a84b906f 00007fd8a8531f60 00007fd8a85337de 00007fd8a853db49 00007fd8a853e416 00007fd8a853ad8f 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62
0709 12:24:06.506033 (+325508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.506250 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410585472}
I20260709 12:24:06.454260  7499 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:06.476740  7723 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:06.518836  5141 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:06.483814  7604 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:06.521152  5189 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:06.483750  7735 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:06.530819  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:06.532382  5942 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:06.505956  7432 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.537263  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:06.529815  5149 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:06.536595  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:06.538884  5948 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:06.516551  7625 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:06.514422  7626 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:06.487957  7719 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.536504  5143 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:06.530828  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 176369us
I20260709 12:24:06.541868  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:06.537951  7550 consensus_meta.cc:287] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.004s
I20260709 12:24:06.525728  7709 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:06.549886  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:06.556722  5223 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:06.533391  7444 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:06.550156  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:06.529699  7712 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:06.511163  7624 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:06.558561  5215 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:24:06.560144  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:06.542313  7318 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:06.563652  7703 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:06.550792  6921 consensus_meta.cc:287] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.000s
I20260709 12:24:06.574582  5948 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:06.582803  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:06.556900  7550 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:06.531739  7604 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.534265  7735 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:06.558916  7691 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:06.561125  7722 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:06.590452  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.560529  7444 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:06.542819  7432 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:06.528492  7537 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:06.598004  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:06.598107  5142 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:06.599234  5948 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:06.600384  5223 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:06.601819  5943 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:06.606376  5943 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:06.546761  7660 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:06.596000  7577 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.562038  7709 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:06.606751  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:06.556172  7617 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:06.612504  7617 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:06.614677  5190 leader_election.cc:400] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8.
I20260709 12:24:06.615170  5190 leader_election.cc:403] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:06.601255  7634 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:06.615976  7762 consensus_queue.cc:1175] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ceec344dd00f40c18ef58414ce147016" responder_term: 7 status { last_received { term: 7 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 7 index: 5 } }
W20260709 12:24:06.615973  7241 consensus_queue.cc:1175] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ceec344dd00f40c18ef58414ce147016" responder_term: 7 status { last_received { term: 7 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 7 index: 4 } }
I20260709 12:24:06.617358  7577 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.624506  7577 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:06.626132  7709 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:06.627882  7432 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:06.619992  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 495) took 1289 ms. Trace:
W20260709 12:24:06.629432  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.664680  5188 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:06.630741  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:06.635162  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:06.639879  7723 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:06.639720  7604 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:06.639750  7719 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:06.640702  7329 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:06.636305  7634 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.631594  7712 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:06.646874  7735 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:06.619771  7624 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:06.648089  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:06.626142  7499 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:06.598834  7722 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:06.650744  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:06.651649  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 969) took 1059 ms. Trace:
I20260709 12:24:06.653892  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 426) took 1357 ms. Trace:
I20260709 12:24:06.660696  5076 rpcz_store.cc:276] 0709 12:24:05.330866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.113995 (+783129us) service_pool.cc:224] Handling call
0709 12:24:06.114527 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.290431 (+175904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.619680 (+329249us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b54003dfed4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:06.619900 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24140544}
W20260709 12:24:06.690979  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 79583.6us
I20260709 12:24:06.688413  5072 rpcz_store.cc:276] 0709 12:24:05.592514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.136765 (+544251us) service_pool.cc:224] Handling call
0709 12:24:06.171675 (+ 34910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.455101 (+283426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.651350 (+196249us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540058af54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:06.651575 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455506176}
I20260709 12:24:06.689911  5220 rpcz_store.cc:276] 0709 12:24:05.296937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.297062 (+   125us) service_pool.cc:224] Handling call
0709 12:24:06.653671 (+1356609us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b5400514854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:06.653794 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800374144}
I20260709 12:24:06.693614  5223 raft_consensus.cc:2410] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:24:06.676270  7721 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:06.677676  7712 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.698813  5899 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:06.699090  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:06.699862  5222 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:06.700945  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:06.689507  7709 consensus_queue.cc:1059] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
W20260709 12:24:06.702246  7750 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.668s	user 0.001s	sys 0.011s
I20260709 12:24:06.702622  7709 raft_consensus.cc:3055] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:06.703065  7709 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:06.703989  7709 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:06.704536  7660 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:06.689206  7703 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:24:06.705729  7537 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:06.707059  7722 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:06.703084  7719 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:06.710757  7719 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:06.711215  7719 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:06.711146  7702 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:06.712510  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 728) took 1555 ms. Trace:
I20260709 12:24:06.712940  5073 rpcz_store.cc:276] 0709 12:24:05.156642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.026910 (+870268us) service_pool.cc:224] Handling call
0709 12:24:06.051941 (+ 25031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.070911 (+ 18970us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:06.263284 (+192373us) log.cc:844] Serialized 14 byte log entry
0709 12:24:06.494738 (+231454us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.494823 (+    85us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:06.494900 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.712210 (+217310us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b540041e1d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:06.712264 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.712436 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6310,"prepare.queue_time_us":365488,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":7962,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":8811,"spinlock_wait_cycles":50903552,"thread_start_us":351245,"threads_started":2,"wal-append.queue_time_us":241513}
I20260709 12:24:06.715384  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 494) took 1384 ms. Trace:
I20260709 12:24:06.715854  5070 rpcz_store.cc:276] 0709 12:24:05.330593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.082880 (+752287us) service_pool.cc:224] Handling call
0709 12:24:06.198878 (+115998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.199742 (+   864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.715140 (+515398us) spinlock_profiling.cc:243] Waited 90.3 ms on lock 0x7b5400589650. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:06.715311 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207719296}
I20260709 12:24:06.720382  7444 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:06.726855  7630 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:06.728286  7698 consensus_meta.cc:287] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.001s
I20260709 12:24:06.727820  7719 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:06.728922  7712 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:06.722143  7197 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:06.723343  5071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 963) took 1728 ms. Trace:
I20260709 12:24:06.724392  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 997) took 1062 ms. Trace:
W20260709 12:24:06.749281  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 219582us
I20260709 12:24:06.734102  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:06.730086  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:06.758358  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:06.743638  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 674) took 1274 ms. Trace:
I20260709 12:24:06.779704  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:06.779717  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:06.780683  5070 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:06.775439  5143 rpcz_store.cc:276] 0709 12:24:05.662494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.663586 (+  1092us) service_pool.cc:224] Handling call
0709 12:24:05.664064 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.544773 (+880709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.724087 (+179314us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b54007aadd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:06.724324 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2005933952}
I20260709 12:24:06.740594  4898 catalog_manager.cc:5697] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 3 to 5, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:06.793736  5899 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:06.824746  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:06.908993  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.781334  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:06.776304  5071 rpcz_store.cc:276] 0709 12:24:04.995349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.995600 (+   251us) service_pool.cc:224] Handling call
0709 12:24:05.015309 (+ 19709us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:05.944563 (+929254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.944619 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:06.019723 (+ 75104us) log.cc:844] Serialized 26 byte log entry
0709 12:24:06.264415 (+244692us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:06.264489 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.722794 (+458305us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b540017fa54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:06.722859 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:06.722918 (+    59us) raft_consensus.cc:1685] finished
0709 12:24:06.722937 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.723244 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6948,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":60742,"spinlock_wait_cycles":2135800576,"thread_start_us":190367,"threads_started":1,"wal-append.queue_time_us":342939}
I20260709 12:24:06.782765  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:06.784809  5215 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:06.776543  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:06.778121  5217 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:06.926127  5073 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:24:06.927454  5221 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:06.926838  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:06.928225  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 134667us
I20260709 12:24:06.780692  7432 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:06.780803  5218 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
W20260709 12:24:06.912550  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 154281us
I20260709 12:24:06.762820  7757 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:06.750842  7563 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.839s	user 0.005s	sys 0.010s
I20260709 12:24:06.934672  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:24:06.937855  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 161421us
I20260709 12:24:06.765971  7708 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:06.705456  7709 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:06.945694  7698 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:06.950508  7698 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:06.950933  7698 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:06.750080  7702 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:06.749902  7626 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:06.954821  7626 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:06.958480  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:06.730441  7719 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:06.959556  7500 raft_consensus.cc:2764] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:06.942359  7432 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:06.947690  5189 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:06.911872  7666 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.749706  7707 consensus_queue.cc:1059] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.200s
I20260709 12:24:06.949975  7703 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:06.912106  7585 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:06.741067  7721 raft_consensus.cc:697] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:06.780385  5945 rpcz_store.cc:276] 0709 12:24:05.469047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.469197 (+   150us) service_pool.cc:224] Handling call
0709 12:24:05.491714 (+ 22517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.022658 (+530944us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.022723 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.022796 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:06.022847 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.620610 (+597763us) spinlock_profiling.cc:243] Waited 7.6 ms on lock 0x7b180010be80. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a541c2a6 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419
0709 12:24:06.743270 (+122660us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b5400003a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:06.743347 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.743549 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75352,"raft.queue_time_us":62951,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":11986,"spinlock_wait_cycles":1056624768}
I20260709 12:24:06.934105  5899 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:06.947870  5220 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:06.964720  5189 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:06.932472  7618 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:06.953287  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:06.781018  7197 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:06.970744  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:06.991392  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.983187  5224 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:06.984181  5941 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:06.932024  7693 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8.
I20260709 12:24:06.729437  7630 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:06.997128  5189 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:06.998917  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:06.988500  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 54477.7us
I20260709 12:24:06.932891  7616 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:06.935926  7686 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:06.932684  7584 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:06.970641  7746 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.000s
I20260709 12:24:06.781555  7770 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.224s
I20260709 12:24:06.763518  7386 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:06.985085  7703 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:07.008682  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:07.040129  5944 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:06.980791  7666 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:07.012774  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:07.047854  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.013890  7604 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.020509  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.997826  7722 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:07.025713  7634 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:06.948139  7757 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:07.004474  7709 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:06.996680  7637 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:06.968155  7708 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.034194  7432 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:07.037550  7686 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:07.010695  5946 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:07.050729  5899 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:07.054620  7729 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.006s
W20260709 12:24:07.093832  7769 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:24:07.091501  7634 consensus_queue.cc:1059] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:24:07.024120  5221 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
W20260709 12:24:07.064872  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 93377.3us
I20260709 12:24:07.076154  5944 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:07.030253  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:07.038848  7386 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:07.047664  7539 consensus_meta.cc:287] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.003s
I20260709 12:24:07.092181  7617 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:07.097333  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:07.109155  5943 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:07.045809  7585 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:06.995139  7618 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.100466  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:07.046984  7703 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:07.111567  5944 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:07.107579  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:07.112254  5940 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:07.113749  5076 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:07.089196  7619 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.119864  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.120858  5939 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:07.096460  7666 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:07.103237  7761 log.cc:927] Time spent T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:24:07.050256  7616 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:07.107812  7749 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:24:07.049043  7584 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.106833  7694 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:07.041304  7746 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.097985  7686 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:07.063889  7604 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.090356  7637 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.110824  7585 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.095562  7712 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:07.127718  7432 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.123896  7630 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.111116  7539 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:07.170928  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 134247us
I20260709 12:24:07.172334  5067 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:07.138480  7694 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.181375  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.146358  7637 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:07.181723  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.188933  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.146864  7585 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:07.189699  5068 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:07.190672  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.191462  5068 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 9: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:07.119199  7617 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.130283  7619 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.194235  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.198227  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.199077  5068 raft_consensus.cc:2376] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8. Re-sending same reply.
I20260709 12:24:07.204825  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.205554  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.205768  5073 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:24:07.206314  5074 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:07.112329  7721 consensus_queue.cc:237] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.110782  7197 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.209622  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.215811  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.222805  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.228883  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.183379  7694 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:07.233829  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.242892  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.243278  5074 raft_consensus.cc:2376] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8. Re-sending same reply.
I20260709 12:24:07.243665  5073 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:07.238432  5945 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:24:07.240588  5067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 964) took 1649 ms. Trace:
I20260709 12:24:07.170455  7630 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:07.244858  5067 rpcz_store.cc:276] 0709 12:24:05.591555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.121914 (+530359us) service_pool.cc:224] Handling call
0709 12:24:06.122205 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.176599 (+1054394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.240293 (+ 63694us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000307d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:07.240520 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2414608640}
I20260709 12:24:07.207208  7708 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.176571  7698 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:07.035113  7689 consensus_queue.cc:1175] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9847a7797ddc4df689a1ab0911af8785" responder_term: 8 status { last_received { term: 8 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:07.165300  7432 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.255575  5074 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:07.251526  5067 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:07.250581  7757 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:07.252893  5072 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:07.258632  5067 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:07.254455  5068 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:07.253522  7197 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:07.270965  5075 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:07.271482  5074 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:24:07.287519  5940 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:07.263782  7619 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:07.308466  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 98963.4us
I20260709 12:24:07.296243  7781 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.351069  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:07.351913  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.353103  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.354370  5071 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:07.319983  7584 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:07.318632  7616 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:07.322808  7774 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:24:07.359306  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.363610  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.357877  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.364372  5070 raft_consensus.cc:2376] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9. Re-sending same reply.
I20260709 12:24:07.364615  5071 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:07.325688  7604 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.367616  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.372819  5072 raft_consensus.cc:2393] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 6: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:07.325920  7746 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:07.345861  7712 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:07.316524  7781 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.357604  7785 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:07.356658  7705 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:07.405716  7770 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.364717  7617 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:07.406960  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.407124  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.407951  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.364468  7539 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:07.406898  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.411742  5073 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:24:07.411811  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:07.416818  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:07.422402  5217 raft_consensus.cc:2393] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:07.426379  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:07.402030  7791 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.431630  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 815) took 1476 ms. Trace:
I20260709 12:24:07.408262  7746 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.432037  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 333185us
I20260709 12:24:07.433072  5222 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:07.418694  7784 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:24:07.435808  5899 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:07.437718  7746 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.414505  7630 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:07.441715  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.414376  7792 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:07.415870  7705 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:07.430460  7787 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.444010  5114 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.512s (SENT) [suppressed 33 similar messages]
I20260709 12:24:07.432654  7708 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:07.445043  7698 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:07.445343  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:07.446516  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.473065  5074 raft_consensus.cc:2393] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 6: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:07.447443  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.448168  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.450822  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:07.433969  5219 rpcz_store.cc:276] 0709 12:24:05.955614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.955850 (+   236us) service_pool.cc:224] Handling call
0709 12:24:05.956227 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.020615 (+ 64388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.020666 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:06.447855 (+427189us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b18003d87c0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a541c2a6 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a
0709 12:24:06.726729 (+278874us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b2c000af528. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:24:06.727317 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 12:24:06.970260 (+242943us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:06.970319 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.970436 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:07.431258 (+460822us) raft_consensus.cc:1685] finished
0709 12:24:07.431293 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.431540 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3639,"raft.queue_time_us":64143,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":11618,"spinlock_wait_cycles":17420928,"thread_start_us":242468,"threads_started":1,"wal-append.queue_time_us":323175}
I20260709 12:24:07.414891  7634 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:07.439735  7444 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.414922  7641 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.461592  5067 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:24:07.435149  7763 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:24:07.482388  7634 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.482452  7641 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.413553  7785 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.406477  7770 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.479722  5073 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:07.453209  7617 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:07.480662  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.494455  5072 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:07.470332  7432 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:07.504487  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 59240.8us
W20260709 12:24:07.506165  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:07.522289  7797 log.cc:927] Time spent T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:24:07.524714  6921 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:07.528941  7641 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.488406  7444 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:07.531268  7444 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:07.497277  7241 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
W20260709 12:24:07.525285  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 81319.3us
W20260709 12:24:07.534473  5114 leader_election.cc:336] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.681s (SENT)
I20260709 12:24:07.534812  7705 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:07.526597  5223 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:07.502629  7617 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:07.472255  7792 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.535109  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:07.536319  7617 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:07.537488  7792 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:07.546578  7630 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:07.549132  7539 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.553009  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 49478.1us
I20260709 12:24:07.454418  7791 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.495107  7604 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.480945  7787 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.532539  7444 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:07.552493  7694 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:07.580814  7539 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.576119  7705 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:24:07.587405  7686 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.562366  5898 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:07.576419  7616 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:24:07.560842  7702 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 12:24:07.583437  7802 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:24:07.583623  7746 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:07.580359  7585 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:07.584288  7604 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.589401  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:07.590097  7241 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:07.592425  5114 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:07.593194  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 87791.7us
I20260709 12:24:07.599998  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:07.600263  5188 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:07.603789  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:07.605566  7630 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:07.606242  7630 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:07.607028  7604 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:07.607525  7585 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:07.607780  7241 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:07.604372  7577 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:07.608493  7577 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:07.608384  7585 consensus_queue.cc:1059] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:07.609089  7686 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:07.609836  7803 raft_consensus.cc:3055] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:07.610313  7803 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:07.610740  7787 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:07.611279  7803 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.612223  7785 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.612804  7803 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:07.613018  7585 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:07.613982  7386 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.614596  7386 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.615363  7791 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.615124  7585 consensus_queue.cc:1059] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:07.616405  7746 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.616643  7585 consensus_queue.cc:1243] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:07.617484  7539 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:07.594477  7729 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:07.616981  7746 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.618467  7729 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.618976  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.609159  7577 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:07.622498  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.625557  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.614030  7694 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:07.627745  7788 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:07.628229  7686 consensus_queue.cc:1059] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:07.628490  7788 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.629029  7770 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.607757  7630 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.631110  7641 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.631738  7641 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.632359  7500 raft_consensus.cc:3055] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:07.632799  7500 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:07.633772  7500 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.634783  7500 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:07.635766  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.603322  7619 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:07.604837  7790 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:07.637776  7790 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.638159  7790 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:07.626955  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:07.647568  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.647902  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.648669  5073 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:07.656148  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.658140  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.661173  5218 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:07.663111  5948 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:07.663210  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:07.663955  5943 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:07.596100  7637 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:07.641402  7577 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.665182  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.666602  7637 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:07.668340  7776 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.454s	user 0.002s	sys 0.002s
I20260709 12:24:07.666610  7577 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.669512  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.670125  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:07.670593  7641 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:07.670939  5940 raft_consensus.cc:2368] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:07.672714  7634 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.608983  7781 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:07.673202  5940 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:07.667955  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:07.676748  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:07.666630  5224 raft_consensus.cc:2410] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:24:07.687117  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.650286  7791 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.699062  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.702174  5073 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:07.661015  7616 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:07.713289  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:07.714468  5945 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:07.660724  7770 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:07.627738  7703 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:07.694193  7634 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:07.736757  7634 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.710754  7791 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:07.751976  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:07.719547  7616 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:07.753110  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 96616.7us
I20260709 12:24:07.752557  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:07.756513  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.764097  7789 raft_consensus.cc:3055] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:07.764748  7789 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:07.765782  7789 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.768385  7789 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:07.769137  7721 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.769862  7721 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.771059  7729 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:07.777307  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.777822  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.785267  5940 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:07.754720  7634 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:07.788331  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.791589  7722 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.795019  5070 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:07.799966  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.801101  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.801268  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:07.802006  5222 raft_consensus.cc:2410] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:24:07.806255  7805 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:07.782835  7791 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:07.811620  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 682) took 1359 ms (client timeout 1547 ms). Trace:
W20260709 12:24:07.812160  5941 rpcz_store.cc:269] 0709 12:24:06.441122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.442075 (+   953us) service_pool.cc:224] Handling call
0709 12:24:07.800623 (+1358548us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b540000ad54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:07.800741 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1144223488}
I20260709 12:24:07.804864  5041 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:07.740186  7770 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.775206  7721 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.819599  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.820529  5073 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:07.822474  7386 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:07.780001  7729 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.835381  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:07.801994  7722 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:07.835057  7619 raft_consensus.cc:3055] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:07.843068  7386 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:07.861654  7634 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:07.865931  5217 raft_consensus.cc:2393] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:07.868579  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:07.852133  7807 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 12:24:07.892015  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.860495  7386 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.858429  7619 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:07.846426  7729 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.867058  7746 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:07.899377  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.900367  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:07.901517  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.901044  7386 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:07.904994  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.905602  7708 consensus_queue.cc:1059] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:07.906497  7708 raft_consensus.cc:3055] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:07.906841  7708 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:07.903740  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:07.907649  7708 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.909091  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.909857  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:07.911609  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.912802  7729 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.913808  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.902753  5041 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:07.913026  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:07.914732  5074 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:07.915481  5068 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:07.914587  5070 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:24:07.914099  7788 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.892740  7432 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.899739  7799 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.921828  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.920615  5041 leader_election.cc:400] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:07.924481  7818 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:07.914153  7708 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:07.913810  7619 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.928469  7619 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:07.935415  6921 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:07.936393  5220 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:07.927244  7818 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.926774  5041 leader_election.cc:403] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:24:07.942063  7729 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:07.943346  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:07.944794  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:07.945896  7788 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:07.955927  7812 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:24:07.950163  7788 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.948274  7818 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:07.956899  7788 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:07.938707  7432 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.964149  7432 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:07.948454  7386 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.965494  7770 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.967258  7386 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:07.970033  7616 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:07.940937  7634 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:07.940351  7703 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:07.974045  7614 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:07.975185  7709 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:07.975644  7709 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:07.976552  7709 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:07.977209  7788 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.977962  5073 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:07.980588  7788 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:07.941111  7799 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:07.983923  7780 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.750s	user 0.002s	sys 0.001s
W20260709 12:24:07.990700  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.994642  7619 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.995277  7619 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:07.996193  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 433) took 1333 ms. Trace:
I20260709 12:24:07.996704  5216 rpcz_store.cc:276] 0709 12:24:06.662839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.738468 (+ 75629us) service_pool.cc:224] Handling call
0709 12:24:06.738949 (+   481us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:06.739538 (+   589us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:06.967055 (+227517us) log.cc:844] Serialized 14 byte log entry
0709 12:24:07.276531 (+309476us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:07.290040 (+ 13509us) log.cc:844] Serialized 52 byte log entry
0709 12:24:07.290284 (+   244us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:07.290332 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.290445 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:07.983164 (+692719us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:07.995822 (+ 12658us) raft_consensus.cc:1685] finished
0709 12:24:07.995943 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.996120 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":316165,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":210677,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":15752,"spinlock_wait_cycles":38219648,"thread_start_us":503409,"threads_started":3,"wal-append.queue_time_us":174264}
I20260709 12:24:08.000389  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:08.001312  5072 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:07.990762  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:07.997649  7619 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:08.003276  5114 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:08.005764  7702 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:08.006446  7702 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:08.007453  7702 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:08.008492  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.009500  5146 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:08.009931  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:08.010566  5141 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:08.010779  7705 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:08.011231  7705 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:08.011408  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.011698  7705 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:08.012050  5149 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:24:08.015123  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:08.015726  5142 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:08.016269  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.017649  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.017881  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:08.018620  5947 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:08.019042  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.017818  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:08.020411  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:08.019738  5146 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:24:08.021972  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:08.023674  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:08.025384  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:08.026254  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:08.026433  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:08.026580  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:08.026945  5948 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:08.027320  5940 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:08.027750  7790 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:08.028326  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:08.028952  5219 raft_consensus.cc:2393] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:08.029182  5144 raft_consensus.cc:2393] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:07.985303  7799 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:08.031243  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.031996  5144 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:08.032059  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.033025  5150 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:08.033278  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:08.034006  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:08.034622  5150 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:24:08.035313  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:08.036814  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.021461  7709 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:08.038511  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.033118  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:08.039199  5144 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:08.039209  5149 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:24:08.031912  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.031334  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.041965  5148 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:08.004992  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:08.046873  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.047646  5146 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:08.047677  5148 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:08.050910  5898 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:08.054198  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:08.054982  5075 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:08.057931  5141 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:08.064983  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.065788  5148 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:08.067811  5075 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:08.070775  5898 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:08.075477  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 129455us
I20260709 12:24:08.077756  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:08.070933  5899 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:08.078575  5076 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:08.081240  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:08.083155  7618 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:08.062453  7796 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:08.086054  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 81186.8us
W20260709 12:24:08.089708  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.092628  5144 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:08.102473  5947 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:08.101526  5150 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:08.103423  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:08.104424  5942 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:24:08.093410  5142 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:08.067544  5948 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:08.068183  7719 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:08.071998  7799 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.115124  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:08.076478  7563 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:08.118088  5147 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:08.086113  7762 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:08.124961  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.088562  7782 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:08.086160  7539 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:08.086622  7641 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:08.163333  7818 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:08.164666  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 107224us
W20260709 12:24:08.169863  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.116220  7719 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.086018  7796 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:08.186354  5947 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:08.107483  7788 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:08.184005  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:08.192019  5898 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:08.193513  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:08.194959  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:08.117657  7799 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.244318  7585 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:08.245911  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:08.247144  7824 log.cc:927] Time spent T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:24:08.247366  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.248018  5146 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:08.126883  7563 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.184767  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:08.248827  7563 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:08.212363  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:08.245911  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:08.254323  7823 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:08.254473  5221 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:08.245955  5147 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:08.126157  7762 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.237757  7796 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:08.239686  5144 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:08.240218  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.194526  7721 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.212611  7719 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:08.241823  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:08.242548  5220 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:08.317000  5142 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:08.243239  7788 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:08.243757  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 50326.4us
I20260709 12:24:08.244465  7799 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.123152  7729 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:08.257421  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:08.253512  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.260388  5188 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:08.264328  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 80490.8us
W20260709 12:24:08.344375  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:08.265269  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 73347.7us
W20260709 12:24:08.345778  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 85438.9us
I20260709 12:24:08.224985  5942 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:08.240547  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.268237  5148 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:08.239715  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:08.351388  5150 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:08.279098  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:08.352263  5144 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:08.266089  7762 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:08.339124  5221 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:08.332850  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:08.343225  5146 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:08.261180  7703 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:08.360935  7703 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:08.364575  7703 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:08.262583  7585 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:08.365603  7585 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:08.368629  7585 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:08.316800  7500 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.644s	user 0.015s	sys 0.000s
I20260709 12:24:08.212131  7655 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 12:24:08.356551  5188 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:08.233610  7619 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 12:24:08.278445  7694 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:08.380457  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 47743.2us
I20260709 12:24:08.381742  5144 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:08.383814  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:08.385026  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:08.386235  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:08.338692  7386 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:08.386561  5216 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:08.388722  7789 consensus_meta.cc:287] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.619s	user 0.014s	sys 0.001s
I20260709 12:24:08.389730  5217 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:08.389989  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:08.390571  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:08.394254  4898 catalog_manager.cc:5697] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 1 to 2, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:08.400207  5220 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:08.396560  5076 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:24:08.383996  7822 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:24:08.406863  5222 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:24:08.412101  5222 pending_rounds.cc:85] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 4
I20260709 12:24:08.412515  5222 pending_rounds.cc:107] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:24:08.412850  5222 raft_consensus.cc:2889] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:08.410153  7788 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:08.375582  7796 raft_consensus.cc:2764] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:08.415232  5224 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:08.335258  7721 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:08.390677  6921 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:08.409983  7563 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:08.394083  7694 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:08.443050  5899 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:08.450439  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:08.451073  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.408514  5189 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:08.398838  7803 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:08.414371  7703 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:08.451591  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.460126  7770 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.464484  7789 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:08.474396  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:08.476816  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:08.478541  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:08.465161  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 56739.5us
I20260709 12:24:08.479246  5075 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:08.479851  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:08.480001  5189 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:08.484665  5943 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:08.407771  7702 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:24:08.488052  5188 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:08.488875  7712 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:08.489464  5069 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:08.415377  7806 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:24:08.493739  5076 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:24:08.494889  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.480918  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:08.496161  5074 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.495915  7785 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:08.498965  7825 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:24:08.501039  7829 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:24:08.502347  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:08.507423  5074 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
W20260709 12:24:08.512151  7722 consensus_meta.cc:287] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.658s	user 0.010s	sys 0.001s
I20260709 12:24:08.491513  7444 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:24:08.467283  7803 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.466543  7790 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:08.445439  6921 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.466914  7757 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:08.520610  5220 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:08.521428  5188 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:08.512863  7722 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:08.563925  5216 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:08.538370  5218 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
W20260709 12:24:08.578696  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 275157us
W20260709 12:24:08.585953  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 330502us
W20260709 12:24:08.586243  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 135354us
I20260709 12:24:08.525631  7803 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:08.640251  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 62860us
W20260709 12:24:08.635349  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 114021us
I20260709 12:24:08.633646  4898 catalog_manager.cc:5697] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 1 to 3, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.519143  7789 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:08.644520  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.561205  6921 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:08.526129  7585 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:08.664860  7585 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.665346  7585 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:08.666282  4899 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43226 (request call id 129) took 15713 ms (client timeout 15000 ms). Trace:
I20260709 12:24:08.641398  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:08.670647  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:08.672091  5041 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:08.653999  7703 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:08.674187  7703 raft_consensus.cc:697] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:08.675328  7703 consensus_queue.cc:237] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:08.678541  7500 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:08.679386  7500 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:08.680646  7500 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:08.664285  7624 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.665185  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:08.663177  7789 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:08.686173  7634 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.684839  7624 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:08.687307  7634 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:08.685949  5218 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
W20260709 12:24:08.666774  4899 rpcz_store.cc:269] 0709 12:23:52.952615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.957682 (+  5067us) service_pool.cc:224] Handling call
0709 12:24:08.666001 (+15708319us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:00.329379 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:00.370239 (+ 40860us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:00.370311 (+    72us) write_op.cc:435] Acquired schema lock
0709 12:24:00.370334 (+    23us) tablet.cc:662] Decoding operations
0709 12:24:00.371969 (+  1635us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:00.372074 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:00.372111 (+    37us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:24:00.373467 (+  1356us) tablet.cc:701] Row locks acquired
0709 12:24:00.373492 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:24:00.402879 (+ 29387us) write_op.cc:270] Start()
0709 12:24:00.413299 (+ 10420us) write_op.cc:276] Timestamp: P: 1783599840402803 usec, L: 0
0709 12:24:00.420796 (+  7497us) op_driver.cc:348] REPLICATION: starting
0709 12:24:00.435516 (+ 14720us) log.cc:844] Serialized 6698 byte log entry
0709 12:24:01.532995 (+1097479us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:01.643525 (+110530us) op_driver.cc:464] REPLICATION: finished
0709 12:24:02.029038 (+385513us) write_op.cc:301] APPLY: starting
0709 12:24:02.030642 (+  1604us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:02.055877 (+ 25235us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:02.055941 (+    64us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:02.163549 (+107608us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:02.319550 (+156001us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 12:24:02.324696 (+  5146us) write_op.cc:312] APPLY: finished
0709 12:24:02.378832 (+ 54136us) log.cc:844] Serialized 118 byte log entry
0709 12:24:02.379127 (+   295us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:02.379441 (+   314us) write_op.cc:454] Released schema lock
0709 12:24:02.379552 (+   111us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:02.380086 (+   534us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:02.380140 (+    54us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":503493,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":349995,"apply.run_cpu_time_us":8171,"apply.run_wall_time_us":376513,"num_ops":17,"prepare.queue_time_us":355277,"prepare.run_cpu_time_us":85051,"prepare.run_wall_time_us":329792,"raft.queue_time_us":60732,"raft.run_cpu_time_us":11196,"raft.run_wall_time_us":305078,"replication_time_us":1220626,"spinlock_wait_cycles":31427328,"thread_start_us":716439,"threads_started":4,"wal-append.queue_time_us":470699,"wal-append.run_cpu_time_us":18231,"wal-append.run_wall_time_us":2464487}]]}
I20260709 12:24:08.682174  7788 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:08.578047  7712 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:08.485504  7770 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:08.701731  5041 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:08.555462  7798 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.144s
I20260709 12:24:08.642061  7790 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:08.757179  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.753989  7619 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:08.731235  7729 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:08.774093  7619 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:08.774389  7729 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:08.776618  7729 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:08.777642  7712 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:08.726555  7788 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.772877  4898 catalog_manager.cc:5697] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 1 to 3, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.754508  7782 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:08.755582  7818 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:08.776561  7619 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:08.783092  4898 catalog_manager.cc:5697] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 1 to 3, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 3 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:08.799365  7563 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:08.698453  7694 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:08.810462  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 405707us
I20260709 12:24:08.782532  7788 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:08.766178  7708 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:24:08.801294  7563 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:08.764230  7790 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:08.801919  7782 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:08.834590  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:08.795665  7386 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:08.835664  5067 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:08.837158  7837 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 12:24:08.849113  7634 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:08.851698  7790 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:08.820773  7652 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:08.835187  6921 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:08.830622  7563 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:08.861822  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.867489  5188 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:08.868908  7790 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:08.869428  7790 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.869967  7790 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:08.870312  7834 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:24:08.895514  7698 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.896258  7698 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:08.897917  7722 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:08.898882  7698 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:08.902594  7770 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:08.836330  7721 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:08.905925  7624 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:08.906657  7563 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:08.907059  7782 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:08.908195  6921 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:08.907579  7721 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.909512  7721 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:08.913301  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:08.914295  5075 raft_consensus.cc:2393] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:08.919683  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 332178us
I20260709 12:24:08.930509  5067 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:24:08.932088  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.941727  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:08.945914  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:08.946693  5942 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:08.946955  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:08.947530  5071 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:08.928912  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:08.963531  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:08.970367  5941 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:08.983784  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 395650us
W20260709 12:24:09.039132  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.977322  5942 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:08.984191  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:08.964301  7698 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:08.975508  7584 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:09.036516  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 253515us
W20260709 12:24:09.039157  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.918428  6921 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.979562  7721 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:09.045006  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 60952.7us
I20260709 12:24:09.044344  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:09.063437  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.069553  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.066624  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:09.065253  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 112708us
W20260709 12:24:09.038059  7762 consensus_meta.cc:287] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.653s	user 0.012s	sys 0.002s
W20260709 12:24:09.056921  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.076720  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.019886  7846 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:09.075491  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.026160  7641 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:09.077440  5941 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:09.079331  5041 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:09.080317  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.064281  7837 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:09.082186  5942 raft_consensus.cc:2410] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:09.085049  5068 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:09.029731  7844 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:09.103024  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:09.103535  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:09.105400  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:09.105842  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.106549  5948 raft_consensus.cc:2368] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9.
I20260709 12:24:09.107194  5221 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:09.107503  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.108090  5942 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:09.110308  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:09.110937  5221 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:09.100965  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:09.114285  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:09.115370  5216 raft_consensus.cc:2368] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
W20260709 12:24:09.117636  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 43937.8us
I20260709 12:24:09.081573  7847 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.081840  7386 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:09.093712  7705 consensus_meta.cc:287] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.081s	user 0.005s	sys 0.011s
I20260709 12:24:09.119190  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:09.122668  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:09.101086  7799 consensus_meta.cc:287] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.727s	user 0.014s	sys 0.000s
I20260709 12:24:09.056528  7698 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:09.132773  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:09.123899  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:09.084574  7584 raft_consensus.cc:697] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:09.088846  7689 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:09.135486  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:09.074769  7845 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:24:09.072687  6921 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.137285  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:09.138835  5899 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:09.090812  7803 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:09.080624  7762 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:09.120244  7539 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:09.167654  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:09.192914  5942 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 7: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:09.144698  5939 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:09.130492  7386 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:09.119544  7619 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:24:09.154018  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:09.152603  5217 raft_consensus.cc:2410] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:24:09.191725  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 56362.4us
I20260709 12:24:09.200593  7689 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:09.204478  7584 consensus_queue.cc:237] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.205010  7689 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:09.206039  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.206897  5948 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:09.200788  7746 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:09.208653  7641 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:09.209301  7641 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:09.211973  7563 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:09.212620  7563 raft_consensus.cc:697] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:09.213696  7563 consensus_queue.cc:237] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.122012  7585 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:09.216683  7585 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:09.218343  7790 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:09.219403  6921 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:09.220597  7790 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:09.221975  7719 consensus_meta.cc:287] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.869s	user 0.019s	sys 0.000s
I20260709 12:24:09.161928  7652 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:09.194953  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 56267.1us
I20260709 12:24:09.222422  7719 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:09.222967  7803 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:09.222700  7652 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.198019  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.223305  5899 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:09.135859  7847 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:09.224352  5899 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:09.224913  7694 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:09.225476  7694 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:09.225747  7842 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:09.222646  7849 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:24:09.226585  7694 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.203759  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:09.228686  7708 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:09.228962  5945 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:09.101023  7844 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:09.234254  5042 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:09.236197  7844 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:09.236344  7839 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 12:24:09.162447  7444 consensus_queue.cc:1059] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:24:09.137501  7705 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:09.237905  7444 raft_consensus.cc:3055] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:09.238335  7444 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:09.238380  7619 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:09.238814  7619 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:09.239317  7444 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:09.239653  7619 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:09.240062  7705 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:09.240702  7444 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:09.240854  7619 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:09.241899  5076 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:24:09.238428  5143 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:09.237919  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 761) took 1026 ms. Trace:
I20260709 12:24:09.243546  5073 rpcz_store.cc:276] 0709 12:24:08.211201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.211365 (+   164us) service_pool.cc:224] Handling call
0709 12:24:08.211737 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:08.263469 (+ 51732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:08.263505 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:08.326297 (+ 62792us) log.cc:844] Serialized 26 byte log entry
0709 12:24:08.720107 (+393810us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:08.720174 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.720283 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:09.237681 (+517398us) raft_consensus.cc:1685] finished
0709 12:24:09.237708 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.237847 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51242,"raft.queue_time_us":52340,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":58755,"thread_start_us":336005,"threads_started":1,"wal-append.queue_time_us":603890}
I20260709 12:24:09.245738  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:09.250623  5221 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:09.243144  5216 raft_consensus.cc:2368] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
I20260709 12:24:09.241789  7698 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:09.262012  5071 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:09.263044  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:09.265365  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.154596  7818 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:09.272089  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:09.272498  5215 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:09.270056  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:09.279175  5042 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:09.282841  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:09.282341  5217 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:09.142032  7799 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:09.143489  7837 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:09.203311  7634 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.285115  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.287593  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.288549  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.289932  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:09.314249  5940 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:09.323904  5899 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:09.293217  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 273480us
I20260709 12:24:09.299523  7757 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.300024  7641 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:09.273912  7539 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:09.273504  7690 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:09.275458  7729 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:09.301254  7698 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:09.302147  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.303649  5898 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:09.201896  7386 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:09.309443  7655 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.327872  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.328083  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:09.371585  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:09.372373  5944 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:09.282593  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:09.337270  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 922) took 1321 ms. Trace:
I20260709 12:24:09.374727  5143 rpcz_store.cc:276] 0709 12:24:08.015416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.017090 (+  1674us) service_pool.cc:224] Handling call
0709 12:24:08.017385 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.264123 (+1246738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.336796 (+ 72673us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003421d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:09.337176 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2807673216}
I20260709 12:24:09.343158  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:09.345342  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:09.283576  7624 raft_consensus.cc:2764] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:24:09.280843  7818 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.351436  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:09.353940  5946 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:09.377754  7818 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:09.378718  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.380713  7585 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:24:09.382606  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.386296  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:09.387223  5070 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:09.356247  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:09.368659  5939 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:09.199633  5222 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:09.374290  5948 raft_consensus.cc:2393] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:09.401953  5945 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:09.404827  5940 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:09.411266  5221 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:24:09.423648  5115 connection.cc:256] RPC call timeout handler was delayed by -1.2645890900002996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24150000000000002s for a response
I20260709 12:24:09.428267  5944 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:09.430353  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:09.438592  7539 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.439836  7721 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:09.423280  7705 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:09.441737  7846 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:09.444842  5115 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:09.477628  7762 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:09.478389  7762 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:09.478669  7846 raft_consensus.cc:2764] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:24:09.351167  7757 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.480204  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:09.452361  7641 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:09.445447  7652 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:09.329348  7634 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.481923  7757 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:09.457315  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:09.482745  5942 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:09.483628  7845 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:09.483951  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.484067  5219 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:09.481434  7705 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:09.406438  7585 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:09.349772  7837 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.486552  7585 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:09.486925  7719 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:09.487874  5115 leader_election.cc:400] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9.
I20260709 12:24:09.489478  5115 leader_election.cc:403] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:09.481820  7652 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:09.491840  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:09.386607  7655 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:09.498404  5069 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.010s	sys 0.006s
W20260709 12:24:09.498909  5069 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.010s	sys 0.006s
I20260709 12:24:09.494415  5070 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:09.486822  7705 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:09.514180  5947 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:09.496913  7618 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:24:09.499317  5115 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:09.515781  5188 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:09.492527  7652 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:09.498695  7719 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:09.500849  7846 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:09.520732  5115 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:09.525676  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:09.527225  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:09.531368  7719 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:09.532534  7719 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:09.534885  7655 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:09.536373  7789 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:09.536962  7789 raft_consensus.cc:697] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:09.538008  7789 consensus_queue.cc:237] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:09.542814  7799 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:09.524195  7846 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.543460  7846 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:09.543802  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.544200  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.545120  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.556967  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:09.520905  7652 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:09.523809  7705 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:09.518478  7850 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:09.548209  7788 consensus_meta.cc:287] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.698s	user 0.011s	sys 0.008s
I20260709 12:24:09.560904  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:09.590788  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.591997  5144 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:09.592165  5148 raft_consensus.cc:2393] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:09.565035  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.525765  7806 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:09.594825  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.598341  5150 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:09.605695  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.595875  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.607889  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.608992  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.609133  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:09.609848  5143 raft_consensus.cc:2410] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:09.606652  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.611107  5147 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:24:09.606745  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.605369  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:09.546448  7840 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:09.595533  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.598047  7788 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:09.617274  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.620510  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.527040  7831 consensus_queue.cc:1059] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:24:09.564458  7584 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:09.627591  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.628393  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.630235  5150 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:09.632275  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.632543  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.633358  5141 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:09.636193  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:09.592895  7842 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:09.642554  5072 raft_consensus.cc:2368] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
W20260709 12:24:09.642940  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 316800us
I20260709 12:24:09.642978  5068 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:09.670944  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:09.651202  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:09.683207  5189 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:09.692646  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:09.694797  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:09.625767  7840 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:09.693718  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:09.696844  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.597312  7850 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:09.712301  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.715915  7618 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9.
I20260709 12:24:09.716576  5943 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:09.726981  5114 leader_election.cc:400] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
I20260709 12:24:09.727519  5114 leader_election.cc:403] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:24:09.728930  7618 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:09.734903  7616 raft_consensus.cc:3055] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:09.735648  7616 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:09.739846  7618 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
I20260709 12:24:09.743762  7634 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:09.757805  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 341808us
I20260709 12:24:09.759675  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:09.744938  7854 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
W20260709 12:24:09.776371  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 186021us
I20260709 12:24:09.781723  5068 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:09.783025  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:09.783756  5143 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 7
W20260709 12:24:09.791960  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 71643.7us
I20260709 12:24:09.788357  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:09.796414  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 492817us
I20260709 12:24:09.802838  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:09.804665  5148 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:09.805111  5148 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:09.736788  7616 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:09.756664  7698 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:09.772645  7634 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:09.806025  5148 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.815224  5148 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:09.824158  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:09.826519  5150 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:09.829485  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.795318  7757 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.805861  7712 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:09.856842  5148 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:24:09.857498  5148 pending_rounds.cc:85] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 5
I20260709 12:24:09.857911  5148 pending_rounds.cc:107] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:24:09.858227  5148 raft_consensus.cc:2889] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:09.806236  7877 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:09.860915  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.834432  7444 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:09.843307  7843 raft_consensus.cc:2764] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:24:09.898591  5141 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:09.842113  7840 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.911422  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:09.913723  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.916494  7844 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:09.918444  7770 raft_consensus.cc:3055] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:09.918885  7770 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:09.919966  7770 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:09.923791  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:09.925431  5141 raft_consensus.cc:2376] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8. Re-sending same reply.
I20260709 12:24:09.814312  7634 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.927445  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.860200  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:09.930877  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.931052  5150 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:09.929654  5141 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:09.933482  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:09.917222  7844 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:09.846220  7616 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:09.960630  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 267045us
I20260709 12:24:09.970894  5189 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:09.962802  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.823231  7867 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.129s
I20260709 12:24:09.972458  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.993714  5150 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:09.999928  5141 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:09.902978  7845 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:10.002105  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:09.923288  7712 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:10.010962  5141 raft_consensus.cc:2393] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:09.967062  7873 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.348s	user 0.002s	sys 0.000s
I20260709 12:24:10.008867  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:09.932507  7634 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:09.937084  7837 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:10.020653  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:10.027297  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 56514.5us
I20260709 12:24:09.809317  7842 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:09.844431  7757 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:10.037427  7868 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:24:10.039760  7757 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:10.040547  7874 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:24:10.040632  7803 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:10.031237  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.044513  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 770) took 1200 ms. Trace:
I20260709 12:24:09.968321  7788 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:09.970430  7877 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:10.034255  6921 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:10.051189  6921 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.051718  6921 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:10.035987  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:10.036497  7866 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.219s	user 0.001s	sys 0.001s
I20260709 12:24:10.012343  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.044963  5069 rpcz_store.cc:276] 0709 12:24:08.844501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.891132 (+ 46631us) service_pool.cc:224] Handling call
0709 12:24:08.910780 (+ 19648us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:08.911289 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:09.213068 (+301779us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b18003504a0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:09.514326 (+301258us) log.cc:844] Serialized 14 byte log entry
0709 12:24:09.791256 (+276930us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.824550 (+ 33294us) log.cc:844] Serialized 26 byte log entry
0709 12:24:09.824720 (+   170us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:09.824753 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.824836 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:10.044100 (+219264us) raft_consensus.cc:1685] finished
0709 12:24:10.044217 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.044426 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1560,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":2861,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":33078,"spinlock_wait_cycles":23122816,"thread_start_us":878724,"threads_started":2,"wal-append.queue_time_us":487975}
I20260709 12:24:10.079864  5150 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:24:10.077445  5190 leader_election.cc:336] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.705s (SENT)
I20260709 12:24:10.086349  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:10.086992  5945 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:10.089677  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.091634  6921 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:10.105973  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.022279  7845 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:10.116613  5190 leader_election.cc:336] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.723s (SENT)
I20260709 12:24:10.117250  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.109321  6921 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:10.121042  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.032827  7837 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:10.121834  5150 raft_consensus.cc:2376] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10. Re-sending same reply.
I20260709 12:24:10.123611  7878 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:10.130584  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.131685  5150 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 9
W20260709 12:24:10.132534  5146 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.002s	sys 0.015s
W20260709 12:24:10.133627  5146 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.002s	sys 0.015s
I20260709 12:24:10.004385  7698 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:10.128736  7770 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:10.137432  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:10.141695  5941 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:10.139147  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.145493  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.142807  5148 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:24:10.149045  5146 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:10.151515  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.161824  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.073633  7844 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:10.162768  5146 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:10.170353  7887 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:24:10.173629  7877 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:10.174276  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.180652  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.182780  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.151821  7878 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:10.177619  7884 log.cc:927] Time spent T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:24:10.190275  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.191423  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.191789  5143 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:10.191980  5146 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:10.182569  7690 consensus_meta.cc:287] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.812s	user 0.012s	sys 0.001s
I20260709 12:24:10.134845  7895 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:24:10.243113  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.121094  7712 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:10.243831  5150 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:10.193840  7878 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:10.241619  7690 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:10.263075  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:10.267288  5150 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:10.284111  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:10.294384  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 460) took 1023 ms. Trace:
I20260709 12:24:10.298600  5224 rpcz_store.cc:276] 0709 12:24:09.271043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.271190 (+   147us) service_pool.cc:224] Handling call
0709 12:24:09.292032 (+ 20842us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.292573 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:09.292616 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.293219 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 12:24:09.467998 (+174779us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b1800132dc0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db0d1 00007fd8a84f6a0f
0709 12:24:09.783468 (+315470us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:09.783528 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.783624 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:10.293999 (+510375us) raft_consensus.cc:1685] finished
0709 12:24:10.294024 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.294304 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"spinlock_wait_cycles":43613952,"thread_start_us":315240,"threads_started":1,"wal-append.queue_time_us":727721}
I20260709 12:24:10.306273  5146 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:10.309597  5945 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:24:10.309607  5150 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:10.310388  5217 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:10.313769  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:10.308040  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.317564  5146 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:10.319070  5941 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:10.320441  5189 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:10.258266  7901 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:24:10.244199  7386 consensus_meta.cc:287] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.835s	user 0.013s	sys 0.000s
W20260709 12:24:10.286773  7872 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.664s	user 0.003s	sys 0.000s
W20260709 12:24:10.292351  7818 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.914s	user 0.014s	sys 0.000s
I20260709 12:24:10.328393  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:10.312775  7894 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:24:10.337571  5190 leader_election.cc:336] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.812s (SENT)
I20260709 12:24:10.351413  5190 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.352918  6921 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:10.281065  7847 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:10.353405  6921 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.354022  6921 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:10.261446  7878 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:10.358848  5190 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.362203  5945 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:10.368237  6921 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:10.339593  7770 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:10.382510  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.382798  7770 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:10.384416  5115 leader_election.cc:400] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
I20260709 12:24:10.384789  5115 leader_election.cc:403] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:10.374356  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.383986  7770 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:10.331600  7386 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:10.391396  7848 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:10.391808  5216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:10.392135  7386 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:10.392441  5071 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:10.392573  5216 raft_consensus.cc:606] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:10.393148  5216 raft_consensus.cc:614] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:10.395601  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.399173  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:10.404896  7721 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:10.365444  7846 consensus_meta.cc:287] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.822s	user 0.009s	sys 0.005s
I20260709 12:24:10.405819  7896 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:10.373072  7878 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:10.406723  7833 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:10.410043  5216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:10.411026  5216 raft_consensus.cc:606] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.412154  5143 raft_consensus.cc:2410] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:10.406821  5071 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:10.413995  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.417480  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:10.418973  7899 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:24:10.419589  5216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5deada35125f4c008544b2ff246b039c"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:10.421953  5222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 835) took 1018 ms. Trace:
I20260709 12:24:10.341008  7818 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:10.406068  7846 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:10.423826  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.426429  5115 leader_election.cc:400] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
I20260709 12:24:10.425148  5222 rpcz_store.cc:276] 0709 12:24:09.404034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.404219 (+   185us) service_pool.cc:224] Handling call
0709 12:24:09.410644 (+  6425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.436141 (+ 25497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.436190 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.446931 (+ 10741us) log.cc:844] Serialized 26 byte log entry
0709 12:24:09.706719 (+259788us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:09.718454 (+ 11735us) log.cc:844] Serialized 14 byte log entry
0709 12:24:09.977404 (+258950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.977513 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:10.421635 (+444122us) raft_consensus.cc:1685] finished
0709 12:24:10.421670 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.421867 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24919,"prepare.queue_time_us":11190,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":120869,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":193,"spinlock_wait_cycles":1504512,"thread_start_us":471728,"threads_started":2,"wal-append.queue_time_us":488152}
I20260709 12:24:10.392210  7848 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7.
I20260709 12:24:10.431118  5190 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:10.431380  5115 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:10.412729  7584 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:10.439024  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.427385  7539 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:10.460016  7833 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:10.461715  7618 raft_consensus.cc:3055] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:10.462158  7618 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:10.465600  7432 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.466149  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.467227  7721 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.467978  7721 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:10.444293  7845 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:10.470134  7806 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:10.438622  7878 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:10.466555  7432 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:10.472435  7878 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.472949  7878 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:10.474676  7875 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.475268  7875 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:10.477334  7652 consensus_meta.cc:287] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.868s	user 0.010s	sys 0.002s
I20260709 12:24:10.477674  7386 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:10.477953  7652 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:10.477432  7821 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:10.479175  5944 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:10.479285  7821 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:10.480320  7652 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:10.480471  7821 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:10.443213  7790 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:10.483331  7790 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:10.478111  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.484333  7790 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:10.484762  7869 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:10.482945  7870 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:24:10.482816  7902 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:24:10.487993  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.494666  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.457501  5115 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:10.504074  5042 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.508263  5900 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.511066  5149 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:10.416633  7896 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:24:10.514492  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:10.455209  7906 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:10.517447  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.454129  7903 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:10.527659  5042 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.532441  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.516460  7652 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:10.478259  7386 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:10.515563  7690 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:10.525835  7847 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:10.538445  5900 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.529115  7788 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:10.543486  7619 consensus_meta.cc:287] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.302s	user 0.014s	sys 0.009s
I20260709 12:24:10.516355  7799 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:24:10.538415  7879 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.522323  7869 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.549438  6921 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:10.553740  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 96341.2us
I20260709 12:24:10.520428  7853 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:24:10.536015  7705 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.928s	user 0.008s	sys 0.007s
I20260709 12:24:10.568753  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:10.602952  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:10.545459  7432 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:10.537113  7624 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:10.546428  7721 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:10.621084  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:10.535944  7906 raft_consensus.cc:697] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:10.580691  5942 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:24:10.628706  5942 pending_rounds.cc:85] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 5
I20260709 12:24:10.629110  5942 pending_rounds.cc:107] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:24:10.594126  5216 raft_consensus.cc:606] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.629395  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:10.578177  7867 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:10.638144  5944 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:10.596163  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:10.640195  5224 raft_consensus.cc:2368] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:10.563978  7584 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:10.597627  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.593319  7788 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.610497  5943 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:24:10.612061  5940 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:10.644379  5943 pending_rounds.cc:85] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 4
I20260709 12:24:10.644658  5943 pending_rounds.cc:107] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:24:10.572600  7875 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:10.626809  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.629421  5942 raft_consensus.cc:2889] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:10.594823  5148 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:10.535877  7903 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:10.644883  5943 raft_consensus.cc:2889] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:10.648583  5947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 708) took 1369 ms (client timeout 1695 ms). Trace:
I20260709 12:24:10.603708  7853 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:10.659152  5940 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:10.628130  7705 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:10.627512  7847 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:10.661854  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:10.682679  5219 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:10.676832  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:10.666679  5947 rpcz_store.cc:269] 0709 12:24:09.279592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.284830 (+  5238us) service_pool.cc:224] Handling call
0709 12:24:10.648394 (+1363564us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400005854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:10.648518 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462318336}
W20260709 12:24:10.670936  7907 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:24:10.671677  7386 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:10.649695  7616 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
W20260709 12:24:10.674237  7911 log.cc:927] Time spent T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:24:10.630964  7432 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:10.650492  7913 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:10.686018  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 153705us
I20260709 12:24:10.686724  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:10.688079  5145 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:10.623181  7619 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:24:10.630858  7652 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:24:10.690614  5144 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:10.614388  7869 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:10.691852  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.599032  7879 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:10.676111  7853 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:10.700011  7634 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.652s	user 0.012s	sys 0.002s
I20260709 12:24:10.692536  7847 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:10.702497  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.681180  7875 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:10.662329  7788 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:10.710448  7432 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:10.719034  7879 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:10.719640  7865 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 1.221s	user 0.003s	sys 0.002s
I20260709 12:24:10.719939  7788 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:10.705768  5221 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:10.721195  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1045) took 1649 ms. Trace:
W20260709 12:24:10.721439  7905 log.cc:927] Time spent T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.315s	user 0.001s	sys 0.002s
I20260709 12:24:10.707818  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.721588  5218 rpcz_store.cc:276] 0709 12:24:09.071661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.073246 (+  1585us) service_pool.cc:224] Handling call
0709 12:24:09.073593 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.073973 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.073999 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.074510 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:24:09.379831 (+305321us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:09.379874 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.379969 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:10.718968 (+1338999us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:10.720817 (+  1849us) raft_consensus.cc:1685] finished
0709 12:24:10.720839 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.721125 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3955,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":7048,"spinlock_wait_cycles":8945024,"thread_start_us":304967,"threads_started":1,"wal-append.queue_time_us":421426}
I20260709 12:24:10.723259  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.724402  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.703066  7846 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:10.725361  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.727138  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.728992  5146 raft_consensus.cc:2376] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7. Re-sending same reply.
I20260709 12:24:10.729837  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.730477  5146 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:10.731338  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.731936  5146 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:10.732738  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.733238  5146 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:10.734786  7875 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:10.738613  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.740394  7869 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:10.741940  7845 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:10.713582  7903 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:10.699025  7818 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:10.743778  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:10.745278  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:10.745599  5900 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:10.746084  5219 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:10.747218  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:10.750517  5218 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:10.717103  5144 raft_consensus.cc:2376] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7. Re-sending same reply.
I20260709 12:24:10.705960  7703 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.699735  7913 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:10.757519  5942 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:10.744604  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:10.763804  5216 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:10.737277  7875 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:10.757035  5218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7be74bdf01fe4190992575e84da14996"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:10.775767  5218 raft_consensus.cc:606] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:10.776172  5218 raft_consensus.cc:614] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:10.778232  5146 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:10.794320  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:10.797712  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 102856us
I20260709 12:24:10.798678  5042 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.800838  5218 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:10.803435  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.805378  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:10.810766  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 126615us
I20260709 12:24:10.812075  5218 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:10.815323  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:10.773890  7903 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:10.819468  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:10.715986  7634 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:10.823424  5218 raft_consensus.cc:2410] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:10.825132  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:10.825703  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:10.826630  5068 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:10.831525  6921 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.832093  6921 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:10.834162  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:10.816613  7875 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:10.837925  5218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:10.840323  5218 raft_consensus.cc:606] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.840873  5218 raft_consensus.cc:614] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:10.772382  7539 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:24:10.848623  6921 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:10.846666  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.850607  6921 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:10.774359  7846 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:24:10.853972  5224 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:10.854907  5069 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:10.821192  7818 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:10.822567  7903 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.859738  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:10.851145  6921 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.774170  7831 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 12:24:10.858508  7770 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:10.860576  6921 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:10.710615  7906 consensus_queue.cc:237] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:10.866536  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:10.859218  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 44038.2us
I20260709 12:24:10.778905  7703 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:10.867981  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:10.865644  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:10.868896  5070 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:10.828011  7896 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:24:10.868244  5074 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:10.867763  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:10.869827  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:10.874122  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:10.880204  6921 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:10.861505  7903 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:10.890762  7842 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:10.891414  7842 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:10.884310  7913 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:10.898300  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:10.904649  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:10.905421  5941 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:10.898370  5076 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:10.907025  5221 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:10.898874  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:10.860261  7818 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:10.898885  5075 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:10.908619  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:10.908857  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:10.902597  5942 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:10.909425  5072 raft_consensus.cc:2393] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:10.899266  5074 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:10.910122  5069 raft_consensus.cc:2410] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:24:10.910873  5946 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:10.917526  5070 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:10.921380  6921 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:10.921447  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:10.922777  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 91800.6us
W20260709 12:24:10.923688  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 180914us
I20260709 12:24:10.925127  7698 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.925714  7698 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:10.928583  7698 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:10.931068  5941 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:24:10.932622  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.933365  5148 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:10.934197  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.934283  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.934746  5148 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:10.935524  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.935138  7618 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:10.936579  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:10.936825  7618 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:10.937816  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:10.939137  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:10.939805  5148 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:10.940631  5147 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:10.942106  5142 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:10.946453  5222 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:10.948371  5218 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:10.949815  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:10.950762  5216 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
W20260709 12:24:10.955631  5071 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 1010) took 1613 ms (client timeout 1947 ms). Trace:
W20260709 12:24:10.956271  5071 rpcz_store.cc:269] 0709 12:24:09.342629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.344947 (+  2318us) service_pool.cc:224] Handling call
0709 12:24:10.955387 (+1610440us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400429354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:10.955549 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2334642048}
W20260709 12:24:10.959760  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 89608.1us
I20260709 12:24:10.967859  5074 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:10.968238  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:10.969012  5076 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:24:10.897681  7619 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:10.979875  5900 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.970667  5188 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:10.970288  5070 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:24:10.972388  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 48019.4us
I20260709 12:24:10.982630  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:10.973124  7789 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:10.983125  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:10.982913  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.974095  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:10.976796  7837 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:10.977807  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:10.983485  7789 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:10.985781  5042 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:10.986672  7698 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:10.987255  7789 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:10.974185  7923 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:10.987396  7698 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:10.988946  7698 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.989425  7698 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:10.989043  7689 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:10.983018  5041 leader_election.cc:400] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:10.994709  5041 leader_election.cc:403] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:24:10.995527  7924 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:10.996580  5041 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:10.998373  7924 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:11.000689  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:10.978500  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:11.005447  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:11.006917  5943 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:10.903667  7842 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:11.014801  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:11.019114  5076 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:10.908654  7818 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:10.853178  7847 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:11.050603  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 80023us
I20260709 12:24:11.057688  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:11.059160  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 49931.8us
I20260709 12:24:11.063089  5943 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:11.065690  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:11.053025  7789 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:11.072594  7867 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:11.072674  7837 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:24:11.074276  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 79529.4us
I20260709 12:24:11.075290  7837 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:11.076030  7837 consensus_queue.cc:1243] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:24:11.017552  7870 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:11.001793  7849 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:24:11.078423  7539 raft_consensus.cc:3055] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:11.016072  7924 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.065s
W20260709 12:24:11.096606  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 118278us
I20260709 12:24:11.006912  7619 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:11.079526  5898 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:11.098052  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:11.017298  7853 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:11.052246  7799 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:24:11.109607  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:11.111203  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 55045.4us
I20260709 12:24:11.072216  7845 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:24:11.111459  5945 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 5.
I20260709 12:24:11.062003  7883 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:24:11.092907  7539 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:11.096714  7850 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:11.134123  7806 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:11.134899  7703 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:11.138187  7929 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.147619  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:11.089634  7867 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:24:11.090197  7936 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 12:24:11.154439  5067 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:11.106817  7789 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:11.114208  7849 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:11.169718  5076 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:11.165910  5070 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:11.161365  7845 consensus_queue.cc:1243] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:24:11.171908  7892 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.764s	user 0.006s	sys 0.000s
I20260709 12:24:11.172118  7846 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:11.172588  7846 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:11.173666  7849 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.174197  7849 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:11.173427  7846 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:11.177152  7846 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:11.178164  7789 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:11.173525  7539 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:11.116662  7870 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 12:24:11.183113  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 57234.5us
W20260709 12:24:11.223567  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 220462us
I20260709 12:24:11.158120  7806 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:11.243042  7619 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:11.170279  7929 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:11.181332  7853 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:11.168684  7634 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:11.247372  7929 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:11.249692  7931 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.235s	user 0.000s	sys 0.007s
W20260709 12:24:11.251055  7878 consensus_meta.cc:287] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.778s	user 0.012s	sys 0.003s
I20260709 12:24:11.251540  7878 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:11.254338  5219 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 12:24:11.254962  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 68075.8us
I20260709 12:24:11.187328  7703 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.255847  7703 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:11.259096  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 940) took 1325 ms. Trace:
I20260709 12:24:11.259840  7913 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:11.261658  7719 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:11.263661  5141 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:11.263370  7698 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:11.268247  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:11.259614  5145 rpcz_store.cc:276] 0709 12:24:09.933548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.964591 (+ 31043us) service_pool.cc:224] Handling call
0709 12:24:11.258923 (+1294332us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b540077fbd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:11.259033 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1539062528}
I20260709 12:24:11.272581  5947 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:11.268438  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:11.254566  7903 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:11.277510  7903 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:11.269639  5217 raft_consensus.cc:2410] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:11.303238  7833 raft_consensus.cc:3055] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:11.303949  7833 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:11.308928  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 227421us
I20260709 12:24:11.310539  5041 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:11.311903  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:11.314509  7821 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:11.316037  7821 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:11.316304  7842 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:11.316720  7821 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:11.317098  7842 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:11.318017  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:11.308698  7624 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:11.318574  7842 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:11.320843  7842 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:11.321990  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.322742  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:11.324200  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:11.266696  7806 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:11.327392  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 483) took 1023 ms. Trace:
I20260709 12:24:11.331604  5150 rpcz_store.cc:276] 0709 12:24:10.303553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.310805 (+  7252us) service_pool.cc:224] Handling call
0709 12:24:10.397745 (+ 86940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.397935 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.397962 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.398019 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:10.398052 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.398112 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.327209 (+929097us) spinlock_profiling.cc:243] Waited 86.5 ms on lock 0x7b540045ae50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:11.327329 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198981376}
I20260709 12:24:11.335808  5150 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: [7.4-8.5]
I20260709 12:24:11.336903  5148 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:11.338393  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:11.339166  5945 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:11.342410  5150 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:11.343192  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:11.343894  5142 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:11.353505  5150 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:11.321655  7624 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:11.367543  5900 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:11.342850  7946 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.373562  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:11.347072  7924 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:11.391067  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:11.355347  7879 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:11.305119  7833 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:11.375674  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:11.349053  7878 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:11.361254  7698 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:11.410261  7789 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:11.405704  7712 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:11.371759  7870 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:24:11.369243  7843 consensus_queue.cc:1059] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 12:24:11.388201  7925 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:11.374739  7867 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:11.413328  7703 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:11.417729  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 93764us
I20260709 12:24:11.437343  7789 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.480381  5075 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:11.404489  7946 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:11.520555  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:11.409456  7924 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:11.522063  5188 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:11.420727  7848 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
W20260709 12:24:11.561517  7949 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 12:24:11.564500  7618 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.627s	user 0.016s	sys 0.000s
I20260709 12:24:11.540911  7906 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.662151  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:11.597968  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.526036  7789 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:11.659682  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:11.540441  7925 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:11.541260  7712 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:11.765000  7913 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.782308  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:11.579303  7883 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:11.571142  7896 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:11.818958  7913 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:11.569322  7867 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:11.646812  7843 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:11.886196  5149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1013) took 2330 ms (client timeout 2031 ms). Trace:
W20260709 12:24:11.890661  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 110489us
I20260709 12:24:11.755224  7652 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:11.900192  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 117980us
W20260709 12:24:11.925324  5149 rpcz_store.cc:269] 0709 12:24:09.555684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.555894 (+   210us) service_pool.cc:224] Handling call
0709 12:24:11.885866 (+2329972us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54001b6554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:11.886098 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2044170496}
I20260709 12:24:11.930684  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:11.553387  7849 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:11.886966  7896 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.878355  7883 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:11.969792  7652 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:11.929397  7703 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:12.005175  7959 log.cc:927] Time spent T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.332s	user 0.001s	sys 0.001s
I20260709 12:24:12.009052  7712 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:11.589367  7818 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:12.031057  7925 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.032013  7962 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:11.766841  7906 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:12.036326  7896 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:12.097226  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.557050  7432 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:12.117995  7867 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:11.941576  7843 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.068877  7694 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:12.136039  7961 log.cc:927] Time spent T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:24:12.138361  7906 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:12.141943  7913 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:12.142926  7957 log.cc:927] Time spent T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:24:12.143457  7924 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:12.119962  7883 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.145009  7883 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:12.146569  7956 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.604s	user 0.000s	sys 0.001s
I20260709 12:24:12.146848  7849 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:12.157326  7946 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:12.181243  7950 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.650s	user 0.002s	sys 0.000s
W20260709 12:24:12.194314  7965 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.268s	user 0.000s	sys 0.002s
W20260709 12:24:12.204614  7964 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:24:11.909711  5040 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.009s	sys 0.028s
W20260709 12:24:12.238164  5040 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.009s	sys 0.029s
I20260709 12:24:12.243551  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.245674  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.246783  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:12.250090  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.250952  5072 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:24:12.254768  5070 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: [6.4-7.5]
I20260709 12:24:12.257771  5073 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 6: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:12.305701  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 859) took 1021 ms. Trace:
I20260709 12:24:12.309247  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 548) took 1029 ms. Trace:
I20260709 12:24:12.309701  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 955) took 1044 ms. Trace:
I20260709 12:24:12.309836  5947 rpcz_store.cc:276] 0709 12:24:11.279621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.280288 (+   667us) service_pool.cc:224] Handling call
0709 12:24:11.280849 (+   561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.289254 (+  8405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.289307 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:11.295356 (+  6049us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.401672 (+106316us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:11.401715 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.401869 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.143141 (+741272us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b24003f7508. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a52d49c4 00007fd8a52d2acb 00007fd8a548ddc9 00007fd8a548da55 00007fd8a548cfb0 00007fd8a8103488 00007fd8a8103be8 00007fd8a80db398 00007fd8a80d676a 00007fd8a80d5bb6
0709 12:24:12.308462 (+165321us) raft_consensus.cc:1685] finished
0709 12:24:12.308547 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.308956 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7841,"raft.queue_time_us":8450,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":33100,"spinlock_wait_cycles":31309568,"thread_start_us":44643,"threads_started":1,"wal-append.queue_time_us":128010}
I20260709 12:24:12.310520  5141 rpcz_store.cc:276] 0709 12:24:11.264796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.264904 (+   108us) service_pool.cc:224] Handling call
0709 12:24:11.265288 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.265602 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.265630 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:11.266134 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.396927 (+130793us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:11.396986 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.397100 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.308749 (+911649us) raft_consensus.cc:1685] finished
0709 12:24:12.308806 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.309261 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":130382,"threads_started":1,"wal-append.queue_time_us":306457}
I20260709 12:24:12.314426  7694 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:12.318531  5143 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.929s	user 0.084s	sys 0.216s
W20260709 12:24:12.319317  5143 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.084s	sys 0.216s
W20260709 12:24:12.325505  5898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.001s	sys 0.015s
W20260709 12:24:12.326341  5898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.001s	sys 0.016s
W20260709 12:24:12.331429  7806 consensus_meta.cc:287] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.986s	user 0.012s	sys 0.000s
W20260709 12:24:12.332867  7880 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 1.839s	user 0.009s	sys 0.006s
I20260709 12:24:12.333405  7619 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:12.334895  7619 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.332389  7806 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.076000  7652 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:12.306491  5217 rpcz_store.cc:276] 0709 12:24:11.283588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.286021 (+  2433us) service_pool.cc:224] Handling call
0709 12:24:11.286400 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.288175 (+  1775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.288213 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:11.295708 (+  7495us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.728588 (+432880us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:11.728654 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.728769 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.302797 (+574028us) raft_consensus.cc:1685] finished
0709 12:24:12.302847 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.305344 (+  2497us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3248,"raft.queue_time_us":7213,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":385033,"threads_started":1,"wal-append.queue_time_us":756372}
I20260709 12:24:12.356855  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.313216  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 856) took 1045 ms. Trace:
I20260709 12:24:12.366084  5221 rpcz_store.cc:276] 0709 12:24:11.267190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.267937 (+   747us) service_pool.cc:224] Handling call
0709 12:24:12.311683 (+1043746us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b580013ef98. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:24:12.312182 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.312683 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.312709 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:12.312773 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:12.312825 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.312897 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.313053 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4410624}
I20260709 12:24:12.118808  7432 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:12.379338  7432 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:12.380630  7818 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:12.417797  5070 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:12.424181  7619 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:12.387703  7694 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:12.437013  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 506226us
I20260709 12:24:12.430763  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 489) took 1691 ms. Trace:
I20260709 12:24:12.433914  7539 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:12.433768  7975 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:12.417778  7970 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:12.440889  5145 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:12.430644  5946 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:12.447392  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.447423  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.427883  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:12.456481  7806 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:12.479717  5075 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:12.475653  5076 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:12.482101  5944 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:12.493806  5067 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:12.493824  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1026) took 1360 ms. Trace:
I20260709 12:24:12.498606  5190 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:12.496553  7843 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:12.496501  7977 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:12.501356  7977 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.501780  7803 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:12.498085  5074 rpcz_store.cc:276] 0709 12:24:11.112647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.112947 (+   300us) service_pool.cc:224] Handling call
0709 12:24:11.113395 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.313150 (+199755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.313184 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:11.313823 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.374425 (+ 60602us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:11.374465 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.374583 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.393412 (+1018829us) raft_consensus.cc:1685] finished
0709 12:24:12.401590 (+  8178us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.472705 (+ 71115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199238,"raft.queue_time_us":206380,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":286,"thread_start_us":60214,"threads_started":1,"wal-append.queue_time_us":236924}
I20260709 12:24:12.503078  7803 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.504390  7977 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:12.505816  7585 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:12.506853  7585 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.441946  5146 rpcz_store.cc:276] 0709 12:24:10.740076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.795983 (+ 55907us) service_pool.cc:224] Handling call
0709 12:24:10.796318 (+   335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:10.852627 (+ 56309us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:11.094142 (+241515us) log.cc:844] Serialized 14 byte log entry
0709 12:24:11.253162 (+159020us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:11.253876 (+   714us) log.cc:844] Serialized 52 byte log entry
0709 12:24:11.254037 (+   161us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:11.254271 (+   234us) log.cc:844] Serialized 14 byte log entry
0709 12:24:11.254465 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.254541 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.358525 (+1103984us) raft_consensus.cc:1685] finished
0709 12:24:12.362517 (+  3992us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.430483 (+ 67966us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19335,"prepare.queue_time_us":142041,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"raft.queue_time_us":124904,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":256,"thread_start_us":395575,"threads_started":3,"wal-append.queue_time_us":277286}
I20260709 12:24:12.520737  7975 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.521631  7386 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.522498  7386 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:12.522856  7970 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:12.524153  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 649) took 1548 ms. Trace:
I20260709 12:24:12.524564  5940 rpcz_store.cc:276] 0709 12:24:10.975223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.978711 (+  3488us) service_pool.cc:224] Handling call
0709 12:24:10.979113 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.124683 (+145570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.124731 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:11.124791 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:11.124836 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.523615 (+1398779us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b540000a354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:12.523698 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.523934 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334193408}
I20260709 12:24:12.532471  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:12.532800  7837 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.533887  7837 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.535559  7803 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:12.536456  7803 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:12.536705  7837 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:12.536499  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.525287  7386 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:12.539085  5074 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:12.536795  7962 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:12.527354  7975 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:12.547008  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:12.548602  7962 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:12.549250  7962 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.549734  7962 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:12.554157  7689 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.555110  7689 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.566414  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.563757  7982 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:12.575031  7705 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:12.575673  7705 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:12.557763  7970 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:12.581527  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 845) took 1929 ms. Trace:
I20260709 12:24:12.574901  7982 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.584527  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.582849  5224 rpcz_store.cc:276] 0709 12:24:10.651271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.657100 (+  5829us) service_pool.cc:224] Handling call
0709 12:24:10.657467 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.859958 (+202491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.579698 (+1719740us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54001c0cd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:12.580261 (+   563us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440774528}
I20260709 12:24:12.571554  7585 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:12.592374  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.570976  7803 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.586079  7975 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:24:12.608906  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 490) took 1274 ms. Trace:
I20260709 12:24:12.609618  5145 rpcz_store.cc:276] 0709 12:24:11.334438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.337434 (+  2996us) service_pool.cc:224] Handling call
0709 12:24:11.358614 (+ 21180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.489768 (+1131154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.608449 (+118681us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400238f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:12.608810 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2332227456}
I20260709 12:24:12.610757  7833 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:12.626971  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 789) took 1762 ms. Trace:
I20260709 12:24:12.627281  5069 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:12.628377  5073 rpcz_store.cc:276] 0709 12:24:10.864665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.866300 (+  1635us) service_pool.cc:224] Handling call
0709 12:24:12.626512 (+1760212us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400328054. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:12.626846 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3176405760}
I20260709 12:24:12.653445  7694 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:12.655012  7694 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.660740  7941 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.661360  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 554) took 1765 ms. Trace:
I20260709 12:24:12.666676  7689 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:12.665747  5069 rpcz_store.cc:276] 0709 12:24:10.895807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.910620 (+ 14813us) service_pool.cc:224] Handling call
0709 12:24:12.626866 (+1716246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.628593 (+  1727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.661054 (+ 32461us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b5400328050. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:12.661284 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3946041984}
W20260709 12:24:12.670403  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.678056  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:12.682557  7941 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.686194  5940 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:12.686728  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:12.689088  7842 consensus_meta.cc:287] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.367s	user 0.024s	sys 0.005s
I20260709 12:24:12.691507  7933 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.692298  7933 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.693638  7867 consensus_queue.cc:1059] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:12.694522  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:12.694800  7619 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:12.696246  7619 raft_consensus.cc:3055] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:12.696733  7619 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:12.697134  7539 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:12.697687  7652 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:12.698211  7652 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:12.699462  7619 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:12.691561  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 470) took 1920 ms. Trace:
I20260709 12:24:12.700891  5219 rpcz_store.cc:276] 0709 12:24:10.770595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.773871 (+  3276us) service_pool.cc:224] Handling call
0709 12:24:10.774248 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.254800 (+480552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.690830 (+1436030us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b54003c8f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:12.691249 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1103317760}
I20260709 12:24:12.702450  7903 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.699504  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:12.703282  7903 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.703761  5947 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:12.683666  7694 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:12.705617  7982 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:12.706166  7903 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:12.706375  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 785) took 1844 ms. Trace:
I20260709 12:24:12.699759  7652 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.706933  5068 rpcz_store.cc:276] 0709 12:24:10.862128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.862386 (+   258us) service_pool.cc:224] Handling call
0709 12:24:10.862705 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.520983 (+1658278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.521054 (+    71us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:12.521130 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:12.521192 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.705937 (+184745us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b540058a554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:12.705995 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.706170 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3802710272}
I20260709 12:24:12.708138  7652 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:12.699846  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:12.695199  7933 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:12.711323  5114 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:12.711691  7803 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:12.712872  7941 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:10.905148  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:12.732522  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 659) took 1072 ms. Trace:
I20260709 12:24:12.732983  5942 rpcz_store.cc:276] 0709 12:24:11.659898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.753645 (+ 93747us) service_pool.cc:224] Handling call
0709 12:24:11.754190 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.075003 (+320813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.075048 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.075675 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 12:24:12.568864 (+493189us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:12.570373 (+  1509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.731722 (+161349us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400007154. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:12.731797 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.731906 (+   109us) raft_consensus.cc:1685] finished
0709 12:24:12.731926 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.732301 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":178139,"raft.queue_time_us":49671,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":4896,"spinlock_wait_cycles":736482560,"thread_start_us":455126,"threads_started":1,"wal-append.queue_time_us":580322}
I20260709 12:24:12.736320  7618 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:12.736791  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 190156us
I20260709 12:24:12.737305  7843 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:12.737885  7843 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:12.737720  7933 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:12.737098  7618 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.739176  7618 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:12.740031  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.742851  7844 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:12.743222  7386 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:12.744295  7386 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.739192  7843 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:12.746748  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 83406.7us
I20260709 12:24:12.747124  7843 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:12.747305  7789 consensus_meta.cc:287] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.962s	user 0.013s	sys 0.008s
I20260709 12:24:12.747874  7789 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.748421  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 660) took 1088 ms. Trace:
I20260709 12:24:12.577414  7705 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.748903  5941 rpcz_store.cc:276] 0709 12:24:11.660272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.662923 (+  2651us) service_pool.cc:224] Handling call
0709 12:24:11.663285 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.446889 (+783604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.446942 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:12.447002 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:12.447055 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.748022 (+300967us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b5400006254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:12.748077 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.748251 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438627712}
I20260709 12:24:12.746938  7788 consensus_queue.cc:1059] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:12.751951  7788 consensus_queue.cc:1243] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:12.763321  7976 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:12.764351  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.764248  7976 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.764806  7703 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:12.765720  7845 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.766381  7845 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.767419  7690 raft_consensus.cc:3055] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:12.767669  7977 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:12.768455  7705 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:12.768381  7977 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:12.769912  6921 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:12.771046  7976 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:12.771756  6921 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:12.765506  7703 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:12.774631  7703 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:12.767918  7690 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:12.776618  7690 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:12.778398  7690 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:12.778659  7883 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.633s	user 0.010s	sys 0.018s
W20260709 12:24:12.781450  7896 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.653s	user 0.013s	sys 0.010s
I20260709 12:24:12.781719  7933 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:12.772008  7444 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:12.781734  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 43653.3us
W20260709 12:24:12.781912  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 92378.1us
I20260709 12:24:12.701292  7619 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:12.781528  7831 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.800036  7585 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:12.803490  7840 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:12.782744  7842 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.781944  7616 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.782177  7906 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:12.797314  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:12.803812  7989 log.cc:927] Time spent T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
W20260709 12:24:12.778545  5899 connection.cc:256] RPC call timeout handler was delayed by -1.4724827680001908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1773s for a response
W20260709 12:24:12.820765  5899 connection.cc:256] RPC call timeout handler was delayed by -1.4544827680001617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 12:24:12.821173  5899 connection.cc:256] RPC call timeout handler was delayed by -1.2294827680002527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
W20260709 12:24:12.821501  5899 connection.cc:256] RPC call timeout handler was delayed by -1.1799827680001727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
I20260709 12:24:12.709163  7962 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.803738  7896 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:12.803731  7585 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.816761  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:12.816929  7788 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.806778  7933 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:12.817595  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.817984  7883 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:12.817736  7616 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:12.816605  7840 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:12.818905  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:12.796934  7982 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.820199  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:12.836002  5114 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:12.840252  5190 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:12.842881  5150 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: [7.4-8.5]
I20260709 12:24:12.843081  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:12.843765  7788 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.844303  7977 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:12.844907  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 59505.6us
I20260709 12:24:12.845350  5944 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:12.845695  7896 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:12.847136  7933 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:12.849318  7883 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:12.831390  7906 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:12.849339  5899 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:12.851598  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:12.827594  7444 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:12.852802  5216 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:12.853639  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 492) took 1518 ms. Trace:
I20260709 12:24:12.858866  5148 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:12.875232  5142 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:12.862972  7842 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.876695  7585 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:12.877652  7906 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:12.864223  5141 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:12.861164  7789 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:12.890463  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.831951  7831 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:12.897212  5215 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:12.824676  7703 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:24:12.902652  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:12.902920  7585 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:12.906879  7616 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:12.860553  7941 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:12.880151  7539 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:12.918068  5149 raft_consensus.cc:2410] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:12.879458  5143 rpcz_store.cc:276] 0709 12:24:11.335147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.335272 (+   125us) service_pool.cc:224] Handling call
0709 12:24:11.356053 (+ 20781us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.320514 (+964461us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:12.320595 (+    81us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.358985 (+ 38390us) log.cc:844] Serialized 26 byte log entry
0709 12:24:12.442563 (+ 83578us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:12.442637 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.477650 (+ 35013us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.853383 (+375733us) raft_consensus.cc:1685] finished
0709 12:24:12.853404 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.853580 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49617,"raft.queue_time_us":35805,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":3751,"thread_start_us":996063,"threads_started":2,"wal-append.queue_time_us":200098}
W20260709 12:24:12.924515  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 185103us
I20260709 12:24:12.924993  7840 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:12.925307  7924 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.929742  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:12.930302  7386 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:12.930673  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.932924  7432 consensus_meta.cc:287] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.552s	user 0.015s	sys 0.005s
I20260709 12:24:12.917912  7906 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:12.934309  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 122700us
I20260709 12:24:12.935268  5218 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:24:12.938390  7970 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:12.938273  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:12.909513  7975 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:12.939046  7585 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:12.939141  7444 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:12.941056  7788 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:12.890350  7971 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:12.947178  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:12.955559  5150 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:12.952708  5218 pending_rounds.cc:85] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 4
I20260709 12:24:12.952267  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.959808  5144 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:12.970141  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:12.964515  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:12.965119  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.900292  7982 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:12.975009  5218 pending_rounds.cc:107] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:24:12.998998  5218 raft_consensus.cc:2889] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:12.977849  5068 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:24:12.977983  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:12.975229  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:12.979296  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 49610.6us
I20260709 12:24:12.980726  7831 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:12.982084  7842 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:12.962157  7970 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:12.986549  7883 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:12.988845  7846 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.011922  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:12.990355  7618 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:12.964823  7845 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:12.991078  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:12.991375  7971 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:12.992029  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:12.975467  7386 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:24:12.968294  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:12.998005  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:13.003669  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 154513us
I20260709 12:24:13.043761  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:12.960409  7432 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:12.990793  5146 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:13.014334  5946 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:13.010836  5142 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
W20260709 12:24:13.019392  5189 leader_election.cc:336] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Timed out: RequestConsensusVote RPC to 127.1.123.65:37253 timed out after 1.695s (SENT)
I20260709 12:24:13.050184  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 494) took 1543 ms. Trace:
I20260709 12:24:13.015162  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.060421  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:13.062517  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.061524  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.065479  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:13.067222  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 68447.6us
I20260709 12:24:13.002709  7941 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:13.077454  5150 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:12.967900  7585 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:13.083575  5067 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:13.086520  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.089488  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.090903  7840 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.093801  7994 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:13.094913  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.112958  7618 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:13.113005  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.114037  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.115739  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.119302  7432 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:13.119738  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.064823  7903 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:13.121443  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 61455.2us
I20260709 12:24:13.124259  5149 raft_consensus.cc:2410] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 12:24:13.107005  5215 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:13.125919  5899 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:13.001689  7924 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:13.056942  5147 rpcz_store.cc:276] 0709 12:24:11.335820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.337559 (+  1739us) service_pool.cc:224] Handling call
0709 12:24:11.355321 (+ 17762us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.355760 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.355797 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.592793 (+1236996us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b2c00193f58. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:24:12.593735 (+   942us) log.cc:844] Serialized 26 byte log entry
0709 12:24:12.760565 (+166830us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:12.760633 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.793738 (+ 33105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.859001 (+ 65263us) raft_consensus.cc:1685] finished
0709 12:24:12.859058 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.878920 (+ 19862us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9730,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":3853,"spinlock_wait_cycles":13809920,"thread_start_us":160533,"threads_started":1,"wal-append.queue_time_us":237903}
I20260709 12:24:13.136881  7843 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:13.140355  7843 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:13.142025  7970 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:13.142653  7843 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:13.144281  7967 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:13.131429  7867 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.146018  7867 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:13.129609  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:13.077674  7845 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:13.137177  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.138302  5223 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:13.078876  7846 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:13.158041  7941 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:13.158452  7846 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:13.158957  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 139617us
I20260709 12:24:13.160406  7975 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:13.163178  5189 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.65:37253 timed out after 2.061s (SENT) [suppressed 29 similar messages]
I20260709 12:24:13.165697  7840 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:13.121515  7982 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:13.178120  5148 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:24:13.130429  7705 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:13.181860  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 116501us
I20260709 12:24:13.182595  5148 pending_rounds.cc:85] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 3
I20260709 12:24:13.183220  5148 pending_rounds.cc:107] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:24:13.183465  5148 raft_consensus.cc:2889] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:13.149241  7867 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:13.096728  7962 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:13.190413  5143 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 12:24:13.217789  5143 pending_rounds.cc:85] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 3
I20260709 12:24:13.218084  5143 pending_rounds.cc:107] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 12:24:13.218305  5143 raft_consensus.cc:2889] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:13.192474  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:13.193323  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.212080  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.122238  7903 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:13.213032  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:13.200516  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.195075  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:13.224401  5946 raft_consensus.cc:2410] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:13.229565  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.229305  5150 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:13.229699  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.235524  5147 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:13.223534  5942 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:13.154945  7845 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.180716  7690 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:13.191025  7975 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:13.252398  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:13.252995  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:13.260941  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:13.266286  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:13.271988  5946 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:13.274050  5947 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:13.275573  5948 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:13.278633  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:13.279624  5939 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:13.279989  4897 catalog_manager.cc:5697] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 5 to 6, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 6 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:24:13.281209  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.282131  4897 catalog_manager.cc:5697] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 6, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 6 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:24:13.283341  5944 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:13.283556  4897 catalog_manager.cc:5697] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 7, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 7 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:13.284471  7975 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:13.285702  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:13.286546  5945 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
W20260709 12:24:13.299403  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 216075us
I20260709 12:24:13.300715  5899 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:13.309809  7971 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:13.311600  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52302.3us
I20260709 12:24:13.314376  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.315270  5071 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:13.316035  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.316556  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.316937  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:13.317481  5072 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:24:13.318061  5070 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:13.309144  5939 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:13.335665  7698 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:13.336480  7698 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:13.193585  7883 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.148s
W20260709 12:24:13.339773  4898 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49944 (request call id 77) took 24835 ms (client timeout 15000 ms). Trace:
I20260709 12:24:13.136651  7994 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:13.349359  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:13.350649  5939 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:13.234158  7840 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:13.370512  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 100462us
I20260709 12:24:13.254192  7690 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.190107  7705 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:13.378986  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:13.380278  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.398454  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.400663  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:13.401284  5945 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:13.407084  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:13.412658  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 119255us
I20260709 12:24:13.285297  4897 catalog_manager.cc:5697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 6 to 7, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 7 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:24:13.357847  7975 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:13.415576  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.406433  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.424160  5070 raft_consensus.cc:2410] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:13.408097  5075 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:24:13.400197  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:13.425773  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:13.258567  7845 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:13.439831  5946 raft_consensus.cc:2368] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:13.441609  5068 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:24:13.442092  5068 pending_rounds.cc:85] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 4
I20260709 12:24:13.442322  5068 pending_rounds.cc:107] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:24:13.440934  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.442528  5068 raft_consensus.cc:2889] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:13.398211  7690 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:13.440016  5188 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:13.379928  7840 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:13.447620  5067 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
W20260709 12:24:13.455870  5190 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.768s (SENT)
I20260709 12:24:13.453667  5940 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
W20260709 12:24:13.471797  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 171250us
I20260709 12:24:13.472205  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:13.442827  5900 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:13.455286  7941 consensus_queue.cc:1059] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:24:13.498737  7933 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.499408  7933 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:13.492892  7803 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:13.502947  7803 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:13.503415  7803 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:13.506477  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.509079  7840 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:13.510931  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.511991  7840 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:13.513150  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.514289  7933 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:13.515302  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:13.515995  5074 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:13.520936  7705 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:13.525091  5070 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:13.527210  5070 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:13.529875  7789 consensus_queue.cc:1059] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:13.534114  7903 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:13.535643  7924 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:13.537757  7962 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:13.538638  7962 raft_consensus.cc:3055] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:13.539054  7962 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:13.540355  5074 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:13.540120  7962 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:13.542454  7803 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:13.545893  7652 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:13.546528  7652 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:13.547633  7652 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:13.548918  7652 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:13.559723  5899 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:13.566656  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 156185us
I20260709 12:24:13.587440  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:13.597635  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 101724us
I20260709 12:24:13.591068  4897 catalog_manager.cc:5697] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:24:13.631444  7803 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:13.648640  7994 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:13.649472  7844 consensus_queue.cc:1059] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:13.589205  7975 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:13.634413  4897 catalog_manager.cc:5697] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 4 to 5, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
W20260709 12:24:13.657253  5219 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.015s	sys 0.022s
I20260709 12:24:13.660005  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:13.603961  7842 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:13.662021  5219 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.016s	sys 0.022s
I20260709 12:24:13.616861  7903 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:13.689968  5189 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.014s	sys 0.048s
W20260709 12:24:13.696247  5189 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.014s	sys 0.048s
I20260709 12:24:13.623131  7883 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:13.696719  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 533543us
W20260709 12:24:13.697598  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 138102us
I20260709 12:24:13.701612  5072 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
W20260709 12:24:13.654808  7619 consensus_meta.cc:287] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.850s	user 0.012s	sys 0.002s
I20260709 12:24:13.707732  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.708688  5150 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:13.632550  7933 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:24:13.651316  7616 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.031s	sys 0.056s
I20260709 12:24:13.605382  7705 consensus_queue.cc:1059] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 12:24:13.718825  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.724834  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.732095  5148 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:13.734146  5146 raft_consensus.cc:2368] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10.
I20260709 12:24:13.726231  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:13.662132  7976 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.011s	sys 0.025s
I20260709 12:24:13.735491  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:13.340390  4898 rpcz_store.cc:269] 0709 12:23:48.503891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.504248 (+   357us) service_pool.cc:224] Handling call
0709 12:24:13.339427 (+24835179us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:09.447673 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:09.485029 (+ 37356us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:09.485083 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:24:09.485103 (+    20us) tablet.cc:662] Decoding operations
0709 12:24:09.486069 (+   966us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:09.486173 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:09.486208 (+    35us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:24:09.486887 (+   679us) tablet.cc:701] Row locks acquired
0709 12:24:09.486943 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:24:09.487134 (+   191us) write_op.cc:270] Start()
0709 12:24:09.487313 (+   179us) write_op.cc:276] Timestamp: P: 1783599849487060 usec, L: 0
0709 12:24:09.487334 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:24:09.523604 (+ 36270us) log.cc:844] Serialized 2463 byte log entry
0709 12:24:10.431568 (+907964us) op_driver.cc:464] REPLICATION: finished
0709 12:24:10.710767 (+279199us) write_op.cc:301] APPLY: starting
0709 12:24:10.713220 (+  2453us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:10.734572 (+ 21352us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:10.734754 (+   182us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:10.735638 (+   884us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:10.735890 (+   252us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:24:10.735950 (+    60us) write_op.cc:312] APPLY: finished
0709 12:24:10.736299 (+   349us) log.cc:844] Serialized 52 byte log entry
0709 12:24:10.736497 (+   198us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:10.736622 (+   125us) write_op.cc:454] Released schema lock
0709 12:24:10.736672 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:10.770249 (+ 33577us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:10.775347 (+  5098us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3249520,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":262998,"apply.run_cpu_time_us":3026,"apply.run_wall_time_us":33792,"num_ops":6,"prepare.queue_time_us":507270,"prepare.run_cpu_time_us":43591,"prepare.run_wall_time_us":441720,"raft.queue_time_us":361182,"raft.run_cpu_time_us":72704,"raft.run_wall_time_us":243736,"replication_time_us":942050,"thread_start_us":689610,"threads_started":4,"wal-append.queue_time_us":514454,"wal-append.run_cpu_time_us":16332,"wal-append.run_wall_time_us":1853038}]]}
I20260709 12:24:13.729951  4897 catalog_manager.cc:5697] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 4, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 4 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:13.657603  7975 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:13.760131  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.765659  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:13.729523  7616 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.032s	sys 0.056s
W20260709 12:24:13.757067  7976 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.012s	sys 0.025s
I20260709 12:24:13.784085  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.783313  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.785446  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.786870  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.790709  7698 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:13.791777  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.792630  5141 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:13.793859  5899 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:13.757184  7975 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:13.793967  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:13.798331  5141 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:13.759241  7982 raft_consensus.cc:3055] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:13.798902  7982 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:13.766389  7933 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:13.800180  7982 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:13.800812  7933 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:13.802363  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.803898  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:13.806149  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.807611  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.806934  4897 catalog_manager.cc:5697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 7, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 7 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:13.808322  5147 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:13.808990  4897 catalog_manager.cc:5697] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 7, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 7 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:13.783685  7903 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:13.716459  7883 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:13.715783  7842 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:13.798456  7619 raft_consensus.cc:3055] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:13.809501  5147 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:13.830950  5945 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:13.811682  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:13.833515  5074 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:24:13.812969  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.813498  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.814705  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:13.837345  5218 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:13.813339  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.814138  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:13.835345  7705 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:13.841779  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:13.843115  7999 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.275s	user 0.002s	sys 0.002s
I20260709 12:24:13.820418  5067 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:13.815973  5141 raft_consensus.cc:2410] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:13.822286  5146 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:13.823200  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.812369  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.866016  5075 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:13.836688  5069 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:13.810904  4897 catalog_manager.cc:5697] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:13.877625  5146 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:13.843950  7982 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:13.894271  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.895087  5067 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:13.899292  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:13.818087  7903 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:13.820319  7883 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:13.824469  7619 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:13.917447  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1058) took 1039 ms. Trace:
I20260709 12:24:13.918212  5144 rpcz_store.cc:276] 0709 12:24:12.877830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.087298 (+209468us) service_pool.cc:224] Handling call
0709 12:24:13.111417 (+ 24119us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.138021 (+ 26604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.138073 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.183737 (+ 45664us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.713090 (+529353us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.713156 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.713249 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:13.917102 (+203853us) raft_consensus.cc:1685] finished
0709 12:24:13.917133 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.917368 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7607,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":20100,"thread_start_us":408425,"threads_started":1,"wal-append.queue_time_us":678904}
W20260709 12:24:13.921591  7585 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.006s	sys 0.015s
W20260709 12:24:13.922246  7585 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.006s	sys 0.015s
W20260709 12:24:13.924803  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 264885us
I20260709 12:24:13.926414  5900 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:13.927223  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 563659us
W20260709 12:24:13.929102  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:13.929656  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.932412  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.907663  7903 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:13.933220  5220 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
W20260709 12:24:13.937083  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 429183us
I20260709 12:24:13.892836  7967 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.821030  7975 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:13.952605  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:13.953258  8018 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:13.953866  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:13.954177  8018 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:13.954725  5070 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:13.955933  7962 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:13.956805  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:13.958940  7619 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:13.960713  7967 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:13.962628  8016 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:13.963246  8016 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:13.965277  5900 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:13.824687  7843 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:24:13.903322  7906 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:13.969815  7906 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:13.998924  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 554124us
I20260709 12:24:13.951025  4897 catalog_manager.cc:5697] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
W20260709 12:24:14.006335  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 55673.5us
W20260709 12:24:14.008057  5041 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.011s	sys 0.012s
W20260709 12:24:14.008504  5041 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.011s	sys 0.012s
I20260709 12:24:13.941946  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:14.013547  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 112562us
W20260709 12:24:14.008833  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 741676us
I20260709 12:24:13.867601  7705 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:14.017671  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 498) took 1210 ms. Trace:
I20260709 12:24:14.007313  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:14.024119  5147 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:14.020833  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:14.029781  4897 catalog_manager.cc:5697] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 6 to 7, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 7 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:14.031214  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 146853us
I20260709 12:24:14.030148  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:14.042404  5074 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:14.034320  5188 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:14.021473  5145 rpcz_store.cc:276] 0709 12:24:12.817240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.817496 (+   256us) service_pool.cc:224] Handling call
0709 12:24:12.817942 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.921479 (+103537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.921618 (+   139us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.978525 (+ 56907us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.174362 (+195837us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.174398 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.240795 (+ 66397us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b540019f354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:13.240869 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:13.792946 (+552077us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b24001b18c8. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a52d49c4 00007fd8a52d2acb 00007fd8a548ddc9 00007fd8a548da55 00007fd8a548d09d 00007fd8a80bf10e 00007fd8a80d688c 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221
0709 12:24:14.017297 (+224351us) raft_consensus.cc:1685] finished
0709 12:24:14.017329 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.017528 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7782,"raft.queue_time_us":10968,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":32575,"spinlock_wait_cycles":188979200,"thread_start_us":195288,"threads_started":1,"wal-append.queue_time_us":479236}
I20260709 12:24:14.049988  7585 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:24:14.052944  7619 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:14.057786  7994 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:14.061971  7994 raft_consensus.cc:697] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:14.062918  7994 consensus_queue.cc:237] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:14.066941  7705 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:14.058005  4897 catalog_manager.cc:5697] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 6 to 7, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 7 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:14.068128  4897 catalog_manager.cc:5697] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:24:14.051553  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:14.056793  7690 consensus_meta.cc:287] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.000s
I20260709 12:24:14.071326  7690 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:14.072520  5068 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:14.074405  7690 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:14.069445  4897 catalog_manager.cc:5697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 8, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 8 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:14.008312  7842 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:14.086081  7842 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:14.081210  4897 catalog_manager.cc:5697] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 2 to 3, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:14.086570  7842 raft_consensus.cc:697] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:14.087345  4897 catalog_manager.cc:5697] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:24:14.087596  7842 consensus_queue.cc:237] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:14.089681  7975 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:14.096335  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 64111.9us
I20260709 12:24:14.077167  7843 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:14.022370  7831 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:14.110683  8014 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 12:24:14.118302  7906 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:14.119130  7906 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:14.119517  7906 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.119828  7906 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:14.107178  7843 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:14.122296  7843 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:14.123131  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:14.125000  5190 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:14.134189  5042 leader_election.cc:400] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10.
I20260709 12:24:14.134543  5042 leader_election.cc:403] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:14.133512  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:14.108516  7831 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.150730  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:14.165520  5897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.006s	sys 0.031s
W20260709 12:24:14.171880  5897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.007s	sys 0.031s
W20260709 12:24:14.171145  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 73698.1us
W20260709 12:24:14.173017  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.173062  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:14.178702  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 861) took 1376 ms. Trace:
I20260709 12:24:14.201005  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:14.202529  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:14.203647  5218 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:14.171005  4897 catalog_manager.cc:5697] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 5, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:14.208957  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:14.165992  7689 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.005s	sys 0.050s
I20260709 12:24:14.214809  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:14.220460  5942 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:14.145324  8014 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:14.215682  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:14.216028  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:14.145920  8015 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:14.216996  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:14.218537  4897 catalog_manager.cc:5697] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:24:14.152719  7788 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:24:14.223778  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 50809.2us
I20260709 12:24:14.280136  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:14.280337  5042 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:14.151968  7924 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10.
I20260709 12:24:14.247824  7698 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:14.294893  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:14.294293  8011 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:24:14.295946  5221 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:14.248224  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:14.144335  7432 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 12:24:14.180751  7831 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:14.184706  5219 rpcz_store.cc:276] 0709 12:24:12.823065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.835816 (+ 12751us) service_pool.cc:224] Handling call
0709 12:24:12.877493 (+ 41677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.877902 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:13.696320 (+818418us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.150638 (+454318us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.150719 (+    81us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:14.178399 (+ 27680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.178492 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.178639 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":217944,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":22456,"thread_start_us":1188251,"threads_started":2}
I20260709 12:24:14.285811  4897 catalog_manager.cc:5697] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:14.180063  7967 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:14.152525  7585 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 12:24:14.178669  8016 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:14.220096  7689 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.005s	sys 0.051s
I20260709 12:24:14.294178  7690 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:14.306802  7690 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:14.309459  7690 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:14.210381  7845 consensus_meta.cc:287] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.756s	user 0.008s	sys 0.004s
W20260709 12:24:14.322926  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 350736us
I20260709 12:24:14.202814  7705 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:14.279587  7788 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:14.340122  5898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.024s	sys 0.006s
W20260709 12:24:14.347553  5898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.024s	sys 0.007s
W20260709 12:24:14.348088  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 601274us
W20260709 12:24:14.356253  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 231303us
I20260709 12:24:14.288131  7616 raft_consensus.cc:3055] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:14.370509  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 90257.6us
I20260709 12:24:14.373867  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:14.374001  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:14.365798  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:14.377169  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:14.378149  5899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.028s	sys 0.024s
W20260709 12:24:14.379280  5899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.028s	sys 0.024s
W20260709 12:24:14.379673  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 556913us
I20260709 12:24:14.382190  5221 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:14.345340  7690 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:24:14.384709  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 350534us
I20260709 12:24:14.357595  7788 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:14.388664  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.392083  5900 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:14.386205  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:14.338569  7845 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:14.397200  5068 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:14.414673  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:14.409073  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.384837  7924 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:14.430234  5898 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:14.376302  7616 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:14.393330  8017 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:14.436756  5114 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:14.437175  5142 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:14.451414  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 430653us
I20260709 12:24:14.432960  8015 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:14.437680  7652 consensus_meta.cc:287] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.888s	user 0.016s	sys 0.000s
W20260709 12:24:14.461670  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 43508.7us
I20260709 12:24:14.462716  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:14.452888  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:14.464922  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 72967.1us
I20260709 12:24:14.415819  7903 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:14.440433  7962 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:14.474262  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:14.441200  8017 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:14.491048  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 92369.2us
I20260709 12:24:14.493151  5224 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:14.451311  7933 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:14.494920  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:14.477427  8015 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:14.520521  7831 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:14.501466  7698 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:14.525684  7924 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:14.525840  8020 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:14.490715  7962 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.456596  8009 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:14.526536  7924 raft_consensus.cc:697] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:14.535728  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:14.498090  8017 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:14.525282  7539 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:14.516424  7616 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:14.541610  8045 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:24:14.542402  8022 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 12:24:14.543330  8021 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:14.543713  7831 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:14.544064  8014 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:14.544313  8030 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:14.526860  7698 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:14.535373  7962 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:14.594919  7619 consensus_meta.cc:287] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.002s
I20260709 12:24:14.597172  7788 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:14.554157  8015 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:14.557112  8029 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.619846  8029 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:14.620500  8017 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:14.557605  7846 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:14.433004  7906 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:14.560782  7982 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.665s	user 0.009s	sys 0.000s
I20260709 12:24:14.609579  7788 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:14.563156  7903 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:14.488607  7842 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:24:14.526906  7883 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:14.565358  8009 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:14.566219  7845 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:14.567739  7924 consensus_queue.cc:237] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:14.572714  7585 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:14.587872  7616 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:14.561067  7843 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:14.589717  7698 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:14.585788  8014 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:14.597744  7896 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:14.535694  7867 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:14.600864  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 106037us
W20260709 12:24:14.662079  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 199495us
I20260709 12:24:14.573966  7539 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:14.610263  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 148752us
I20260709 12:24:14.617172  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.584275  7432 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:14.639936  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 187067us
I20260709 12:24:14.663482  5067 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
W20260709 12:24:14.641597  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 188780us
I20260709 12:24:14.605556  7619 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.645346  5150 raft_consensus.cc:2410] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:24:14.652371  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 53339.7us
I20260709 12:24:14.647456  7883 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:14.654928  8024 log.cc:927] Time spent T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.221s	user 0.001s	sys 0.001s
I20260709 12:24:14.657675  7962 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:14.521843  8032 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:24:14.679497  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:14.664597  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.671131  7432 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:14.681560  5075 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:14.682895  7432 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:14.679567  8029 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:14.684402  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:14.685040  5943 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:14.685403  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:14.685240  8032 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.686079  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:14.686930  5939 raft_consensus.cc:2368] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:14.670244  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.665305  5041 leader_election.cc:400] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:14.687414  5041 leader_election.cc:403] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:14.682960  5945 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:14.688676  7788 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:14.688937  7689 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:14.665499  7896 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:14.691910  7896 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:14.686470  8032 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:14.664204  5115 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:14.685005  5899 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:14.711966  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 126454us
I20260709 12:24:14.662133  7867 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.714473  5188 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:14.665190  7539 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:14.714180  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.674983  7619 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:14.716399  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:14.675729  7883 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:14.723096  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 50213.2us
W20260709 12:24:14.721835  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 57699.6us
I20260709 12:24:14.727250  5115 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:14.727722  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.727880  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:14.732339  5114 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:14.733265  5114 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:14.733575  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:14.735402  7896 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:14.735862  7831 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:14.735962  7789 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:14.736405  7831 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.736588  7789 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:14.736953  7831 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:14.737093  7789 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:14.739681  5143 raft_consensus.cc:2368] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:14.728469  5071 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:14.741223  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.741351  5898 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:14.742389  5899 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:14.692317  7652 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.748696  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:14.755821  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:14.752838  5114 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:14.748013  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:14.754010  5041 leader_election.cc:400] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:14.757200  5041 leader_election.cc:403] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:14.751318  5221 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:24:14.761267  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:14.767382  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:14.726392  7867 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:14.774426  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:14.775465  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:14.776064  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1075) took 1018 ms. Trace:
I20260709 12:24:14.777057  8050 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:14.716506  8048 raft_consensus.cc:3055] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:14.729977  7903 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.717499  7619 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:14.723922  7906 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:14.722198  8049 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:14.783146  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:14.780315  5150 rpcz_store.cc:276] 0709 12:24:13.757943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.803619 (+ 45676us) service_pool.cc:224] Handling call
0709 12:24:14.775833 (+972214us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b54007aadd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:14.775993 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1932395776}
I20260709 12:24:14.784219  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:14.785127  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:14.796566  5946 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:14.720827  8032 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:14.738644  7982 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:14.787992  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:14.718541  8054 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:14.789361  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.749372  7845 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:14.747439  8051 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:14.794669  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:14.794822  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.798179  8034 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:14.798314  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:14.834769  5224 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:14.786284  8048 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:14.798588  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:14.798957  8040 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:24:14.799319  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:14.799906  7975 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:14.780953  7690 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:24:14.801190  8053 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:24:14.849068  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:14.801925  8049 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:14.851279  8049 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:14.852360  5224 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:24:14.768637  7652 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:14.855311  7652 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:14.808741  7846 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:14.856380  7846 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:14.844534  7840 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.859423  7840 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:14.843493  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:14.862028  5947 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:14.862278  7976 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:14.864948  7976 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:14.866091  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:14.866523  8055 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:14.866940  5216 raft_consensus.cc:2393] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:14.867086  8055 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:14.867908  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:14.869627  7975 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:14.871829  7975 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:14.872845  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 87843.5us
I20260709 12:24:14.873667  5899 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:14.874912  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:14.875021  7843 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:14.875866  7843 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:14.876057  5899 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:14.876621  7843 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:14.877097  8026 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:14.877776  8015 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:14.822621  5221 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:14.787132  8050 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.879092  8050 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:14.879537  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:14.880198  5223 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:24:14.782402  8009 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:14.721971  8014 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:14.792382  7842 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:14.882272  8014 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:14.838851  5071 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 9: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:14.882853  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:14.838073  5948 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:14.883486  5075 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:14.843662  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:14.884143  5215 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
W20260709 12:24:14.886186  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45494us
I20260709 12:24:14.825608  7845 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.887053  7845 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:14.899354  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.900102  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:14.885632  7994 consensus_queue.cc:1175] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" responder_term: 10 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [9.6-9.6]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:14.900703  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:14.904744  5146 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:14.904949  5150 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:14.861418  7840 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:14.884970  7883 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:14.900127  5072 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:14.907302  5899 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:14.910884  5946 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:14.835036  8051 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.915177  5146 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:14.916244  5947 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:14.916374  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:14.869313  8055 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:14.922439  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:14.925239  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.840317  8017 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:14.840657  7619 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:14.838281  8034 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.927320  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:14.995136  5943 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:14.995461  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:14.796331  7903 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:14.925369  5143 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:14.934409  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:15.000849  5142 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:14.959072  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:14.932204  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:14.960749  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 53557us
I20260709 12:24:14.903692  7982 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.964344  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:14.991853  5944 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:14.996929  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:14.999014  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 148060us
I20260709 12:24:15.004211  5190 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:15.001891  5145 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
W20260709 12:24:14.996369  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 74050.3us
I20260709 12:24:15.006453  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:14.933022  8051 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:15.014953  5067 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 6: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:14.961030  8054 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:15.025614  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:14.959944  7975 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.024616  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:15.039597  7975 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:15.040787  5115 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:15.041746  7933 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:15.043121  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:15.043984  7933 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:15.043246  8016 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:14.841439  8059 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.169s
I20260709 12:24:15.041837  7906 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:15.045202  8016 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:15.046918  8016 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:15.047780  7975 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:15.049319  5115 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:15.050176  7906 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:15.050647  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:15.052541  5939 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:15.048460  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.054509  5946 raft_consensus.cc:2368] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:15.001327  8034 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:15.066470  5075 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:15.027311  7705 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:15.064172  8052 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:15.080770  5943 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:15.079161  7705 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:15.095351  7705 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:15.097859  8039 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:24:15.098766  8064 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:24:15.023628  7982 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:15.100427  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 346666us
I20260709 12:24:15.123688  7831 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:15.079375  7962 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:15.034168  7698 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:15.139657  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.171295  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 103922us
I20260709 12:24:15.175917  5223 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:15.138847  5150 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
W20260709 12:24:15.181568  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.182088  8061 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:24:15.182421  8062 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:15.144431  8060 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.114s	user 0.001s	sys 0.001s
I20260709 12:24:15.184713  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:15.189059  5219 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:15.183130  8062 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:15.191569  7789 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:15.191962  8062 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:15.198541  7967 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:15.199494  7967 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:15.201195  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:15.202207  5947 raft_consensus.cc:2410] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:15.203313  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:15.203773  8066 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.198s	user 0.000s	sys 0.003s
I20260709 12:24:15.215000  7698 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:15.218037  7883 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:15.218056  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:15.218698  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 92820.9us
I20260709 12:24:15.218967  8069 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:15.220264  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:15.036713  8048 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:15.221750  8009 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:24:15.222214  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 296372us
I20260709 12:24:15.222385  8009 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:15.223724  5145 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:24:15.180361  8065 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:24:15.234318  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:15.233309  5143 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:15.235489  5948 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:24:15.236415  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.236720  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:15.231527  5898 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:15.245131  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.241192  7840 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:15.254276  5947 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:15.222536  8048 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:15.263720  5223 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:15.247380  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:15.271509  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:15.275835  5900 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:15.278100  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.278401  5042 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:15.279305  5900 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:15.280422  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:15.281420  5224 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:15.280144  5221 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:15.284404  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:15.285429  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:15.286586  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:15.247884  7883 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:15.291079  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:15.341707  5143 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:15.293115  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.280249  7619 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:24:15.298606  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:15.299264  7789 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:15.302309  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:15.347738  5068 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:15.303386  5067 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:15.304317  7967 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:15.310093  8073 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:24:15.313624  5941 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:15.281478  7867 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:15.317075  8052 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:15.317956  8071 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:15.293085  7975 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:15.321303  7831 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:15.322882  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 318761us
I20260709 12:24:15.241315  7539 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:15.280431  8030 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:15.288216  8049 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:15.289438  8054 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:15.343394  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 57999us
I20260709 12:24:15.223490  8009 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:15.346006  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.392926  5940 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:15.294867  7903 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:15.331830  7616 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:15.399464  7616 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:15.400028  7616 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:15.373519  7975 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.405907  7975 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:15.407855  8083 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.408465  8083 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:15.303725  8051 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:15.411319  8051 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:15.341248  7883 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.412210  7883 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:15.413749  7982 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:15.417938  5149 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:15.344772  8017 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:15.381501  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:15.427829  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 46449.4us
I20260709 12:24:15.393692  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.355849  7698 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:15.390151  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:15.398017  8069 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:24:15.398634  7846 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.002s
W20260709 12:24:15.358868  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 76469.5us
I20260709 12:24:15.435052  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:15.435156  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 103264us
I20260709 12:24:15.435843  5939 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:15.436465  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 174949us
I20260709 12:24:15.436419  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:15.435359  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 134919us
I20260709 12:24:15.386180  8049 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.440060  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:15.443123  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:15.443033  5041 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:15.394538  8054 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:15.377666  8016 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:15.451663  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.431213  7789 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:15.381018  8034 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:15.439141  8030 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:15.450026  8051 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:15.477427  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:15.447669  8049 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:15.490664  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:15.462183  8054 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:15.431360  8069 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:15.495162  5948 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:15.482798  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:15.500475  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:15.491470  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:15.480544  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:15.493325  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 50359.7us
I20260709 12:24:15.461977  7933 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:15.521420  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 78392.9us
I20260709 12:24:15.522526  5939 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
W20260709 12:24:15.509455  7976 consensus_meta.cc:287] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.644s	user 0.008s	sys 0.004s
I20260709 12:24:15.508450  5142 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:15.501434  7698 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:15.509131  5144 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:15.515588  8050 consensus_meta.cc:287] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.636s	user 0.008s	sys 0.005s
I20260709 12:24:15.510815  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:15.438447  8015 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:24:15.513341  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:15.543741  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:15.515061  5150 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:15.451140  8017 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:24:15.486604  7982 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:15.517069  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.518859  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:15.519136  5042 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:15.494609  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.496749  8071 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:15.522408  5041 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:15.530471  8088 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:15.531229  8051 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:15.444257  7846 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:15.572288  8083 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:15.573495  7846 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:15.547278  8050 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:15.576346  5072 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:15.522483  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:15.501602  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.499606  8030 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:15.500135  8086 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:15.516372  7432 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:15.492467  7941 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 12:24:15.543358  7690 consensus_meta.cc:287] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.647s	user 0.006s	sys 0.005s
W20260709 12:24:15.554246  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.566598  5941 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:24:15.566844  7539 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:15.543187  7933 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:15.570209  8051 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:15.580473  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 61415.4us
W20260709 12:24:15.581415  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 59095.8us
I20260709 12:24:15.547464  7976 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:15.589435  5942 raft_consensus.cc:2368] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10.
W20260709 12:24:15.597846  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 87121.3us
I20260709 12:24:15.611591  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:15.604534  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 82164.5us
I20260709 12:24:15.616533  5150 raft_consensus.cc:2368] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:15.587873  7698 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:15.618322  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:15.603585  7845 consensus_meta.cc:287] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.716s	user 0.013s	sys 0.000s
I20260709 12:24:15.622579  5075 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:15.624040  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:15.591357  8085 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:15.629755  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:15.638501  8014 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:15.628916  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:15.644061  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:15.627822  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:15.645644  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:15.652149  5071 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:15.652390  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.632185  5115 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:15.652534  5947 raft_consensus.cc:2410] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:24:15.643500  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:15.668895  5150 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:24:15.669776  5150 pending_rounds.cc:85] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 3
I20260709 12:24:15.670109  5150 pending_rounds.cc:107] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:24:15.671160  5042 leader_election.cc:400] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:15.670538  5150 raft_consensus.cc:2889] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:15.674451  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:15.617050  7690 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:15.594848  8086 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:15.587735  8071 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:15.621038  8075 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 12:24:15.606138  8062 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:24:15.680862  8079 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.649106  8085 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:15.682577  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:15.688688  5948 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:15.733318  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:15.613500  8089 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:15.691172  5946 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:15.692319  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:15.692390  5073 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:15.739502  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 47333us
I20260709 12:24:15.740779  5114 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:15.698616  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:15.699766  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:15.649459  7705 consensus_meta.cc:287] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.554s	user 0.007s	sys 0.006s
I20260709 12:24:15.673799  8087 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:15.608881  7432 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:15.718667  5219 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.529s	user 0.014s	sys 0.003s
I20260709 12:24:15.761226  5219 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:15.588934  8069 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:15.716504  7846 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:24:15.723083  8016 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:15.648182  7845 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:15.724587  8084 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:15.725649  7903 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.715281  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:15.717128  5941 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:24:15.716065  5072 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:15.693888  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:15.621449  8035 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:24:15.748335  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.795881  5941 pending_rounds.cc:85] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 4
I20260709 12:24:15.751398  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:15.799238  5941 pending_rounds.cc:107] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:24:15.757018  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:15.799707  5941 raft_consensus.cc:2889] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:15.801052  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.757256  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.672590  7867 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:15.774250  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 75803.1us
W20260709 12:24:15.776160  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.720966  7539 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:15.814349  7539 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.814960  7539 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:15.752475  8030 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:15.815796  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.778975  8087 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:15.818591  8087 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:15.820336  8083 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:15.824105  8016 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:15.825156  8032 raft_consensus.cc:3055] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:15.825986  8032 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:15.827142  8032 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:15.828389  8032 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:15.782975  8062 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:15.832779  8084 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:15.761137  7705 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:15.828450  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.744005  8085 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:15.806169  5150 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 12:24:15.777442  8017 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.757719  8079 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:15.782557  8034 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:15.719503  7933 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:15.834652  5147 raft_consensus.cc:2368] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:15.835577  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.836521  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.846984  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.845090  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.866819  5946 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:15.874117  5216 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:15.874444  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:15.868999  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:15.878374  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:15.857681  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:15.879773  5219 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:15.857928  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:15.833482  8091 log.cc:927] Time spent T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.199s	user 0.003s	sys 0.000s
I20260709 12:24:15.785332  8095 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.861066  5218 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:15.878558  5150 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:15.821308  8083 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:15.881429  5946 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:15.885502  5944 raft_consensus.cc:2368] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11.
I20260709 12:24:15.885262  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:15.894173  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:15.888823  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:15.887286  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 226286us
I20260709 12:24:15.897811  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:15.899597  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:15.901743  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.810104  7903 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:15.852128  7845 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:15.906512  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.918414  5041 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:15.899430  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:15.907917  5221 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:24:15.915741  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:15.926465  5150 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:15.931617  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:15.912067  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:15.935348  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:15.878280  7982 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:15.921363  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:15.941869  5944 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:15.933414  5215 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:15.858315  8017 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:15.945616  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:15.950544  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:15.955506  8071 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:15.956085  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:15.956915  7705 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:15.957369  5943 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:15.957952  8083 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:15.959355  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.959936  8083 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:15.952575  5223 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:15.963660  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:15.969213  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 228575us
I20260709 12:24:15.971864  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:15.983655  5943 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
W20260709 12:24:15.987284  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.991199  5899 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:15.972906  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 231497us
I20260709 12:24:15.956602  8071 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:15.947541  7976 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:16.001938  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:16.015936  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 97657.5us
I20260709 12:24:15.900188  8095 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:15.984179  8009 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:16.030736  5946 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:16.048900  5943 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:15.972076  8048 consensus_queue.cc:1059] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 12:24:16.037452  7883 consensus_meta.cc:287] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.617s	user 0.018s	sys 0.000s
I20260709 12:24:16.093524  7903 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:16.094435  7883 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:16.095809  8017 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:16.096906  8054 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.584s	user 0.015s	sys 0.000s
I20260709 12:24:16.069886  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:16.024787  8094 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:16.070539  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:16.024477  7976 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:16.096786  8079 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:16.097944  5945 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:16.122066  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.123400  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:16.123876  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:16.124042  5946 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
W20260709 12:24:16.069837  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 78859.4us
I20260709 12:24:16.126443  5943 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:16.129139  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 105356us
W20260709 12:24:16.131480  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.097163  7883 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:16.135411  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:16.137523  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:16.140648  5899 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:16.149823  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:16.152714  5899 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:16.113422  8094 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:16.158536  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.155066  5941 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:16.154340  7883 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:16.164716  5942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 576) took 1409 ms (client timeout 1798 ms). Trace:
I20260709 12:24:16.165087  7867 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:16.165488  8107 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:16.165689  8076 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:16.144062  8108 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.172521  8108 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:16.178614  8101 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.169s	user 0.000s	sys 0.002s
I20260709 12:24:16.183452  7976 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:16.155285  7975 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.749s	user 0.014s	sys 0.004s
I20260709 12:24:16.191475  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:16.144161  8080 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:16.132396  7585 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:24:16.120651  8054 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:16.157982  8017 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:16.164836  7924 raft_consensus.cc:3055] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:16.126682  8059 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 12:24:16.160569  8094 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:16.191921  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 54810.8us
W20260709 12:24:16.165401  5942 rpcz_store.cc:269] 0709 12:24:14.755267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.755474 (+   207us) service_pool.cc:224] Handling call
0709 12:24:16.164390 (+1408916us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b54002e5254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:16.164599 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1882888576}
I20260709 12:24:16.141378  7994 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:24:16.200970  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:16.161267  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:16.251786  8095 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:16.195036  7616 consensus_meta.cc:287] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.794s	user 0.013s	sys 0.005s
W20260709 12:24:16.254160  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 53289.4us
I20260709 12:24:16.193207  8009 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:16.258584  5115 leader_election.cc:400] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10.
I20260709 12:24:16.283725  5115 leader_election.cc:403] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:16.255100  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 58402.7us
I20260709 12:24:16.202744  7982 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:16.211798  8017 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:16.254982  8115 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:16.208899  7924 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:16.290489  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:16.214259  8054 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:16.291898  8109 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:16.252820  8111 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:16.208019  7539 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:16.306248  5944 raft_consensus.cc:2410] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:24:16.207454  7975 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:16.327283  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:16.286327  8088 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:16.294131  8079 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 10
W20260709 12:24:16.351418  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 125634us
I20260709 12:24:16.359366  5939 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:16.360260  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 522) took 1443 ms. Trace:
I20260709 12:24:16.270807  7616 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:16.371671  5944 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
W20260709 12:24:16.375707  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 79228.7us
I20260709 12:24:16.360929  5145 rpcz_store.cc:276] 0709 12:24:14.917006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.005576 (+ 88570us) service_pool.cc:224] Handling call
0709 12:24:15.036826 (+ 31250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.224162 (+187336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.359835 (+1135673us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400115654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:16.360164 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347317504}
I20260709 12:24:16.326866  8109 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:16.320363  7689 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:24:16.410305  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:16.427407  8097 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.428367  8097 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:16.432163  8097 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:16.435555  7975 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:16.439118  7616 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:16.375322  7867 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:16.442108  7867 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:16.323745  7924 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:16.446758  7924 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:16.454327  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 972) took 1720 ms. Trace:
I20260709 12:24:16.455102  5149 rpcz_store.cc:276] 0709 12:24:14.746983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.747104 (+   121us) service_pool.cc:224] Handling call
0709 12:24:16.453959 (+1706855us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b540036e554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:16.454195 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466257280}
W20260709 12:24:16.464821  8077 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.948s	user 0.003s	sys 0.000s
I20260709 12:24:16.466343  7867 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:16.318042  8115 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:16.464882  8032 consensus_meta.cc:287] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.636s	user 0.017s	sys 0.003s
I20260709 12:24:16.321779  8111 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:16.337091  8088 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:16.421232  8072 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.474299  5948 raft_consensus.cc:2410] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:24:16.500983  5147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 974) took 1753 ms (client timeout 1998 ms). Trace:
W20260709 12:24:16.563957  5147 rpcz_store.cc:269] 0709 12:24:14.747640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.747739 (+    99us) service_pool.cc:224] Handling call
0709 12:24:16.500479 (+1752740us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540019ee54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:16.500826 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2477807488}
I20260709 12:24:16.502488  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 975) took 1754 ms. Trace:
I20260709 12:24:16.566378  5144 rpcz_store.cc:276] 0709 12:24:14.747939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.748069 (+   130us) service_pool.cc:224] Handling call
0709 12:24:16.502158 (+1754089us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b5400710054. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:16.502382 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1640233856}
I20260709 12:24:16.508028  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:16.617719  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.513250  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.702706  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 188491us
W20260709 12:24:16.558575  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.536541  5142 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:16.508587  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:16.551350  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.514160  7840 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:16.525429  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:16.537083  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 580) took 1746 ms. Trace:
I20260709 12:24:16.510360  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:16.703692  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:16.538146  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:16.539573  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:16.756116  5942 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:24:16.774600  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 235092us
I20260709 12:24:16.518098  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:16.780740  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 255339us
I20260709 12:24:16.752100  5940 rpcz_store.cc:276] 0709 12:24:14.790365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.790531 (+   166us) service_pool.cc:224] Handling call
0709 12:24:16.536782 (+1746251us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b5400008a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:16.536994 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263433984}
W20260709 12:24:16.620112  8049 consensus_meta.cc:287] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.112s	user 0.021s	sys 0.000s
I20260709 12:24:16.756690  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:16.729672  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:16.473412  8115 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:16.535080  8070 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:16.548933  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:16.514369  7967 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.787950  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:16.782300  5072 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:16.524922  8118 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:16.509559  8016 raft_consensus.cc:3055] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:16.794512  5946 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:16.795284  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 277313us
I20260709 12:24:16.485508  7432 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:16.611982  8107 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.508239  7705 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.303s
I20260709 12:24:16.835767  8107 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:16.836079  7585 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:16.819033  7975 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:16.838557  8107 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:16.802598  5144 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:16.842109  8108 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:16.844812  5142 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:16.785588  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:16.810006  5074 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:24:16.538352  8100 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:16.535315  8111 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:16.813957  5899 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:24:16.782361  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.531365  7976 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:16.703964  8009 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:16.731357  8104 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:16.822319  7967 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:16.823607  8030 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.824666  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:16.801508  8116 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:16.830444  7432 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:16.832638  8069 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:16.811879  8118 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:16.819962  8016 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:16.539047  8072 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:16.834510  7690 consensus_queue.cc:1059] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:24:16.847734  8085 consensus_meta.cc:287] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.988s	user 0.016s	sys 0.000s
I20260709 12:24:16.848554  8070 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:16.798489  8034 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:16.849591  7539 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:16.537985  8088 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:16.851565  8071 consensus_meta.cc:287] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.805s	user 0.011s	sys 0.002s
I20260709 12:24:16.530908  7789 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.523914  8086 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:16.979984  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 440484us
W20260709 12:24:16.809664  8083 consensus_meta.cc:287] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.849s	user 0.021s	sys 0.000s
I20260709 12:24:16.486685  7924 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.330s
I20260709 12:24:16.796079  8054 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:16.854017  8109 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:16.994234  8067 log.cc:927] Time spent T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 1.757s	user 0.007s	sys 0.008s
I20260709 12:24:16.994418  8054 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.995169  8054 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:16.944626  8116 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:16.996196  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 582) took 2331 ms. Trace:
W20260709 12:24:16.902578  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 88856.3us
I20260709 12:24:16.903004  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:24:16.855338  7933 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.016s	sys 0.014s
I20260709 12:24:16.905042  8032 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:16.778559  7840 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:16.902053  7976 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:16.808897  8049 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:16.893695  8107 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:16.909798  8059 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.021s	sys 0.034s
W20260709 12:24:16.969674  8089 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.172s	user 0.026s	sys 0.028s
I20260709 12:24:16.901773  8100 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:16.983990  7789 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:17.008674  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 105755us
W20260709 12:24:17.005972  7933 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.016s	sys 0.015s
W20260709 12:24:16.984321  8079 consensus_meta.cc:287] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.006s
I20260709 12:24:16.939376  8030 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:16.927110  8104 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.038196  7975 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.016640  8032 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.047254  8095 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:16.957152  8085 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.049598  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:17.047737  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:17.057905  5224 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:17.064293  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:16.998188  8050 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:17.066431  5147 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:24:17.051869  8089 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.254s	user 0.027s	sys 0.029s
W20260709 12:24:17.052958  8059 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.021s	sys 0.036s
I20260709 12:24:17.070529  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:16.996774  5076 rpcz_store.cc:276] 0709 12:24:14.665026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.665844 (+   818us) service_pool.cc:224] Handling call
0709 12:24:14.670653 (+  4809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.671132 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.671161 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.671670 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.807504 (+135834us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:14.807556 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.807641 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.788470 (+1980829us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b240050bda8. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a52d49c4 00007fd8a52d2acb 00007fd8a548ddc9 00007fd8a548da55 00007fd8a548d09d 00007fd8a80bf10e 00007fd8a80d688c 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221
0709 12:24:16.995812 (+207342us) raft_consensus.cc:1685] finished
0709 12:24:16.995836 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.996103 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2318,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"spinlock_wait_cycles":53252992,"thread_start_us":76245,"threads_started":1,"wal-append.queue_time_us":224067}
I20260709 12:24:17.073436  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:17.075400  5148 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:24:16.981884  8071 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:17.019937  8062 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:17.052042  8116 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.050287  7924 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:17.083315  5216 raft_consensus.cc:2410] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:17.085376  5217 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:17.047348  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:17.083925  5141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.040s	sys 0.080s
I20260709 12:24:17.068583  8079 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10.
I20260709 12:24:17.094403  8089 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:17.134842  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.052359  8107 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:17.151027  5141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.042s	sys 0.093s
I20260709 12:24:17.150749  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:17.155193  5072 raft_consensus.cc:1240] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:17.151609  5069 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:17.125437  5223 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:17.165014  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:17.114463  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:17.091110  7933 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:17.081414  7975 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.160899  7840 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.165907  5076 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:17.060758  8016 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.055903  8118 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.064086  8009 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:17.081637  8032 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:17.214833  7976 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.213481  8016 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:17.181247  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 69945.5us
W20260709 12:24:17.242723  5189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.016s	sys 0.020s
W20260709 12:24:17.249378  5189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.017s	sys 0.020s
W20260709 12:24:17.156756  8128 log.cc:927] Time spent T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:24:17.089488  8095 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:17.181809  8126 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:24:17.189271  8030 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:17.277271  5948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 771) took 1452 ms (client timeout 1825 ms). Trace:
W20260709 12:24:17.277813  5948 rpcz_store.cc:269] 0709 12:24:15.825278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.896646 (+ 71368us) service_pool.cc:224] Handling call
0709 12:24:17.277033 (+1380387us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b540023ffd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:17.277207 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317568384}
I20260709 12:24:17.279202  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:17.279750  8085 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:17.279939  5947 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:17.184396  8072 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.183938  8100 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.113269  8062 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:17.281525  8070 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.181155  8089 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:17.112871  7432 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:17.286387  7432 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.181670  8130 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.282102  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:17.186553  8107 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.295212  7789 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.165335  7924 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:17.180660  8104 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:17.182916  7967 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:17.181774  8034 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.212325  8118 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:17.298971  5947 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:24:17.299804  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.188825  8049 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:17.299856  5942 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:17.312914  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1095) took 1023 ms. Trace:
I20260709 12:24:17.329106  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:17.329967  5947 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:17.313112  5149 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:17.314561  5071 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
W20260709 12:24:17.323088  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.323316  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 593) took 1257 ms. Trace:
I20260709 12:24:17.282016  8062 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.329487  5142 rpcz_store.cc:276] 0709 12:24:16.289573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.289870 (+   297us) service_pool.cc:224] Handling call
0709 12:24:17.312595 (+1022725us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54004fae54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:17.312835 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":526937984}
I20260709 12:24:17.335631  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1094) took 1047 ms. Trace:
I20260709 12:24:17.336376  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:17.336967  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.336376  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:17.345481  5142 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:17.346565  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:17.344023  5141 rpcz_store.cc:276] 0709 12:24:16.288853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.289075 (+   222us) service_pool.cc:224] Handling call
0709 12:24:16.315253 (+ 26178us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:16.330716 (+ 15463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:16.330794 (+    78us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:16.372084 (+ 41290us) log.cc:844] Serialized 52 byte log entry
0709 12:24:16.470918 (+ 98834us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b480004c758. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ad464aa1 00007fd8ad45bb37 00007fd8ad45b9ca 00007fd8ad55326f 00007fd8ad552f37 00007fd8ad56755c 00007fd8ad567508 00007fd8ad567488 00007fd8ad567443 00007fd8ad5666ec 00007fd8ac490a96
0709 12:24:17.151653 (+680735us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:17.309569 (+157916us) log.cc:844] Serialized 14 byte log entry
0709 12:24:17.320012 (+ 10443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.320132 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:17.320188 (+    56us) raft_consensus.cc:1685] finished
0709 12:24:17.335424 (+ 15236us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.335576 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10026,"raft.queue_time_us":21047,"raft.run_cpu_time_us":3262,"raft.run_wall_time_us":149208,"spinlock_wait_cycles":45961472,"thread_start_us":875389,"threads_started":2,"wal-append.queue_time_us":813576}
I20260709 12:24:17.347760  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:17.347977  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:17.348369  5142 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:17.268424  7933 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:17.350485  5073 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 595) took 1731 ms (client timeout 1762 ms). Trace:
W20260709 12:24:17.358176  5073 rpcz_store.cc:269] 0709 12:24:15.618638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.618933 (+   295us) service_pool.cc:224] Handling call
0709 12:24:17.350220 (+1731287us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b540058aa54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:17.350407 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51285760}
I20260709 12:24:17.308513  8035 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:17.327389  7432 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:17.365609  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.301565  8030 consensus_queue.cc:1059] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:24:17.301713  8130 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.335347  7967 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:17.405328  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:17.415076  5947 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:17.377372  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:17.424415  8094 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.201s	user 0.014s	sys 0.001s
I20260709 12:24:17.390372  5189 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:17.389394  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:17.330552  8034 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.335234  8134 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:17.440821  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:17.380390  8014 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:24:17.349895  8070 consensus_queue.cc:1059] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 12:24:17.444156  8070 raft_consensus.cc:3055] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:17.444664  8070 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:17.341770  7975 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:17.429219  5071 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:17.431370  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 56542.1us
I20260709 12:24:17.425331  7924 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:17.460577  8130 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:17.437491  8095 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.418783  7967 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.462488  8130 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.463078  8130 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:17.463359  7967 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:17.438315  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.371575  8072 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:24:17.442718  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:17.375301  7976 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:17.425257  8094 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.378530  8035 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:17.440306  8134 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.455873  8034 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:17.438699  7896 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:17.472211  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.479132  7967 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:17.515775  5143 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:24:17.504360  8054 consensus_meta.cc:287] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.007s
I20260709 12:24:17.451131  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:17.485574  5190 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:17.405588  7585 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
W20260709 12:24:17.538589  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 53225.2us
I20260709 12:24:17.500136  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:17.475162  8069 consensus_meta.cc:287] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.526s	user 0.013s	sys 0.000s
I20260709 12:24:17.513875  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.546975  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:17.547695  5146 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
W20260709 12:24:17.534149  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 83096.7us
I20260709 12:24:17.483767  5076 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:17.548666  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.497373  5947 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:17.536618  5900 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:17.506929  8104 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:17.542600  5071 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:17.563189  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:17.552942  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.581656  5068 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:17.586799  5189 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:17.588889  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:17.589605  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:17.591087  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:17.596553  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:17.591825  5073 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:17.587976  7432 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.538166  8076 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:17.597386  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:17.558722  8014 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:17.559978  7924 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:17.564718  7788 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:17.608114  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:17.615437  5067 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:17.567873  8136 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:17.588196  8095 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:17.591521  8089 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:17.538064  8054 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.603793  7705 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:17.628263  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 313895us
I20260709 12:24:17.631364  8014 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:17.612063  7975 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.658547  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.658928  7975 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:17.659999  5898 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.660120  8017 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:17.660924  8017 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:17.614315  8069 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:17.661286  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.607170  8049 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.580574  7896 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
W20260709 12:24:17.675974  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.665655  8069 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:17.632136  8095 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:17.604159  8097 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
W20260709 12:24:17.674142  8141 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:24:17.620038  8136 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.626786  8016 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:17.676959  5898 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.507648  8070 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.678627  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:17.670713  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 501) took 1133 ms. Trace:
I20260709 12:24:17.632951  8075 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:17.665189  7585 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:17.681200  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:17.682178  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 95515.3us
I20260709 12:24:17.694342  5224 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:17.628944  8089 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.606891  7432 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.697666  8059 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:17.654060  8009 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.636034  8130 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.695092  5189 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.708400  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:17.691372  5216 rpcz_store.cc:276] 0709 12:24:16.537446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.537674 (+   228us) service_pool.cc:224] Handling call
0709 12:24:17.670428 (+1132754us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54002c1ad4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:17.670608 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":700634496}
I20260709 12:24:17.678817  7933 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:17.686923  8136 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:17.720351  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:17.721063  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:17.675132  7705 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.704967  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:17.681425  8049 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.692384  8075 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:17.717578  8032 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.683490  8017 consensus_queue.cc:1059] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:24:17.719211  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:17.710971  8070 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:17.702805  8089 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:17.742398  5146 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:17.698724  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:17.679011  8069 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:17.726760  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:17.725629  5218 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:17.705415  7975 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:17.658572  8014 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.678812  8116 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.743350  5189 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:24:17.750846  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.755872  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.729964  8107 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:17.703184  7585 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.760583  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:17.762451  5948 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:24:17.763765  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:17.710093  8030 consensus_queue.cc:1059] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:24:17.710958  8059 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.751390  8035 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.769742  5069 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:17.769774  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 92912.6us
I20260709 12:24:17.713471  8009 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:17.780288  8075 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:17.785178  8075 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:17.785604  8143 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:24:17.778825  5898 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:17.769730  8072 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:17.789240  8151 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:17.793864  8151 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:17.795042  8151 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:17.801110  7867 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:17.807479  8049 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:17.809548  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.805984  8009 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:17.787927  8142 log.cc:927] Time spent T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.197s	user 0.001s	sys 0.001s
W20260709 12:24:17.818296  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 54579.3us
I20260709 12:24:17.805984  8095 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.808986  7867 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:17.820741  5189 leader_election.cc:400] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11.
I20260709 12:24:17.822335  5189 leader_election.cc:403] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:17.778679  8154 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:17.829718  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.830621  5189 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:17.823386  8054 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:17.836899  8123 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.930s	user 0.000s	sys 0.006s
I20260709 12:24:17.827426  8072 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11.
I20260709 12:24:17.813467  8062 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.821658  7789 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.845574  5189 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:24:17.849443  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 171755us
I20260709 12:24:17.850449  5899 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:17.842510  8149 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:24:17.854537  7705 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:17.855947  7967 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:17.852439  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:17.858556  8125 raft_consensus.cc:3055] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:17.865726  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1087) took 2197 ms. Trace:
W20260709 12:24:17.867408  8144 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 12:24:17.869294  8151 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.870131  7867 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:17.851945  8054 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.872385  8095 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:17.881337  8107 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:17.842960  8153 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:17.879446  5148 rpcz_store.cc:276] 0709 12:24:15.667966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.720008 (+ 52042us) service_pool.cc:224] Handling call
0709 12:24:15.720598 (+   590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.123852 (+1403254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.865290 (+741438us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400265054. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:17.865607 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3114871168}
I20260709 12:24:17.906877  8059 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:17.907142  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:17.888926  8098 raft_consensus.cc:3055] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:17.907909  8098 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:17.876228  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:17.908762  5146 raft_consensus.cc:2368] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:17.908893  8098 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.909842  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:17.910149  7788 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:17.910564  8098 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:17.911005  5076 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:17.885607  5189 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.881068  8154 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.920141  8153 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:17.920675  8055 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:17.892866  8158 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.921205  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.879205  8125 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:17.892946  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:17.924535  5219 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:17.924820  8125 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:17.925652  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:17.893857  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:17.926051  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:17.888466  8032 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:17.926318  8125 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:17.888727  7789 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:17.926997  7789 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.927496  7789 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:17.927919  8136 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:17.929745  7705 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:17.930071  8089 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.930559  7705 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.897003  5223 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:17.921918  8158 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:17.934299  5220 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:17.935801  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 987) took 1430 ms. Trace:
I20260709 12:24:17.334074  5941 rpcz_store.cc:276] 0709 12:24:16.066432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.211136 (+144704us) service_pool.cc:224] Handling call
0709 12:24:16.212447 (+  1311us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:16.255806 (+ 43359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:16.255851 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:16.295380 (+ 39529us) log.cc:844] Serialized 52 byte log entry
0709 12:24:16.761062 (+465682us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:17.087078 (+326016us) log.cc:844] Serialized 14 byte log entry
0709 12:24:17.087332 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.087406 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:17.322930 (+235524us) raft_consensus.cc:1685] finished
0709 12:24:17.322968 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.323214 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":527717,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":6470,"raft.queue_time_us":3859,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":8278,"thread_start_us":707451,"threads_started":2,"wal-append.queue_time_us":624312}
I20260709 12:24:17.899081  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:17.936959  5217 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:17.938711  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:17.939220  5217 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:17.900477  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:17.940181  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:17.881431  8157 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:17.941116  8157 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:17.935825  8032 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.942328  8032 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:17.939237  8118 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.611s	user 0.013s	sys 0.005s
I20260709 12:24:17.905901  8107 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:17.944527  8032 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.945555  8107 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:17.936244  5145 rpcz_store.cc:276] 0709 12:24:16.505293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.778724 (+273431us) service_pool.cc:224] Handling call
0709 12:24:16.779171 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.069147 (+289976us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:17.254212 (+185065us) log.cc:844] Serialized 14 byte log entry
0709 12:24:17.343896 (+ 89684us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.343974 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:17.344036 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.935571 (+591535us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b5400239454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:17.935619 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.935742 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6654,"prepare.queue_time_us":230153,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":6785,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":15879,"spinlock_wait_cycles":665901440,"thread_start_us":260140,"threads_started":2,"wal-append.queue_time_us":271807}
W20260709 12:24:17.948632  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.949301  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.916908  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:17.987478  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:17.996788  5944 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:17.919131  5148 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:17.919911  8154 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.892180  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:17.998479  5216 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:17.999996  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:17.921280  8055 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:17.890841  8054 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:17.890987  8095 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.935665  5146 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:17.871570  5067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1092) took 1070 ms. Trace:
I20260709 12:24:17.945120  5221 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:18.021406  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:17.951496  8159 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:24:17.948781  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:18.020977  5067 rpcz_store.cc:276] 0709 12:24:16.801618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.802394 (+   776us) service_pool.cc:224] Handling call
0709 12:24:16.822792 (+ 20398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.621048 (+798256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.871224 (+250176us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b540047e354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:17.871486 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1822315264}
I20260709 12:24:17.953583  8094 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:17.955636  7585 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:17.956780  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1096) took 1666 ms. Trace:
I20260709 12:24:18.027070  5143 rpcz_store.cc:276] 0709 12:24:16.290024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.292089 (+  2065us) service_pool.cc:224] Handling call
0709 12:24:16.292578 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.529042 (+1236464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.956477 (+427435us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54007aadd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:17.956691 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2812403968}
I20260709 12:24:17.957991  8089 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:17.958398  8104 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:17.944299  8157 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:17.960659  8151 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:17.961977  8059 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:17.969034  5072 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:17.970130  8118 consensus_queue.cc:1059] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:17.970830  5189 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:17.882035  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:18.038298  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 156321us
I20260709 12:24:18.040910  8055 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:17.975627  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:17.915552  8069 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:18.041829  5141 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:18.023710  8095 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:18.043251  8069 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:18.002456  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.001777  7788 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:18.043310  8098 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:18.013286  5114 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:18.015784  5148 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:18.018177  5142 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:18.053225  5114 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:17.941372  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.054972  5075 raft_consensus.cc:2410] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:18.022809  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:17.935627  8158 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:18.025908  5945 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:18.030104  5219 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:18.030788  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:18.029119  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:17.999174  8117 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.032691  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:18.000275  8032 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.059319  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.020913  8049 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:18.046320  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 75568.7us
I20260709 12:24:18.046953  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.067746  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:18.011590  5150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 528) took 1717 ms (client timeout 1762 ms). Trace:
I20260709 12:24:18.049085  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:18.054965  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:17.957373  8107 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:18.066960  5217 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:18.068037  5218 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:18.070549  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:18.070926  5939 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:24:18.068706  5150 rpcz_store.cc:269] 0709 12:24:16.293904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.372566 (+ 78662us) service_pool.cc:224] Handling call
0709 12:24:18.005127 (+1632561us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54006e2154. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:18.011522 (+  6395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2636320128}
I20260709 12:24:18.047652  8110 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.093919  5141 raft_consensus.cc:2410] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:24:18.077361  5148 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 12:24:18.094859  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1103) took 1011 ms. Trace:
I20260709 12:24:18.095391  5148 rpcz_store.cc:276] 0709 12:24:17.083613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.052369 (+968756us) service_pool.cc:224] Handling call
0709 12:24:18.077046 (+ 24677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:18.094532 (+ 17486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.094795 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.058059  8094 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:18.097131  5150 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:18.097481  8154 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:18.072762  8118 consensus_queue.cc:1243] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:24:18.081347  8160 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:24:18.076212  8049 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:18.057969  7788 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:18.069842  8089 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:18.068842  8162 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.106871  8089 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.072783  8117 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:18.107260  8049 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:18.109262  8017 raft_consensus.cc:3055] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:18.076419  8032 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:18.109858  8017 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:18.107373  8089 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:18.114313  7789 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.115680  5223 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:24:18.116245  7432 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:18.088310  5145 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:18.090929  5899 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:18.079460  8035 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:18.080531  8097 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:18.116882  7789 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:18.119041  8035 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:18.120528  8138 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.586s	user 0.003s	sys 0.000s
I20260709 12:24:18.120283  8035 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:18.120779  5216 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
W20260709 12:24:18.117302  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.122535  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.122892  8116 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:18.123334  8116 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:18.123744  8116 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:18.123879  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.124738  5216 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:18.125244  7619 raft_consensus.cc:3055] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:18.118680  7788 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:18.125581  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.126578  5216 raft_consensus.cc:2410] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:18.126758  5145 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:18.127781  5941 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:18.109423  8117 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:18.129832  8097 raft_consensus.cc:697] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:18.130119  8095 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.130937  8134 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:18.130853  8097 consensus_queue.cc:237] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:18.098131  5148 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:18.107178  8162 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:18.059288  8098 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.136972  8098 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:18.138446  8095 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:18.087530  7705 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:18.093945  8110 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.142679  8110 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:18.087718  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.145146  5143 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:18.064456  8136 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:18.148017  5224 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:18.123901  8062 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:18.153334  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.162443  5224 raft_consensus.cc:2393] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:18.057461  8072 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:18.259353  8136 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:18.150151  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 712) took 1143 ms. Trace:
I20260709 12:24:18.262059  5076 raft_consensus.cc:3055] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:18.261200  5946 rpcz_store.cc:276] 0709 12:24:17.006385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.006561 (+   176us) service_pool.cc:224] Handling call
0709 12:24:17.044739 (+ 38178us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:17.045293 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.045337 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:17.063176 (+ 17839us) log.cc:844] Serialized 52 byte log entry
0709 12:24:17.332929 (+269753us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:17.332975 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.333044 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.119709 (+786665us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:18.144992 (+ 25283us) raft_consensus.cc:1685] finished
0709 12:24:18.145025 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.150057 (+  5032us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":813,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":7840,"spinlock_wait_cycles":17033088,"thread_start_us":240428,"threads_started":1,"wal-append.queue_time_us":410195}
I20260709 12:24:18.262477  5076 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:18.263279  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.264366  5946 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:18.144178  7619 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:18.266130  7619 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:18.267175  7619 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:18.110882  8017 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:18.267001  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1104) took 1177 ms. Trace:
I20260709 12:24:18.175130  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 506) took 1117 ms. Trace:
I20260709 12:24:18.270989  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:18.271239  5221 rpcz_store.cc:276] 0709 12:24:17.057700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.057829 (+   129us) service_pool.cc:224] Handling call
0709 12:24:17.072959 (+ 15130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.021778 (+948819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.174730 (+152952us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b5400114254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:18.175036 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2004673920}
I20260709 12:24:18.270772  5142 rpcz_store.cc:276] 0709 12:24:17.090055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.053345 (+963290us) service_pool.cc:224] Handling call
0709 12:24:18.053829 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.100485 (+ 46656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:18.100518 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:18.100567 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:18.100601 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.266645 (+166044us) spinlock_profiling.cc:243] Waited 46.4 ms on lock 0x7b54003421d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:18.266709 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.266919 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106806144}
I20260709 12:24:18.184552  5222 raft_consensus.cc:3055] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:18.274868  5222 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:18.276073  5222 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.185761  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1101) took 1116 ms. Trace:
I20260709 12:24:18.263306  5076 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:18.278512  5076 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:18.277732  5147 rpcz_store.cc:276] 0709 12:24:17.069353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.098907 (+ 29554us) service_pool.cc:224] Handling call
0709 12:24:17.099343 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.413936 (+314593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.413972 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.414018 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:17.414056 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.185466 (+771410us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b540036ef54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:18.185518 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.185680 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722809728}
I20260709 12:24:18.187671  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.192893  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1099) took 1133 ms. Trace:
I20260709 12:24:18.149001  5216 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:18.282059  5223 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:24:18.282500  5223 pending_rounds.cc:85] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 5
I20260709 12:24:18.282720  5223 pending_rounds.cc:107] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:24:18.282939  5223 raft_consensus.cc:2889] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:18.284195  5216 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:18.185768  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1105) took 1074 ms. Trace:
I20260709 12:24:18.197580  5943 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [7.6-9.7]
W20260709 12:24:18.285686  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.279740  5144 rpcz_store.cc:276] 0709 12:24:17.059229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.059421 (+   192us) service_pool.cc:224] Handling call
0709 12:24:17.081422 (+ 22001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.081982 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:17.316998 (+235016us) log.cc:844] Serialized 14 byte log entry
0709 12:24:17.930729 (+613731us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b18001bbce0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:17.982948 (+ 52219us) spinlock_profiling.cc:243] Waited 8.37 ms on lock 0x7b18001bbce0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e557 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:18.191672 (+208724us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b18001bbce0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a80e34bc 00007fd8a543148b 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db23a
0709 12:24:18.192139 (+   467us) log.cc:844] Serialized 14 byte log entry
0709 12:24:18.192368 (+   229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:18.192491 (+   123us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:18.192590 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.192682 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.192824 (+   142us) inbound_call.cc:177] Queueing success response
0709 12:24:18.220904 (+ 28080us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b18001bbce0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:18.258385 (+ 37481us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b18001bbce0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a541c2a6 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e557 00007fd8a54507c1
Metrics: {"mutex_wait_us":9350,"prepare.queue_time_us":260762,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":9400,"raft.queue_time_us":3844,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":271,"spinlock_wait_cycles":91776000,"thread_start_us":293665,"threads_started":2,"wal-append.queue_time_us":106396}
I20260709 12:24:18.200826  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.202878  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.203337  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.203812  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:18.285068  5143 rpcz_store.cc:276] 0709 12:24:17.110992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.063837 (+952845us) service_pool.cc:224] Handling call
0709 12:24:18.064167 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.145870 (+ 81703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.185565 (+ 39695us) spinlock_profiling.cc:243] Waited 80.7 ms on lock 0x7b540022ea54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:18.185710 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185611520}
I20260709 12:24:18.206856  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:18.212814  8030 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.217298  5114 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:18.217589  8116 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:18.218071  8110 consensus_queue.cc:1059] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:18.289463  5215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:18.290385  5215 raft_consensus.cc:606] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:18.289825  8030 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.290756  5215 raft_consensus.cc:614] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:18.291008  8110 raft_consensus.cc:3055] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:18.291383  8110 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:18.289499  8095 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:18.218528  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.220034  8089 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:18.300034  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.314502  5943 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:18.224457  8107 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:18.226222  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.230778  7896 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:18.234339  8032 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:18.235718  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.236963  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:18.337311  5219 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:18.242247  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 988) took 1736 ms. Trace:
I20260709 12:24:18.338039  5149 rpcz_store.cc:276] 0709 12:24:16.505723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.782168 (+276445us) service_pool.cc:224] Handling call
0709 12:24:17.312832 (+530664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.335747 (+ 22915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.241966 (+906219us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54004fae50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:18.242161 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211274112}
I20260709 12:24:18.253225  8162 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:18.254110  8054 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:18.260296  8136 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:18.270371  8017 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:18.174796  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:18.285214  5223 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:18.292752  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 75567.3us
I20260709 12:24:18.273238  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:18.296342  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 89610.2us
W20260709 12:24:18.296954  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 256840us
W20260709 12:24:18.375970  5041 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.68:43449 timed out after 2.118s (SENT) [suppressed 19 similar messages]
I20260709 12:24:18.298900  5946 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:18.302570  5042 leader_election.cc:400] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7.
I20260709 12:24:18.315567  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:18.318441  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:18.378008  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 75630.5us
I20260709 12:24:18.319278  5222 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:18.378660  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.342418  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.348884  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.379644  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:18.380641  5219 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:18.349749  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.352897  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.381809  5141 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:18.367026  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:18.382815  5144 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:18.368012  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:18.367158  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:18.368835  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:18.376704  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:18.330339  5216 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:18.378046  5943 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:18.330093  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:18.379492  5942 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:18.379943  5218 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:18.391434  5189 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:18.393392  8163 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:18.395197  8014 raft_consensus.cc:3055] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:18.395772  8014 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:18.397894  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:18.399925  8118 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:18.402566  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:18.407099  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.414956  8163 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:18.422876  8154 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:18.421048  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.336671  7705 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:18.432603  4950 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 54285.3us
I20260709 12:24:18.445996  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:18.432585  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 53779.9us
I20260709 12:24:18.347978  7982 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:18.350879  8109 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.320001  7896 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:18.448253  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:18.466565  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:18.472829  5147 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:18.462405  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:18.473062  5076 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:24:18.458036  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 128018us
I20260709 12:24:18.472829  5219 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:18.476720  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.479950  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.480942  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.545158  5218 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:18.546581  5218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:18.547452  5218 raft_consensus.cc:606] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:18.547889  5218 raft_consensus.cc:614] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:18.548840  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.478142  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:18.484090  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:18.550401  5943 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:18.445398  7705 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:18.497718  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 121804us
I20260709 12:24:18.498731  5948 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:18.554983  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 539) took 1107 ms. Trace:
I20260709 12:24:18.555964  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.555600  5141 rpcz_store.cc:276] 0709 12:24:17.447721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.105394 (+657673us) service_pool.cc:224] Handling call
0709 12:24:18.317299 (+211905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.382375 (+ 65076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.554679 (+172304us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b540012d4d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:18.554903 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147786496}
I20260709 12:24:18.557003  5219 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:18.555406  5220 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:18.501039  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.508329  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.432603  8134 consensus_queue.cc:1059] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:24:18.475092  8107 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:18.526255  8158 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.517685  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.483074  7698 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:18.522189  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.520780  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:18.526786  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.527066  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.476200  8168 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:18.528528  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:18.603406  5075 raft_consensus.cc:2410] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:18.603549  5221 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:18.603920  5217 raft_consensus.cc:2393] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:18.495980  7924 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:24:18.352201  8110 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:18.530781  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.608808  5067 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:18.467528  7982 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:18.549165  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:18.554464  5041 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:18.574004  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.474046  8109 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:18.576246  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.567070  5223 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:18.516430  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.618955  5068 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:18.619210  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.591382  5943 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:24:18.620309  5075 raft_consensus.cc:2410] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:18.593099  8072 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:18.596019  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.599529  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:18.624339  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.625028  5071 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:18.592326  5899 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:18.615216  5219 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:24:18.616070  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 170390us
I20260709 12:24:18.616631  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:18.579217  7705 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:18.632687  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:18.587765  8107 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:18.631062  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:18.631930  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 77582.3us
I20260709 12:24:18.634012  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:18.626322  5899 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:18.634850  5069 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:18.629048  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:18.641968  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.644039  5069 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:18.612149  8030 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:18.639569  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.607124  8168 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.628485  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:18.649995  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 151609us
I20260709 12:24:18.590382  7975 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:24:18.650996  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:18.658123  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.655860  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.626042  8094 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.665496  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.666533  5072 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:18.673197  7982 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:18.674901  8109 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:18.676091  5067 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:24:18.677356  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.533239  7896 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.681921  8089 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:18.636586  8032 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:18.637893  8117 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:18.683287  8032 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:18.683823  8032 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:18.683758  8117 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:18.685163  5075 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:18.686689  8117 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:18.687350  8094 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.690901  8176 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:18.646492  8107 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:18.691516  8035 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:18.607476  8098 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.694761  8098 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:18.696133  5939 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:18.691524  8176 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:18.647863  8055 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:18.682552  8054 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:18.698377  8054 raft_consensus.cc:697] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:18.698408  8176 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.699465  8054 consensus_queue.cc:237] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.607190  8158 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.646620  8162 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.682070  7896 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:18.666220  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:18.720330  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.666591  8134 raft_consensus.cc:3055] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:18.785241  8110 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:18.789750  8134 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:18.791033  8134 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:18.793677  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.738806  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:18.738698  8035 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:18.749451  5945 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:18.754609  8014 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:18.764895  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.782972  5217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:18.657712  8168 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:18.798424  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 132408us
W20260709 12:24:18.720559  8016 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:24:18.803416  8035 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:18.803951  8035 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:18.806295  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:18.807909  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 69186.9us
I20260709 12:24:18.811108  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:18.816623  8049 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:18.817225  8049 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.817696  8049 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:18.820132  8136 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:18.838558  8098 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:18.840693  8157 raft_consensus.cc:3055] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:18.841262  8157 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:18.842444  8157 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:18.843973  8157 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:18.809823  8017 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:18.821396  8179 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:18.815055  8094 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:18.860569  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 234344us
I20260709 12:24:18.825727  8014 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:18.862815  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 228881us
I20260709 12:24:18.721402  8162 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:18.804996  8035 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.874688  5114 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:18.870645  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:18.881388  5222 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:18.881796  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:18.858405  8030 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:18.865828  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 233233us
W20260709 12:24:18.866272  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 217660us
I20260709 12:24:18.876993  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:18.828239  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:18.869715  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.865198  8179 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.891868  5041 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:18.899343  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:18.900202  5224 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:18.896461  8180 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:18.852962  8136 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:24:18.913430  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:18.913898  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:18.918841  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 90703.1us
I20260709 12:24:18.919984  5944 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [7.6-9.7]
W20260709 12:24:18.883663  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.895695  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:18.917347  5898 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:18.926055  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:18.896624  8072 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:18.936419  5070 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:18.944233  7432 raft_consensus.cc:3055] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:18.901787  8179 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:18.865907  8107 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:18.901523  7788 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:18.937235  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:18.919265  8035 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:18.950739  7432 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:18.931926  7982 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:18.985321  5224 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
W20260709 12:24:18.986443  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 69195.5us
I20260709 12:24:18.987453  5898 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:18.954466  8180 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:18.988217  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:18.906401  7705 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:18.957980  7867 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:19.008114  8188 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:18.982465  8075 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.008994  8188 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.009812  8075 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:18.970963  7698 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:24:19.013181  8107 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.014029  8107 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:19.018760  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:19.004216  7867 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.020483  7867 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:19.020830  5898 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:19.021852  8136 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:19.022356  8136 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.022714  8136 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:19.023119  8189 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:19.024017  8189 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:19.024500  8189 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:19.024370  8190 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:19.026902  8189 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:18.984151  8163 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.028996  8162 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:18.969270  8072 raft_consensus.cc:697] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:19.031145  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:19.056490  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:19.000332  8180 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.071180  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:19.067467  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:18.992264  8176 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:19.052232  5150 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:19.091197  8116 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.048439  8017 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:19.075352  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 155676us
I20260709 12:24:19.094197  8017 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:19.094813  8134 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:19.091176  8176 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:19.096596  7975 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:19.096614  8176 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:19.099311  8098 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:19.099812  8098 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:19.100956  8098 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.102494  8098 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:19.107300  8176 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:19.069347  8188 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:19.112112  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 186646us
I20260709 12:24:19.073702  8180 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:19.070490  8107 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:19.121464  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:19.077579  8162 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:19.123664  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:19.122257  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:19.128139  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:19.128583  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.205096  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:19.124534  8158 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:19.066121  8163 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.130143  8162 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:19.126883  8107 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:19.202379  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 228882us
W20260709 12:24:19.182451  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.220080  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.198460  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.197255  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 75808.7us
I20260709 12:24:19.224416  5939 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:19.122257  8017 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.228945  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:19.200026  8084 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:19.230098  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:19.124671  8193 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 12:24:19.231915  5041 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:19.248509  5041 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:19.235355  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:19.248859  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:19.125268  8089 consensus_queue.cc:1059] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 12:24:19.245615  8136 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:19.253188  8194 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:24:19.231168  8162 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:19.238528  5224 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
W20260709 12:24:19.238796  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 118319us
I20260709 12:24:19.237617  5219 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:19.240115  5898 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:19.084733  7432 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:19.251576  5041 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:19.253050  8125 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:19.236364  8107 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:19.254987  8205 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:19.261927  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:19.262120  8117 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:19.262466  8202 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:19.261648  4895 catalog_manager.cc:5697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.263051  8089 raft_consensus.cc:3055] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:19.263137  8188 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:19.237735  8191 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.280129  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:19.280992  5076 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:19.271911  5115 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:19.269212  5041 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:19.250679  8097 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:19.272374  8089 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:19.257704  8151 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:19.280886  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.282888  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:19.238510  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.287742  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.295568  5074 raft_consensus.cc:2410] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:19.297451  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.286288  4895 catalog_manager.cc:5697] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 2 to 6, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 6 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:19.298954  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:19.299651  8059 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.300386  5068 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:19.302017  8151 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.302515  8151 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:19.303355  8163 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:19.300236  8059 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:19.305184  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:19.228300  8072 consensus_queue.cc:237] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:19.306768  4897 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43222 (request call id 142) took 13899 ms (client timeout 15000 ms). Trace:
I20260709 12:24:19.307955  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.245987  8017 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.312876  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.308673  5069 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:19.273741  8030 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:19.315913  5070 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:19.293340  7924 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.032s
W20260709 12:24:19.324971  8032 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.641s	user 0.007s	sys 0.005s
I20260709 12:24:19.326819  4895 catalog_manager.cc:5697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 4 to 7, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 7 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:19.328466  4895 catalog_manager.cc:5697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 4 to 5, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:19.315429  8202 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:19.329764  8030 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.330332  8030 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:19.307340  7705 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:19.330681  8202 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:19.331588  8199 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:19.275803  8080 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:24:19.293483  8117 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:19.332858  8117 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.333292  8117 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:19.336280  8017 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:19.292936  8191 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.339726  8089 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.341738  8084 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:19.342841  8110 consensus_meta.cc:287] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.003s
I20260709 12:24:19.343539  5217 raft_consensus.cc:606] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:24:19.344058  5217 raft_consensus.cc:614] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:19.342522  8084 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.341949  8109 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:19.347348  8084 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:19.307231  4897 rpcz_store.cc:269] 0709 12:24:05.442876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.443304 (+   428us) service_pool.cc:224] Handling call
0709 12:24:19.306669 (+13863365us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:14.631551 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:14.632503 (+   952us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:14.632570 (+    67us) write_op.cc:435] Acquired schema lock
0709 12:24:14.632597 (+    27us) tablet.cc:662] Decoding operations
0709 12:24:14.634965 (+  2368us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:14.635078 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:14.635120 (+    42us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:24:14.637749 (+  2629us) tablet.cc:701] Row locks acquired
0709 12:24:14.637810 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:24:14.638065 (+   255us) write_op.cc:270] Start()
0709 12:24:14.638289 (+   224us) write_op.cc:276] Timestamp: P: 1783599854637963 usec, L: 0
0709 12:24:14.638314 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:24:14.786106 (+147792us) log.cc:844] Serialized 7849 byte log entry
0709 12:24:15.686447 (+900341us) op_driver.cc:464] REPLICATION: finished
0709 12:24:15.956367 (+269920us) write_op.cc:301] APPLY: starting
0709 12:24:15.956506 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:15.956955 (+   449us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:15.957162 (+   207us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:15.982751 (+ 25589us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:16.070590 (+ 87839us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0709 12:24:16.114072 (+ 43482us) write_op.cc:312] APPLY: finished
0709 12:24:16.214998 (+100926us) log.cc:844] Serialized 138 byte log entry
0709 12:24:16.215322 (+   324us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:16.215627 (+   305us) write_op.cc:454] Released schema lock
0709 12:24:16.215737 (+   110us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:16.216348 (+   611us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:16.216387 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":910685,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":269030,"apply.run_cpu_time_us":9573,"apply.run_wall_time_us":259958,"num_ops":20,"prepare.queue_time_us":106075,"prepare.run_cpu_time_us":23740,"prepare.run_wall_time_us":655586,"raft.queue_time_us":260381,"raft.run_cpu_time_us":9714,"raft.run_wall_time_us":221876,"replication_time_us":1047857,"thread_start_us":783404,"threads_started":4,"wal-append.queue_time_us":552439,"wal-append.run_cpu_time_us":49258,"wal-append.run_wall_time_us":1860203}]]}
I20260709 12:24:19.307199  8059 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:19.358179  5939 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
W20260709 12:24:19.358588  8049 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.005s
I20260709 12:24:19.347611  5041 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:19.368010  5188 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:19.369215  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 49759.4us
I20260709 12:24:19.381453  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:19.375934  8089 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:19.398056  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:19.399572  5224 raft_consensus.cc:2410] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:24:19.394570  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:19.401067  5219 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:19.413462  5076 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:19.380787  8055 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:19.359930  8191 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.379786  8017 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:19.380508  8136 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:19.380779  8188 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:19.381476  8134 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:19.426431  4895 catalog_manager.cc:5697] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 5 to 7, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 7 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:19.382121  8204 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:24:19.360843  8032 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:19.420163  7896 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.003s
I20260709 12:24:19.382501  8125 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 12:24:19.463274  8168 consensus_meta.cc:287] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.003s
I20260709 12:24:19.464087  8168 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:19.426419  8162 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:19.465500  8107 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:19.472849  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.469411  8162 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.470323  5071 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:19.482365  8168 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:19.420680  8179 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.440250  7788 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:19.443028  8186 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:19.419957  7698 raft_consensus.cc:2764] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:24:19.423316  8180 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:19.447386  8032 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:19.431588  8134 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.431167  8017 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:19.433831  8188 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:19.420228  8176 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.464934  7867 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.380735  8049 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.489012  8163 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:19.494262  5899 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:19.555086  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:19.555802  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 61685us
I20260709 12:24:19.496860  5224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:19.556596  5224 raft_consensus.cc:606] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:19.439500  8075 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.498662  5189 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:19.496201  8207 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.194s	user 0.005s	sys 0.000s
I20260709 12:24:19.504667  5219 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 12:24:19.509829  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 227712us
I20260709 12:24:19.541460  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:19.535068  4895 catalog_manager.cc:5697] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 4 to 6, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 6 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.567970  5898 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:19.473420  8107 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:19.555925  5217 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:19.557842  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 59305.7us
I20260709 12:24:19.534929  7619 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:19.571134  5939 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
W20260709 12:24:19.489327  8200 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.191s	user 0.000s	sys 0.009s
I20260709 12:24:19.545413  7432 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:19.577500  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 305643us
I20260709 12:24:19.578919  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:19.567289  5114 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:19.550032  8134 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:19.595448  4895 catalog_manager.cc:5697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 3 to 6, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 6 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:19.600071  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:19.600878  5067 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:19.603035  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:19.560184  8097 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.605108  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.581888  8072 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:19.609225  5073 raft_consensus.cc:2368] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11.
I20260709 12:24:19.585800  7788 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:19.612748  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.573799  7975 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:19.613612  5076 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:19.608215  7982 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.583739  8162 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:24:19.626037  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 185334us
I20260709 12:24:19.608815  8193 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:19.622107  8080 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:24:19.573644  8163 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:19.629383  7982 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:19.641003  8179 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:19.642373  7982 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.643472  8151 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:19.646194  8032 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.646812  7867 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:19.637404  8206 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:19.649101  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:19.651895  8154 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:19.654287  8154 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:19.654788  5222 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:19.656530  8154 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.662899  8180 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:19.591863  8201 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 12:24:19.636889  5900 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.762s (SENT)
I20260709 12:24:19.664152  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:19.665024  8136 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:19.614077  8098 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:19.673909  8098 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:19.615003  8072 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.622393  7975 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.629597  8059 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:19.646836  8032 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:19.632102  4895 catalog_manager.cc:5697] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:24:19.650916  8151 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.651605  8014 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:19.634934  8162 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:19.547616  8186 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:19.679378  5222 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:19.611640  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:19.598105  7619 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:19.617359  8097 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:19.699334  8097 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:19.698793  4895 catalog_manager.cc:5697] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 4 to 6, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 6 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:19.700418  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.700534  8014 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:19.701084  8014 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:19.615312  8107 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:24:19.700642  4895 catalog_manager.cc:5697] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:24:19.648805  8206 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.697072  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:19.636047  8193 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:19.711992  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.685853  8168 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.684100  7975 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:19.628507  7788 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:19.708690  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.752435  5114 leader_election.cc:400] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11.
I20260709 12:24:19.752671  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.752933  5114 leader_election.cc:403] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:24:19.752750  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:19.753614  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.753741  5069 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:19.709153  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.709735  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:19.685099  8072 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:19.702718  8032 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.712388  4895 catalog_manager.cc:5697] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 2 to 3, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 3 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:19.771035  8097 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:19.711256  8080 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:19.775130  5071 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:19.712736  5073 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:19.688323  8095 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.758833  8154 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:19.760217  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50575.2us
I20260709 12:24:19.778461  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:19.705137  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 93629.4us
W20260709 12:24:19.723654  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.756234  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.782976  5115 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:19.689841  8059 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:19.693431  8117 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:19.785106  5115 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:19.783042  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 358653us
I20260709 12:24:19.788787  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:19.790676  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:19.735352  8193 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:19.788188  5041 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:19.768380  8209 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:19.763849  7867 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:19.805442  5141 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:19.805495  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:19.805912  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:19.702044  8014 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:19.814117  4895 catalog_manager.cc:5697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:19.807580  5074 raft_consensus.cc:2368] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11.
I20260709 12:24:19.811481  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:19.816672  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:19.813232  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:19.817816  5142 raft_consensus.cc:2410] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:19.819350  5945 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:19.822198  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:19.823856  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:19.824679  5943 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:19.827260  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:19.827880  5939 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:19.831537  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:19.832441  5948 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:19.817665  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:19.794057  8097 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:19.795794  8201 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:19.840926  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.843737  5222 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:19.847551  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:19.839015  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:19.848007  5147 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:19.886379  5141 raft_consensus.cc:2410] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:24:19.798187  8179 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:19.850642  5939 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:19.714025  8030 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:19.810092  8202 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:19.802577  7705 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:19.800523  8163 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:19.876921  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:19.877885  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:19.909080  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:19.879030  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:19.763701  8168 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:19.809029  8186 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:19.786935  8095 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:19.892091  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 53175.1us
I20260709 12:24:19.893782  5945 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: [9.6-10.7]
I20260709 12:24:19.891801  5217 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:19.901288  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:19.918561  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.919555  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:19.902125  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:19.793377  8154 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:19.923954  5940 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:24:19.906252  4895 catalog_manager.cc:5697] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 2 to 4, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 4 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:19.924850  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:19.902884  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:19.835043  8049 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:19.781881  8162 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:19.910152  5146 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
W20260709 12:24:19.914822  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 149630us
I20260709 12:24:19.919967  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:19.901648  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:19.893077  8201 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.929800  5147 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:19.897655  8070 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:19.932281  5142 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:24:19.940235  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.843235  8125 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:24:19.903877  8202 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.903982  7867 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:19.964008  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:19.964946  5146 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:19.936640  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:19.932897  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:19.938365  5899 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:19.909592  7896 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:19.972461  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.958045  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:19.959944  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:19.989494  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.990317  5072 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:19.991292  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.977470  5067 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:19.931952  8205 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:19.991695  5898 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:19.938406  8201 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:19.933511  7924 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:20.013104  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:19.938163  8017 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:19.967347  8202 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:20.018690  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:20.018877  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:20.024302  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 98098.8us
I20260709 12:24:20.030088  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.031437  5150 raft_consensus.cc:2410] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:24:19.968600  7619 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.993814  8189 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:19.990109  8163 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:20.003842  7867 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:19.978657  8098 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:19.958341  8070 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:20.018021  8190 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.054826  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.055266  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.064010  5142 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:20.055946  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:19.989095  7896 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:20.056210  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.039376  8017 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:20.058341  5148 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:20.067008  5141 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 9: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:20.054921  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.070220  5147 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:20.070643  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 79053.4us
W20260709 12:24:20.073346  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.075539  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:20.082108  8221 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
W20260709 12:24:20.085379  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.085858  8227 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.086580  8227 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:20.047894  7619 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:20.090404  5939 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:20.090821  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.050487  8189 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:20.094709  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.068913  8017 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.096787  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.061674  7788 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:20.066346  8098 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.105015  5142 raft_consensus.cc:2410] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:24:20.061918  8230 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:20.066301  8206 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:20.106696  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.095722  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.116441  5147 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:20.116915  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:20.118774  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:20.130554  5899 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:20.093775  8136 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:20.069119  8190 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:20.145301  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.147084  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.152058  5147 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:20.152936  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.155571  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:20.156261  5941 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:20.142772  5948 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:20.148413  5141 raft_consensus.cc:2410] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:24:20.157296  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:20.159543  5941 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:20.117681  8017 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:20.163477  5939 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:20.157733  5189 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:20.157871  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:20.165302  5946 raft_consensus.cc:2393] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:20.167258  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.167397  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.099568  7619 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:20.126641  8059 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:20.170106  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 83781.9us
I20260709 12:24:20.133109  8107 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:20.171773  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.172986  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.174542  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.174769  5144 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:20.160984  8035 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.177053  5143 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:24:20.179057  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.179934  5148 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 9: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:20.145949  8075 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:20.183691  5141 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:20.186081  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:20.175541  8220 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 12:24:20.186450  5069 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:20.188480  5946 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:24:20.190351  5143 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:20.195185  8225 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:24:20.196451  8229 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:24:20.190819  8075 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:20.195230  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.199425  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.200117  8075 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:20.200614  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.156394  8098 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:20.202386  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.203482  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.203626  8098 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:20.204211  8098 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:20.204164  5141 raft_consensus.cc:2368] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:20.197856  8095 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:20.207271  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.211126  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:20.207347  5898 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:20.214021  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:20.216482  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:20.242348  5143 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:20.242904  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.244891  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 156517us
I20260709 12:24:20.247709  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.251367  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.161310  8206 consensus_queue.cc:1059] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:24:20.266604  5042 leader_election.cc:400] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:20.268122  5042 leader_election.cc:403] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:24:20.255048  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:20.272951  5946 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:20.273608  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:20.204948  8098 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:20.161379  8136 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:20.273138  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:20.272501  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:20.297761  5076 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:20.187161  8168 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:20.288534  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:20.302597  5071 raft_consensus.cc:2393] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:20.288723  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:20.304042  5948 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:20.290395  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:20.239612  8219 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 12:24:20.291548  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:20.180399  8035 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:20.323195  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 61189.3us
I20260709 12:24:20.160616  7867 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:20.328079  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 300590us
I20260709 12:24:20.276602  8154 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:20.301396  5190 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:20.258380  8158 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:20.287673  7698 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:20.290637  8238 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:20.330375  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:20.332013  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 65599us
I20260709 12:24:20.378578  5042 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:20.307363  8107 raft_consensus.cc:697] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:20.339404  5900 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:20.367985  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.207720  8049 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 12:24:20.370801  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:20.372821  5143 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:20.369807  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:20.380118  5150 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:20.384891  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:20.390100  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 97679us
I20260709 12:24:20.184234  8189 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:20.392645  5146 raft_consensus.cc:2410] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 12:24:20.382840  8224 log.cc:927] Time spent T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
I20260709 12:24:20.364770  7698 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:20.416841  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:20.389900  8176 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:20.332547  8095 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 12:24:20.435449  8176 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:20.439527  7975 consensus_meta.cc:287] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.002s
I20260709 12:24:20.440119  7975 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:20.442835  7975 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:20.448894  8117 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:20.374727  8242 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:20.431516  8080 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:20.433019  8098 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:20.434499  8134 consensus_meta.cc:287] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.818s	user 0.013s	sys 0.000s
I20260709 12:24:20.374365  8158 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:20.390233  8070 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:20.393784  8237 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:20.458628  7619 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:20.495246  8134 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:20.420704  7698 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:20.395829  7705 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:24:20.398885  8089 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:20.461392  8250 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:20.425881  8179 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
W20260709 12:24:20.416275  8228 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.353s	user 0.002s	sys 0.000s
I20260709 12:24:20.400561  8094 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:20.436880  8176 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:20.413041  8236 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:20.495966  5948 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:20.434031  8107 consensus_queue.cc:237] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:20.524505  8014 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:20.526292  8250 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:20.514649  8089 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:20.565526  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 165545us
I20260709 12:24:20.526329  8227 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:20.552065  8202 consensus_meta.cc:287] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.003s
I20260709 12:24:20.539971  8158 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:20.574093  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 157351us
W20260709 12:24:20.564136  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 356825us
W20260709 12:24:20.563002  8249 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:24:20.576053  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:20.546938  8190 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:20.576131  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 237315us
I20260709 12:24:20.553579  8250 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:20.580127  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:20.581113  5144 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:20.565008  8176 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:20.565366  8089 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:20.582597  5075 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 629) took 1005 ms. Trace:
I20260709 12:24:20.585247  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:20.590010  5041 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:20.586889  5075 rpcz_store.cc:276] 0709 12:24:19.576940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.577170 (+   230us) service_pool.cc:224] Handling call
0709 12:24:19.577759 (+   589us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:19.584398 (+  6639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.584447 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:19.585286 (+   839us) log.cc:844] Serialized 52 byte log entry
0709 12:24:19.656293 (+ 71007us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b4800001a58. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ad464aa1 00007fd8ad45bb37 00007fd8ad45b9ca 00007fd8ad55326f 00007fd8ad552f37 00007fd8ad56755c 00007fd8ad567508 00007fd8ad567488 00007fd8ad567443 00007fd8ad5666ec 00007fd8ac490a96
0709 12:24:19.894758 (+238465us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:19.894801 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.894874 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.577911 (+683037us) raft_consensus.cc:1685] finished
0709 12:24:20.577938 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.582502 (+  4564us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6039,"raft.queue_time_us":7289,"raft.run_cpu_time_us":2692,"raft.run_wall_time_us":63941,"spinlock_wait_cycles":11882240,"thread_start_us":268843,"threads_started":1,"wal-append.queue_time_us":362614}
I20260709 12:24:20.587589  8035 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:20.596616  8134 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:20.600770  7432 consensus_meta.cc:287] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.994s	user 0.012s	sys 0.000s
I20260709 12:24:20.602041  5939 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:20.581841  8094 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:20.605967  8094 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:20.606865  8188 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:20.606828  5114 leader_election.cc:400] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11.
I20260709 12:24:20.608986  5114 leader_election.cc:403] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:24:20.610154  5071 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:20.611282  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:20.613126  5069 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:20.588086  8136 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:20.614423  8072 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.843s	user 0.014s	sys 0.000s
I20260709 12:24:20.577111  8202 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:20.611236  8244 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.608227  8125 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:20.608570  8242 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:24:20.631335  8242 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:20.631770  8242 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:20.618820  7705 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:20.632750  8242 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:20.641085  8242 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:20.627108  8250 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:20.607000  8094 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:20.647390  8201 consensus_meta.cc:287] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.624s	user 0.012s	sys 0.000s
I20260709 12:24:20.617187  8262 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11.
I20260709 12:24:20.688194  5147 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:20.692436  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 391145us
I20260709 12:24:20.693337  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:20.694192  5947 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:20.694094  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:20.621886  8179 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:24:20.694831  5215 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:24:20.699246  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:20.685151  8250 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:20.708385  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 91082us
I20260709 12:24:20.709800  5215 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:20.627220  8072 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:20.713941  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:20.702852  8209 consensus_meta.cc:287] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.894s	user 0.012s	sys 0.003s
I20260709 12:24:20.718434  5215 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:20.695266  7705 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:20.652940  8216 consensus_queue.cc:1175] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9847a7797ddc4df689a1ab0911af8785" responder_term: 11 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: [9.6-10.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:20.700625  8255 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.699965  8154 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:20.726845  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:20.712247  8193 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.905s	user 0.007s	sys 0.010s
I20260709 12:24:20.727213  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:20.726497  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:20.724335  5899 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:20.715778  8163 consensus_meta.cc:287] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.668s	user 0.005s	sys 0.009s
I20260709 12:24:20.723304  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:20.683776  8201 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:20.739077  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:20.731431  5076 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:20.746186  5218 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:20.716611  8179 consensus_queue.cc:1243] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:20.747359  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:20.749315  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:20.753366  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:20.754935  8158 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.755697  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 62450.5us
I20260709 12:24:20.763358  5217 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:20.756240  8134 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:20.757865  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:20.759274  8072 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:20.764909  5221 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:20.728363  8254 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:20.747856  8163 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:20.771539  5224 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:20.741338  8193 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:20.664772  8244 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:20.736922  8255 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:20.771965  8158 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:20.772768  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:20.775553  5141 raft_consensus.cc:2368] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:20.776896  8201 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:20.733255  8209 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11.
I20260709 12:24:20.733872  8154 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:20.782681  8269 log.cc:927] Time spent T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:24:20.788271  8017 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.004s
I20260709 12:24:20.792817  5223 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:20.797761  5073 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:20.798496  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:20.799175  8070 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:20.816354  8030 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:20.821581  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:20.920419  5219 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:20.840173  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.865694  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 156497us
I20260709 12:24:20.798540  8241 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.739004  8204 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:20.869211  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 711553us
W20260709 12:24:20.869042  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 91153.7us
I20260709 12:24:20.719699  8252 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:20.928037  5042 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:20.931660  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 133266us
I20260709 12:24:20.890900  5076 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:20.891897  5217 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:20.932974  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:20.884078  8274 raft_consensus.cc:3055] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:20.902456  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:20.936514  5221 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:20.902439  5070 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:20.931499  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:20.905642  8158 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:20.905642  7705 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:20.906505  7698 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:20.911372  8193 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:20.912353  5218 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:20.913064  5145 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:20.885581  8097 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:24:20.882005  8213 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:20.902779  8154 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:20.891788  8230 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:20.940848  5215 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:20.939776  8216 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:20.942749  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1107) took 1138 ms. Trace:
I20260709 12:24:20.944304  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:20.950728  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:20.930179  8201 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.938140  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:20.971065  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:20.976732  5215 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:20.967705  8206 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.029s	sys 0.019s
I20260709 12:24:20.975276  5149 rpcz_store.cc:276] 0709 12:24:19.804495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.804654 (+   159us) service_pool.cc:224] Handling call
0709 12:24:19.846338 (+ 41684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.910945 (+ 64607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.910997 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.911062 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:19.911128 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.942516 (+1031388us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b54002de6d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:20.942568 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.942693 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147876736}
I20260709 12:24:20.942760  8117 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:20.945142  8202 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:20.936122  8274 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:20.972086  5074 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:20.918686  8017 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:20.977715  8097 consensus_queue.cc:1243] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:24:20.935339  8241 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:20.980872  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:20.983458  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:20.983999  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:20.987135  5221 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:24:20.962363  8163 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:20.940761  8252 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:20.969158  8254 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:24:20.996559  8252 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:20.947400  8277 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:20.981973  8117 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.998266  8255 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:20.998289  8117 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:21.000977  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35624 (request call id 1126) took 1184 ms. Trace:
I20260709 12:24:20.981750  8201 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:21.001471  5223 rpcz_store.cc:276] 0709 12:24:19.816218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.816385 (+   167us) service_pool.cc:224] Handling call
0709 12:24:21.000783 (+1184398us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b5400553a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:21.000903 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2243110144}
I20260709 12:24:21.001865  8274 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:21.002944  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:21.003283  5219 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:20.987761  8080 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:21.009989  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:21.010242  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:21.010666  5067 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:21.010953  5218 raft_consensus.cc:2393] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:20.986510  8216 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:21.014410  8216 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:21.017100  5224 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:21.003732  8201 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:21.003619  8230 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:20.935474  8204 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:21.004277  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:20.987977  8206 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.030s	sys 0.019s
I20260709 12:24:21.006191  8241 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:21.008046  8017 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:21.008373  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:21.011411  5943 raft_consensus.cc:2368] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9.
I20260709 12:24:21.003060  8274 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:21.005369  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:21.020406  5218 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:24:21.069664  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 64399us
I20260709 12:24:21.071254  5218 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:24:21.023197  5069 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:21.025619  5073 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:21.028110  5067 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:20.986501  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:20.970559  5217 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:21.032390  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:20.987910  8158 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:21.035550  8244 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:21.037431  7698 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:21.037961  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:21.039670  5219 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:20.971957  8205 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:20.981727  8168 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:21.042480  5221 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:21.044248  8080 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:20.967267  8213 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:21.047796  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 64401.2us
I20260709 12:24:21.048870  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:20.941581  8180 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:24:21.058358  5223 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:21.068245  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 59958.1us
I20260709 12:24:21.117353  5188 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:21.117980  8089 consensus_meta.cc:287] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.509s	user 0.006s	sys 0.006s
I20260709 12:24:21.118551  8089 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:21.075043  5215 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
W20260709 12:24:21.076228  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 44002.2us
I20260709 12:24:21.122884  8204 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:21.075471  5114 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:21.055624  8072 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:21.081997  8241 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:21.126503  8072 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:21.111460  5222 pending_rounds.cc:85] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 3
I20260709 12:24:21.081286  8055 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:21.089618  8205 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:21.089843  8168 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.081585  8201 raft_consensus.cc:2764] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:24:21.114497  5219 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:21.109587  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:21.140064  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:21.140686  5217 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:21.141085  5076 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:21.131029  5222 pending_rounds.cc:107] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 7.4
W20260709 12:24:21.134246  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 58881.8us
I20260709 12:24:21.132522  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:21.142132  5222 raft_consensus.cc:2889] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:21.135243  8176 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:24:21.144237  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:21.145252  5223 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:21.142611  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:21.149869  5188 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:21.127082  8072 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:21.154949  5190 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:21.139328  8080 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:21.161909  8163 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:21.163007  8032 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:21.163188  8098 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.160841  8252 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:21.163918  8098 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:21.164494  8252 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.164877  8252 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:21.133467  8055 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:21.138677  8168 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:21.166905  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:21.170220  8202 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:21.170994  8080 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:21.172259  8080 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.173661  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 623) took 1055 ms. Trace:
I20260709 12:24:21.174079  5943 rpcz_store.cc:276] 0709 12:24:20.118330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.118491 (+   161us) service_pool.cc:224] Handling call
0709 12:24:21.173396 (+1054905us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b540000b254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:21.173592 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1985501952}
I20260709 12:24:21.174504  8072 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:21.176961  8239 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.858s	user 0.001s	sys 0.003s
I20260709 12:24:21.163923  8117 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:21.178570  8193 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:21.179028  8193 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:21.179106  7896 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:21.180166  8035 raft_consensus.cc:3055] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:21.180063  8193 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:21.181350  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1134) took 1356 ms. Trace:
I20260709 12:24:21.180409  8117 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:21.181764  5216 rpcz_store.cc:276] 0709 12:24:19.824671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.825907 (+  1236us) service_pool.cc:224] Handling call
0709 12:24:19.826231 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.939554 (+113323us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:19.968179 (+ 28625us) log.cc:844] Serialized 14 byte log entry
0709 12:24:20.138385 (+170206us) log.cc:844] Serialized 14 byte log entry
0709 12:24:20.652445 (+514060us) spinlock_profiling.cc:243] Waited 945 us on lock 0x7b18003e7b80. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a53c8c0a 00007fd8aeb08219 00007fd8a5450a40 00007fd8a544b4e4 00007fd8a544b455 00007fd8a8533060 00007fd8a8545716 00007fd8a8556287 00007fd8a8556208
0709 12:24:20.771872 (+119427us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b48002211d8. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ad464aa1 00007fd8ad45bb37 00007fd8ad45b9ca 00007fd8ad55326f 00007fd8ad552f37 00007fd8ad56755c 00007fd8ad567508 00007fd8ad567488 00007fd8ad567443 00007fd8ad5666ec 00007fd8ac490a96
0709 12:24:20.819127 (+ 47255us) spinlock_profiling.cc:243] Waited 6.51 ms on lock 0x7b18003e7b80. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a80e34bc 00007fd8a543148b 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8a8533060 00007fd8a8545716 00007fd8a8556287
0709 12:24:20.819262 (+   135us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.864646 (+ 45384us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.864808 (+   162us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:20.864841 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.864925 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.176330 (+311405us) spinlock_profiling.cc:243] Waited 36.9 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:21.181101 (+  4771us) raft_consensus.cc:1685] finished
0709 12:24:21.181123 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.181283 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86603,"prepare.queue_time_us":37123,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":101,"raft.queue_time_us":1185015,"raft.run_cpu_time_us":3071,"raft.run_wall_time_us":102760,"spinlock_wait_cycles":108975616,"thread_start_us":237662,"threads_started":3,"wal-append.queue_time_us":201223}
I20260709 12:24:21.184021  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:21.168581  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:21.184019  8055 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:21.188721  7896 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:21.180572  8035 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:21.190542  8035 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:21.143132  8213 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.191795  8035 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:21.193941  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 618543us
I20260709 12:24:21.182713  7432 consensus_queue.cc:1059] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:24:21.195542  7924 raft_consensus.cc:3055] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:21.195677  8202 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:21.195864  5898 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:21.197295  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 612098us
I20260709 12:24:21.185174  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 568) took 1103 ms. Trace:
I20260709 12:24:21.198082  5142 rpcz_store.cc:276] 0709 12:24:20.081782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.176927 (+ 95145us) service_pool.cc:224] Handling call
0709 12:24:21.018078 (+841151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.018430 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.018461 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.018504 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:21.018543 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.018601 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.184997 (+166396us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b540036ea50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:21.185112 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":878,"raft.queue_time_us":545,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":10315,"spinlock_wait_cycles":1901959424}
I20260709 12:24:21.198915  5900 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:21.196074  7924 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:21.213177  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:21.214749  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 46253.9us
I20260709 12:24:21.229090  5900 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:21.138386  8275 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:24:21.234658  8230 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:21.235901  7705 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:21.237491  8188 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:21.237994  8188 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.238348  8188 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:21.241042  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1138) took 1305 ms. Trace:
I20260709 12:24:21.241755  5073 rpcz_store.cc:276] 0709 12:24:19.935342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.935566 (+   224us) service_pool.cc:224] Handling call
0709 12:24:19.935937 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.074396 (+1138459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.240720 (+166324us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b5400327b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:21.240939 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1981464832}
I20260709 12:24:21.166446  8254 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:21.246569  8254 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:21.247037  8254 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:21.226137  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:21.249296  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:21.250140  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45005.1us
I20260709 12:24:21.191795  8213 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:21.252744  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1124) took 1063 ms. Trace:
I20260709 12:24:21.260253  5141 rpcz_store.cc:276] 0709 12:24:20.189263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.207032 (+ 17769us) service_pool.cc:224] Handling call
0709 12:24:21.252451 (+1045419us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b5400238a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:21.252663 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1228582144}
I20260709 12:24:21.262301  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:21.262931  5216 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 8
W20260709 12:24:21.263072  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 538800us
I20260709 12:24:21.264285  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:21.264917  5147 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:21.265058  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:21.265422  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:21.266528  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.268579  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.267557  5041 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:21.196775  8202 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:21.247749  7432 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:21.272473  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 640) took 1063 ms. Trace:
I20260709 12:24:21.282938  5074 rpcz_store.cc:276] 0709 12:24:20.209558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.212291 (+  2733us) service_pool.cc:224] Handling call
0709 12:24:20.284968 (+ 72677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.982926 (+697958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.272236 (+289310us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54003d1154. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:21.272407 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579662208}
I20260709 12:24:21.274389  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:21.274811  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:21.284417  5217 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: [10.4-10.4]
I20260709 12:24:21.284778  5141 raft_consensus.cc:2368] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8.
I20260709 12:24:21.280812  5041 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:21.288614  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 89014.1us
I20260709 12:24:21.257287  8089 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:21.306493  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:21.293622  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:21.293814  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:21.294801  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:21.307772  5069 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:21.307772  5073 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:21.295080  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:21.296188  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:21.258785  8180 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:24:21.300053  8294 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:24:21.300025  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:21.318883  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:21.259641  8097 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:21.321236  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:21.274466  8275 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:21.328524  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 99559.5us
I20260709 12:24:21.313112  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 625) took 1171 ms. Trace:
I20260709 12:24:21.329445  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:21.251794  8264 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:21.275465  8030 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.261445  8107 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.319767  5149 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:21.259047  7896 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:24:21.333915  5073 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:21.278148  8094 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.329864  5944 rpcz_store.cc:276] 0709 12:24:20.141554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.143059 (+  1505us) service_pool.cc:224] Handling call
0709 12:24:20.143407 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.143691 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:20.564229 (+420538us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.312590 (+748361us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.312670 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:21.312746 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.312846 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.313030 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":694873,"prepare.queue_time_us":95242,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":703304,"thread_start_us":465513,"threads_started":2,"wal-append.queue_time_us":131259}
W20260709 12:24:21.338519  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.303587  7432 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:21.341277  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:21.342259  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:21.291393  8216 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:21.302930  8204 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:21.272080  8265 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:21.343995  5948 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
W20260709 12:24:21.341455  8291 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.183s	user 0.002s	sys 0.000s
I20260709 12:24:21.315089  8180 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.389968  5069 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:21.422777  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.376083  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:21.423779  5946 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 4.
I20260709 12:24:21.425130  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.288973  8209 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:24:21.426900  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:21.431159  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:21.485743  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:21.491559  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.374665  7867 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.524874  7867 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:21.336860  8030 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:21.527335  8030 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:21.420866  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 91483.8us
I20260709 12:24:21.425978  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.533991  5945 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:21.535409  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:21.376358  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:21.420320  8075 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:21.537310  5144 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:21.541095  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.420130  8070 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:21.478163  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.356451  7432 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:21.528928  7975 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.450981  8180 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:21.554606  5942 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
W20260709 12:24:21.572350  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 141347us
I20260709 12:24:21.357823  8216 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.574096  5899 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:21.575479  5900 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:21.360322  8238 raft_consensus.cc:3055] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:21.576982  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:21.450238  8098 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:21.453064  8193 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:21.450464  8089 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:21.556447  8075 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.353506  8094 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:21.479398  8189 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:21.359287  7788 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.239s
I20260709 12:24:21.585304  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.586344  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.646271  5948 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:21.646529  5942 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:21.586998  8258 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:21.647562  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.587391  8216 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:21.587765  8264 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:21.587567  7975 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.588171  8180 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:21.589212  5145 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:21.589849  8246 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:21.479182  8204 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:21.593992  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:21.343050  8284 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:21.479382  8257 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:21.595925  5945 raft_consensus.cc:2368] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9.
W20260709 12:24:21.599140  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.538100  5216 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:21.595866  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.360833  8107 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.531081  8125 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:21.573441  8154 consensus_meta.cc:287] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.592s	user 0.006s	sys 0.005s
I20260709 12:24:21.337023  8275 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:21.678366  8117 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.679056  8117 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.680246  8275 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:21.681218  5147 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:21.634054  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:21.569989  8055 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.685905  5946 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:21.687435  8055 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:21.688565  8205 consensus_meta.cc:287] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.536s	user 0.003s	sys 0.004s
I20260709 12:24:21.343462  8261 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.565644  7867 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:21.689975  8261 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.690883  8094 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:21.691885  8261 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:21.648356  5948 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:21.333685  7924 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.595083  8238 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:21.671277  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.693154  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:21.693735  5942 raft_consensus.cc:2393] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:21.693948  8238 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:21.694538  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:21.695175  5948 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:21.695236  8238 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:21.695577  5141 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:21.675836  8107 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:21.662793  8204 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:21.697292  8180 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:21.699113  8030 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:21.656172  8241 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.001s
I20260709 12:24:21.677588  8154 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.644824  8089 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:21.676443  8125 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:21.702157  5941 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:21.703691  5940 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 12:24:21.571614  8070 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.723131  5946 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
W20260709 12:24:21.701043  8035 consensus_meta.cc:287] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.000s
I20260709 12:24:21.730594  5948 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:21.751006  5144 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:21.681252  8117 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:21.696190  8168 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:21.725490  8239 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:21.756122  5218 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:21.655313  8189 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:21.725930  8241 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:21.765390  5946 raft_consensus.cc:1217] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Deduplicated request from leader. Original: 7.4->[9.5-9.5]   Dedup: 9.5->[]
I20260709 12:24:21.777184  5946 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:24:21.779726  5215 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:21.689014  8205 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:21.784090  5946 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:24:21.792161  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.745302  5948 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:21.739383  8030 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:21.800863  5072 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:21.777654  8117 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.804916  5946 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:21.790544  8252 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.004s
W20260709 12:24:21.825416  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.831892  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:21.833741  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:21.840595  8075 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:21.828840  8055 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:21.812453  8117 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:21.856115  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.819136  8252 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:21.864552  8244 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.866792  8168 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:21.865161  7975 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:21.879642  5217 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:21.869078  8117 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:21.865051  8055 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:21.914724  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 84532.4us
I20260709 12:24:21.920719  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:21.922406  5190 leader_election.cc:400] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:21.925235  5190 leader_election.cc:403] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:21.926409  5190 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:21.927119  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:21.927493  8075 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:21.927966  5221 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:21.927301  8257 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:21.928393  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:21.929100  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:21.929458  7896 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:21.930042  7896 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:21.930444  7896 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:21.929033  5224 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:21.933014  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:21.933121  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:21.933826  5219 raft_consensus.cc:2393] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:21.934114  5947 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:21.934988  5190 leader_election.cc:400] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8.
I20260709 12:24:21.935324  5190 leader_election.cc:403] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:21.937502  8255 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8.
I20260709 12:24:21.938050  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:21.938584  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:21.938717  5943 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:21.940572  8255 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:21.942387  5190 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:21.944447  8255 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:21.944873  8255 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.945240  8255 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:21.948668  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.951664  5224 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:21.952062  5947 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:21.953497  7896 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:21.956773  7896 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:21.959683  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.896998  8089 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:21.964629  5219 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:21.964874  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:21.974619  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 400573us
I20260709 12:24:21.975044  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:21.900413  8125 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:21.975970  5067 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 9: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:21.975963  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:21.977614  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.912832  8117 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.880263  8244 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:21.991730  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:21.981273  8304 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 12:24:21.981849  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 387926us
I20260709 12:24:21.982470  8017 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:21.982767  8168 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:21.984247  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:21.981338  8098 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:21.986775  7788 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.986713  8284 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:21.973873  8189 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:22.021795  5219 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:22.023576  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:22.023917  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.024185  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:21.987366  8309 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.974150  8070 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:21.988845  8252 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:21.914781  8260 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:21.990885  8274 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.913s	user 0.009s	sys 0.004s
I20260709 12:24:22.030104  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:21.991544  8080 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:21.993501  8254 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.746s	user 0.013s	sys 0.001s
I20260709 12:24:21.997099  8244 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:21.980635  8136 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:21.999526  5224 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:22.007505  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:22.008486  5221 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:22.037248  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:22.008634  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:22.008677  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:21.991811  8305 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:24:22.014976  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:22.002111  8213 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.740s	user 0.013s	sys 0.001s
I20260709 12:24:22.004680  8117 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:22.051517  8180 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:22.024923  5946 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:22.025442  5076 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:22.022634  5939 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5751113d37b44af87d3d912da388078"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:22.055080  5939 raft_consensus.cc:606] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:22.055508  5939 raft_consensus.cc:614] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:22.033298  8309 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:22.043936  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:22.026368  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:22.030041  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:21.963491  8204 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:22.057906  5075 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:22.028687  8168 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:22.059259  5071 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:22.059707  8168 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:22.020385  8216 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:22.061910  5074 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:24:22.062716  8216 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:22.040341  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 56184.6us
I20260709 12:24:22.065001  5898 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:22.036057  8260 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:22.065443  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 893) took 1085 ms. Trace:
I20260709 12:24:22.065896  5072 rpcz_store.cc:276] 0709 12:24:20.982262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.982435 (+   173us) service_pool.cc:224] Handling call
0709 12:24:20.982810 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.803090 (+820280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.065242 (+262152us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b540058aa54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:22.065389 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1880708864}
I20260709 12:24:22.067193  8260 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:22.067649  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1142) took 1107 ms. Trace:
I20260709 12:24:22.022074  8162 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.035733  8098 raft_consensus.cc:697] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:22.068349  8162 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:22.068809  8098 consensus_queue.cc:237] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.067824  8117 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:22.069661  8189 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:22.070748  8309 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:22.073390  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 65978.7us
I20260709 12:24:22.045783  8265 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:22.075836  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:22.076236  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:22.076822  5223 raft_consensus.cc:2368] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:22.048599  8313 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:22.028980  7788 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:22.040365  8080 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:22.051494  8303 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:24:22.040154  8254 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.059415  8204 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:22.073515  5143 raft_consensus.cc:2410] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 12:24:22.055224  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.020781  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.066682  5898 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:22.027244  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.018280  7896 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.031054  5219 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:22.037817  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:22.102903  5221 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:22.068120  5215 rpcz_store.cc:276] 0709 12:24:20.959886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.120992 (+161106us) service_pool.cc:224] Handling call
0709 12:24:21.121224 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.780431 (+659207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.067420 (+286989us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b5400041ad4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:22.067588 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1513805696}
W20260709 12:24:22.090360  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 47433.3us
I20260709 12:24:22.104816  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:22.092393  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:22.107424  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.109113  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:22.110327  5940 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:22.110646  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:22.110822  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.111179  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.111557  5939 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:22.112027  5946 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:22.112848  8201 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.093500  8230 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:22.113804  8230 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:22.114282  8230 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.114428  7788 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:22.115321  5070 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:24:22.113445  8201 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:22.115712  5070 pending_rounds.cc:85] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 6
W20260709 12:24:22.121011  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45671.4us
I20260709 12:24:22.121208  5070 pending_rounds.cc:107] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:24:22.121845  5070 raft_consensus.cc:2889] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:22.117414  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.118185  5189 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:22.130012  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:22.086244  8313 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:22.096129  8260 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.137607  5188 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:22.144536  5189 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:22.133059  5041 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:22.145016  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:22.150626  8213 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:22.098280  8180 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:22.121001  8265 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:22.151224  8213 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:22.156371  5115 leader_election.cc:400] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9.
I20260709 12:24:22.156194  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.156594  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.136445  8080 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:22.151934  8252 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:22.157135  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.142371  8035 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:22.179705  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.130918  7788 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:24:22.134640  8188 consensus_meta.cc:287] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.896s	user 0.007s	sys 0.008s
I20260709 12:24:22.100411  8259 consensus_queue.cc:1059] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:24:22.102456  8216 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:24:22.127044  8204 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:22.166910  5945 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:22.178128  5071 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:22.122203  7896 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.176344  8241 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:22.154490  8267 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:24:22.154253  8163 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:22.187182  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:22.151455  8055 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:22.179955  8035 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.166536  8275 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:22.187665  7788 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.176949  8252 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:22.152601  8260 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:22.211772  5115 leader_election.cc:400] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9.
I20260709 12:24:22.179005  8265 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:22.203116  8204 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:22.215304  8274 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:22.219295  5076 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:22.201457  8238 raft_consensus.cc:3055] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:22.199287  8158 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.210459  8163 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:22.185076  5041 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:22.244064  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 59063.8us
I20260709 12:24:22.209146  8201 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:22.195580  8188 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:22.218580  8035 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:22.200011  8255 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.246187  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:22.231444  8275 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 11 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:22.249961  5073 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:22.230760  8252 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:22.256856  5219 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:22.225481  7788 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:22.257838  5215 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:22.259763  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.275470  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.260987  5223 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:22.261596  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:22.260702  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:22.263482  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:22.264539  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.277271  5221 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:22.237155  8238 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:22.249346  8055 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:22.264278  8317 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:22.275425  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 131284us
I20260709 12:24:22.247090  8163 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:22.274147  5944 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:22.246449  8180 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.269980  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.280267  5222 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:22.280649  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.310636  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1139) took 1365 ms. Trace:
I20260709 12:24:22.244567  8080 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:24:22.283210  8275 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.310804  5073 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:22.285035  8260 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:22.216725  8313 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:22.265129  8279 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:22.255359  8162 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:22.249646  8158 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.287482  8265 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:22.288425  8055 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:22.288558  8300 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.272s	user 0.003s	sys 0.000s
I20260709 12:24:22.284459  7896 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:22.291311  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:22.264943  8201 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:24:22.241648  8213 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:22.265044  8193 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.294504  8238 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.267345  8311 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:22.297550  8216 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:22.298575  5072 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:22.314052  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:22.311267  5218 rpcz_store.cc:276] 0709 12:24:20.945152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.072410 (+127258us) service_pool.cc:224] Handling call
0709 12:24:21.072909 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.764970 (+692061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.310254 (+545284us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b5400210f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:22.310538 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1563859712}
I20260709 12:24:22.317981  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:22.284318  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1131) took 1019 ms. Trace:
W20260709 12:24:22.339828  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 48601.8us
I20260709 12:24:22.319710  8254 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:22.355155  8250 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:24:22.360512  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 216065us
I20260709 12:24:22.360548  5221 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:22.337707  8055 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:22.368157  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 54257.6us
I20260709 12:24:22.370303  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:22.368595  5143 rpcz_store.cc:276] 0709 12:24:21.264812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.265208 (+   396us) service_pool.cc:224] Handling call
0709 12:24:22.283948 (+1018740us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b540014f5d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:22.284212 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1856281856}
I20260709 12:24:22.370873  5899 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:22.375542  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:22.376361  8259 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:22.376873  8259 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.377224  8259 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:22.356900  8313 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:22.382946  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.387322  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.358059  8216 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.362946  8274 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.392892  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.398483  5072 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:22.399791  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.394708  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.399932  5898 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:22.373479  8162 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:22.394797  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:22.337040  8279 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:22.377895  8075 raft_consensus.cc:2764] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:24:22.409709  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:22.411283  5075 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:22.383435  8238 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:22.403685  8261 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:22.381981  8201 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.358577  8193 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:22.397710  8216 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:22.392005  8098 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:22.417371  8261 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.413344  8162 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.428669  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:22.435715  5141 raft_consensus.cc:2376] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7. Re-sending same reply.
I20260709 12:24:22.436802  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:22.437598  5141 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 9: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:22.438561  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:22.427439  8201 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.430244  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:22.431551  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:22.409013  8274 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:22.431713  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:22.427353  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:22.432389  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:22.441911  5145 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:22.417497  8257 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:22.454761  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:22.440286  5147 raft_consensus.cc:2410] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:22.442956  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:22.443362  5221 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:22.457755  5141 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:22.443498  5145 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:22.433512  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:22.458724  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:22.424366  8030 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:24:22.453717  8237 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:22.454214  8201 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:22.431356  8098 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.437071  8162 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:22.434469  8075 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:22.495438  5941 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:22.434636  8261 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:22.499240  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:22.442493  8204 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:22.471830  8094 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:22.491617  8237 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:22.503085  8098 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:22.455332  8072 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:24:22.488507  8284 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:24:22.508803  8258 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.515105  8255 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:22.519273  8094 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:22.533713  8331 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:24:22.535130  8094 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:22.535923  8158 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:22.518033  8154 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:22.525365  8328 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:24:22.531965  8258 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.537420  8205 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:22.549165  8258 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:22.549744  8279 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:22.552678  8252 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:22.554627  8302 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.641s	user 0.001s	sys 0.003s
I20260709 12:24:22.556938  8252 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:22.563196  8274 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:22.568179  8259 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:22.629907  5041 leader_election.cc:400] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:22.631364  5041 leader_election.cc:403] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:24:22.632184  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:22.637277  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 274291us
I20260709 12:24:22.643949  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.651151  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.652444  5067 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:22.656972  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.661346  5068 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:22.662792  5221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:24:22.668547  5142 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:22.670409  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:22.670825  5144 raft_consensus.cc:2410] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:22.672649  5150 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:22.677466  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1136) took 1354 ms. Trace:
I20260709 12:24:22.684128  5148 rpcz_store.cc:276] 0709 12:24:21.322945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.323141 (+   196us) service_pool.cc:224] Handling call
0709 12:24:21.345987 (+ 22846us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.346424 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.346463 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:21.347065 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.801384 (+454319us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:21.880929 (+ 79545us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.910143 (+ 29214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.910277 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.547119 (+636842us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b1800411100. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a541c2a6 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db
0709 12:24:22.554149 (+  7030us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:22.676551 (+122402us) raft_consensus.cc:1685] finished
0709 12:24:22.676627 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.677122 (+   495us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66611,"prepare.queue_time_us":114011,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":28884,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":33946,"spinlock_wait_cycles":58282112,"thread_start_us":532990,"threads_started":2,"wal-append.queue_time_us":525020}
I20260709 12:24:22.695784  8193 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:22.698105  5218 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:22.707299  5221 raft_consensus.cc:606] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:22.716228  5945 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:22.719086  8125 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:22.720196  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.722164  8205 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:22.725169  5147 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:22.714195  5221 raft_consensus.cc:614] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:22.726214  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:22.734210  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 49681.5us
I20260709 12:24:22.737046  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 741) took 1204 ms. Trace:
I20260709 12:24:22.737273  8313 consensus_queue.cc:1059] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:22.742478  8279 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.743570  8279 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.746019  8306 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.747005  8306 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:22.753460  8180 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:22.755879  8333 raft_consensus.cc:3055] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:22.756470  8333 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:22.762277  8089 raft_consensus.cc:3055] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:22.763057  8089 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:22.765094  8089 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.771320  8089 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.737597  5944 rpcz_store.cc:276] 0709 12:24:21.532426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.595587 (+ 63161us) service_pool.cc:224] Handling call
0709 12:24:22.736505 (+1140918us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b540039b2d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:22.736799 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1410454784}
W20260709 12:24:22.774921  8230 consensus_meta.cc:287] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.000s
I20260709 12:24:22.775830  8230 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:22.776062  8258 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:22.776765  8299 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.938s	user 0.002s	sys 0.001s
I20260709 12:24:22.777451  8258 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:22.781673  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:22.783085  8311 raft_consensus.cc:3055] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:22.783740  8311 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:22.786042  8201 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.548250  8237 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:22.786944  8311 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:22.788866  8237 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:22.789903  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:22.786964  8201 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:22.793428  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:22.795140  5141 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:24:22.795588  5141 pending_rounds.cc:85] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 3
W20260709 12:24:22.795708  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.795874  5141 pending_rounds.cc:107] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:24:22.796481  5141 raft_consensus.cc:2889] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:22.797552  8333 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:22.793295  8201 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:22.789341  8311 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:22.802578  8188 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:22.808527  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:22.809676  8258 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:22.810053  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:22.811543  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:22.812433  5147 raft_consensus.cc:2410] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:22.813522  5148 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:22.813524  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:22.817200  5149 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:22.520674  8035 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.820322  7924 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:22.821864  8154 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:22.822623  8154 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:22.823792  8317 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:22.824771  8317 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:22.825489  8154 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:22.847035  5148 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:22.828361  8317 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:22.833068  5075 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:24:22.833514  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:22.852630  5074 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:22.834537  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.834998  5072 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:24:22.840054  8338 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 12:24:22.855598  5188 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:22.857496  8252 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:22.858044  8252 raft_consensus.cc:697] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:22.859241  8252 consensus_queue.cc:237] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:22.862032  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.863497  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:22.866189  5068 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:22.868044  8258 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:22.869302  8188 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:22.846243  8265 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:22.875921  8265 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:22.826344  8325 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:22.827857  8070 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.830351  5071 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:22.880637  8325 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.880676  8070 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:22.884294  8070 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:22.884601  8265 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:22.842612  8163 consensus_meta.cc:287] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.017s
I20260709 12:24:22.887287  8163 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:22.873224  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:22.841475  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 960) took 1586 ms. Trace:
W20260709 12:24:22.890517  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 240777us
I20260709 12:24:22.904971  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:22.720503  8125 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:22.902634  8325 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:22.927870  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:22.928465  8265 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:22.937909  8035 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:22.945329  5144 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:22.946025  5069 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:22.888906  5220 rpcz_store.cc:276] 0709 12:24:21.255108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.255539 (+   431us) service_pool.cc:224] Handling call
0709 12:24:21.255869 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.256254 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.256281 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:21.256795 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.637203 (+380408us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:21.637245 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.637313 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.775533 (+1138220us) spinlock_profiling.cc:243] Waited 7.5 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:22.840723 (+ 65190us) raft_consensus.cc:1685] finished
0709 12:24:22.840762 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.841185 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":181,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":4202,"spinlock_wait_cycles":17259264,"thread_start_us":333476,"threads_started":1,"wal-append.queue_time_us":535165}
I20260709 12:24:22.948926  5148 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:22.951040  8261 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:22.953872  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.965440  5072 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 8: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:24:22.966675  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 44321.9us
I20260709 12:24:22.977566  5224 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:22.960675  8317 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.995107  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:22.992280  8163 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:23.006518  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 169829us
I20260709 12:24:23.002725  8333 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:23.012065  5148 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:22.993181  8317 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.010193  8306 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.028383  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.033116  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:23.036854  5072 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:23.001575  8265 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.026795  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:23.033589  8230 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.050730  8125 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:23.053723  5141 raft_consensus.cc:2393] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:23.058585  5144 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:23.049178  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:23.075882  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 73860.4us
I20260709 12:24:23.066821  5142 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:23.071978  5224 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:23.087850  8163 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:23.088639  8163 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:23.089874  8163 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.058602  8279 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:23.094033  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.096411  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.097412  5220 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
W20260709 12:24:23.123054  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.120236  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:23.110910  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:23.133019  5146 raft_consensus.cc:2368] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:23.138614  5148 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:23.145125  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:23.158123  5945 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:23.168248  8162 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.646s	user 0.012s	sys 0.006s
I20260709 12:24:23.168864  8162 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.169734  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:24:23.169781  8327 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:24:23.172827  5945 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:23.161263  8125 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:23.181282  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.169782  7867 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:24:23.184639  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:23.161288  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:23.191020  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 158434us
W20260709 12:24:23.194293  8055 consensus_meta.cc:287] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.812s	user 0.020s	sys 0.001s
I20260709 12:24:23.194643  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.195135  8055 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:23.200100  8163 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:24:23.214490  8265 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.216554  8317 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:23.222805  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:23.234591  5224 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:23.237079  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.237990  5224 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:23.238216  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:23.237998  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:23.239674  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:23.240415  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:23.240650  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:23.241750  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:23.238425  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:23.221662  8035 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:23.246230  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:23.246887  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:23.242408  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:23.247116  5945 raft_consensus.cc:2368] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11.
I20260709 12:24:23.251320  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:23.252148  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 212248us
I20260709 12:24:23.222435  8213 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.266006  5148 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
W20260709 12:24:23.269132  7788 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.962s	user 0.003s	sys 0.016s
W20260709 12:24:23.283825  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 165106us
I20260709 12:24:23.284447  5218 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:23.280093  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 652) took 1298 ms. Trace:
I20260709 12:24:23.287308  5069 rpcz_store.cc:276] 0709 12:24:21.975548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.975685 (+   137us) service_pool.cc:224] Handling call
0709 12:24:23.261957 (+1286272us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b54002ce554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:23.273487 (+ 11530us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2119981440}
I20260709 12:24:23.308525  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:23.309578  5074 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:23.324079  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.326285  7867 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.327032  7867 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:23.323946  4898 catalog_manager.cc:5697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:23.330106  8055 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:23.330192  7867 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:23.338716  8354 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:24:23.342958  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 49925.9us
I20260709 12:24:23.288968  8213 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:23.353794  8098 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.828s	user 0.011s	sys 0.008s
I20260709 12:24:23.282738  7788 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.357522  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:23.354468  8098 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:23.372557  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.377382  5074 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
W20260709 12:24:23.363092  8216 consensus_meta.cc:287] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.785s	user 0.018s	sys 0.003s
I20260709 12:24:23.385119  5073 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:23.389952  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:23.401736  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 86664.8us
I20260709 12:24:23.402832  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.433727  8162 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.435446  5215 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:23.435914  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.436735  5217 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
W20260709 12:24:23.438134  5042 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.926s (SENT) [suppressed 17 similar messages]
I20260709 12:24:23.438339  5222 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:23.438728  5223 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:23.440008  5041 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:23.441680  8259 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.442667  8259 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:23.443403  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:23.444243  5939 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:23.444324  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:23.445626  7788 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:23.446297  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.446496  8327 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:23.447072  5219 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:23.447466  8327 consensus_queue.cc:1059] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:23.448040  5114 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:23.448181  8279 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:23.448552  8327 raft_consensus.cc:3055] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:23.449033  8327 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:23.408947  8216 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.450466  7924 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:23.450953  7924 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.451306  7924 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:23.451975  8216 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:23.453861  5067 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:24:23.455606  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:23.457082  5189 leader_election.cc:400] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11.
I20260709 12:24:23.457442  5189 leader_election.cc:403] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:24:23.457955  5900 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:23.458207  8163 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:23.458797  8163 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:23.459761  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:23.460745  8325 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:23.461756  8265 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:23.443320  8259 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:23.456943  5041 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:23.441741  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 56865.1us
I20260709 12:24:23.462370  8265 raft_consensus.cc:697] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:23.463181  5900 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:23.478852  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.479243  5189 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:23.479611  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:23.481189  8252 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:23.480839  8265 consensus_queue.cc:237] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:23.482407  8351 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:24:23.485378  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:23.486092  5221 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:23.488395  8259 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:23.491158  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.493142  5142 raft_consensus.cc:2410] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:23.496661  8327 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.498368  8327 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:23.498211  5041 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:23.499145  7788 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:23.499712  7788 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:23.499985  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:23.500120  7788 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:23.500799  8311 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:23.501394  8311 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.501781  8311 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:23.501959  8333 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:23.502846  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:23.500819  8125 consensus_queue.cc:1059] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:24:23.504843  8333 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:23.505235  8333 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.505306  8125 raft_consensus.cc:3055] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:23.505746  8125 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:23.454021  8154 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.506001  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:23.506445  8154 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:23.506775  8125 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:23.507927  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 46487.9us
I20260709 12:24:23.508261  8125 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:23.509711  8162 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.509869  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:23.510545  5222 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:23.510185  4898 catalog_manager.cc:5697] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:24:23.512063  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:23.512737  8317 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:23.512327  4898 catalog_manager.cc:5697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:23.512953  8259 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.512362  8098 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.513414  8317 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:23.331918  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:23.514461  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:23.472278  8237 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:23.480909  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.519773  5939 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:23.509231  8154 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:23.471939  8343 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:23.514741  8333 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:23.526903  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.459935  8163 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:23.532167  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 55808.9us
I20260709 12:24:23.520833  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.530969  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:23.530903  5220 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:23.532663  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:23.533783  5075 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:23.510339  8162 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.529943  8343 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.549464  8035 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.550257  8035 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:23.551716  5900 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:23.552568  7432 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:23.554131  8055 raft_consensus.cc:2764] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:24:23.555110  8055 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:23.558266  7924 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:23.560581  5190 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:23.560712  8154 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:23.549640  8343 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:23.561993  5946 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:23.578833  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.582854  8162 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.534523  8259 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:24:23.583441  8309 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.584118  8309 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:23.586314  8309 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:23.587395  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:23.588045  5216 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:23.553175  7432 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:23.591572  7432 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:23.595938  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.596807  5947 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:23.571646  8216 consensus_queue.cc:1059] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:24:23.599684  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:23.603879  8317 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:23.599498  4898 catalog_manager.cc:5697] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 5 to 6, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 6 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:23.600436  8216 raft_consensus.cc:3055] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:23.607007  8216 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:23.583449  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.608572  5946 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:23.608213  8216 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.601097  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:23.610029  5074 raft_consensus.cc:2410] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:23.610069  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:23.610244  8016 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:23.611325  8016 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:23.612083  5190 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:23.613917  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.577282  5899 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:23.565209  8035 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.620903  5222 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:23.615499  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:23.638535  5216 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
W20260709 12:24:23.640657  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 107492us
I20260709 12:24:23.620886  5939 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:23.645351  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:23.646974  5946 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:23.649080  7924 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:23.648762  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:23.621949  8230 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:23.672405  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 120757us
I20260709 12:24:23.667800  8035 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:23.678813  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 63398.9us
I20260709 12:24:23.678987  8035 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.679754  8035 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:23.679801  8259 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:23.680598  8259 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:23.671769  7867 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:24:23.681730  8162 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:23.681825  8259 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.682971  8162 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:23.683445  8162 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:23.683835  8216 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.684567  8162 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:23.653759  8252 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:23.685703  5899 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:23.686741  8237 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:23.686023  8252 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:23.687304  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:23.688026  5942 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:23.688970  8252 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:23.690567  8030 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:23.691883  5947 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:23.692562  8162 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:23.691959  5899 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:23.619436  8333 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:23.709792  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:23.694554  8317 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:23.712016  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:23.712370  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.694739  8311 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:23.714267  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:23.714406  8030 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:23.695848  5042 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:23.715248  8030 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.701846  8309 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.715867  8030 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:23.716326  8309 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:23.717832  8317 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:23.718278  8317 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:23.718681  8317 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:23.659477  7788 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:23.703512  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.720165  5946 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:23.687188  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:23.721294  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.722900  5946 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:23.724985  5946 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:23.727027  8309 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.727888  8325 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11.
I20260709 12:24:23.729022  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:23.731673  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.732793  5071 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:23.733000  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.736546  5221 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:23.738663  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.742347  5224 raft_consensus.cc:2410] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:24:23.738003  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:23.714941  8311 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:23.777817  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.724380  7788 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:23.790118  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 78172.4us
I20260709 12:24:23.792434  5189 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:23.794466  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:23.800197  5947 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:23.802495  8070 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:23.802838  8261 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.803413  8070 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:23.810405  8252 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:23.811376  8252 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.814654  8180 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:23.815337  8180 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:23.819258  5073 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 654) took 1426 ms (client timeout 1701 ms). Trace:
W20260709 12:24:23.819904  5073 rpcz_store.cc:269] 0709 12:24:22.392635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.393564 (+   929us) service_pool.cc:224] Handling call
0709 12:24:22.631805 (+238241us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b5800560198. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416eee 000000000036eb39 00007fd8ad48a1e0 00007fd8ad5530c4 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:24:23.819004 (+1187199us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b5400347454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:23.819175 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1718569728}
I20260709 12:24:23.822703  8265 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.804874  8070 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:23.816706  8180 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:23.742326  7867 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:23.793812  8016 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:23.800268  8230 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.828809  5067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1167) took 1027 ms. Trace:
I20260709 12:24:23.842136  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.928422  5221 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:23.859186  5069 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:23.844352  5217 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:23.926709  5067 rpcz_store.cc:276] 0709 12:24:22.801794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.803044 (+  1250us) service_pool.cc:224] Handling call
0709 12:24:22.803646 (+   602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.454343 (+650697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.821466 (+367123us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b540047e354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:23.828746 (+  7280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1494577408}
I20260709 12:24:23.850118  4898 catalog_manager.cc:5697] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 5 to 9, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 9 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:23.861044  5071 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:24:23.879031  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 230396us
W20260709 12:24:23.880487  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.932548  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.845044  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:23.935999  5220 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:23.899729  5939 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:23.910362  5942 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
W20260709 12:24:23.918346  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.882953  5042 leader_election.cc:400] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
W20260709 12:24:23.906219  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 380508us
I20260709 12:24:23.873893  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:23.845434  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.943115  5224 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:23.840713  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:23.955174  5041 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:23.856794  8252 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.823921  8265 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:23.862113  8180 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:23.891829  8016 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:23.977897  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 98142.7us
W20260709 12:24:23.978315  8388 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:23.876618  8188 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:23.829239  8325 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:23.876281  8333 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:23.827025  8213 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:23.880952  8098 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:23.965174  8306 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:23.986061  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:23.914142  8279 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:23.964089  7788 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:23.997143  8365 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:24:23.879191  7867 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:23.863664  8154 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:24.000531  8343 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:23.965734  5072 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:24.020694  8016 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:24.036114  8279 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:24.016985  5943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.014s	sys 0.007s
I20260709 12:24:24.038151  8098 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:24.038758  8098 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:24.038712  7788 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.041131  7788 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:24.042438  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 112907us
I20260709 12:24:24.044067  8327 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:24.044116  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:24.045917  8333 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:24.046418  8333 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.046826  8333 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:24.029933  8325 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:24.052188  8325 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:24.051448  4898 catalog_manager.cc:5697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:24.052627  5115 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:24.052946  8154 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:24.055420  8154 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:24.054661  4898 catalog_manager.cc:5697] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 3 to 9, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 9 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:24:24.029440  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:24.056278  8327 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:24.058040  8327 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:24.058525  8327 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:24.057819  5115 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:24.056187  8306 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:24.038226  5943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.015s	sys 0.007s
W20260709 12:24:24.097918  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 74066us
I20260709 12:24:24.102666  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:24.111553  5219 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:24.118662  5220 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:24.076048  8265 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:24.122040  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:24.061755  8125 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.123996  4898 catalog_manager.cc:5697] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:24.124692  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:24.086995  8213 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:24.091334  8311 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:24.106542  7788 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:24.132897  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:24.143491  8163 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:24.146682  5224 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:24.146552  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 432421us
I20260709 12:24:24.130867  8125 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.157743  8373 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.157965  7867 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.158576  8373 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:24.160924  8373 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:24.162573  4898 catalog_manager.cc:5697] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:24.176896  8125 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:24.185968  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:24.186870  5072 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:24.188853  8030 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:24.190276  5940 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:24.198421  8265 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.200604  5224 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:24.164897  4898 catalog_manager.cc:5697] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 5 to 7, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 7 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:24.203444  8306 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:24.179556  8213 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:24.234635  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.219381  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:24.233112  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 495166us
I20260709 12:24:24.234467  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:24.236822  5220 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:24.237440  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 378010us
W20260709 12:24:24.221206  8317 consensus_meta.cc:287] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.502s	user 0.001s	sys 0.012s
W20260709 12:24:24.249971  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.233378  8035 consensus_meta.cc:287] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.552s	user 0.012s	sys 0.003s
W20260709 12:24:24.270928  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.273042  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:24.275216  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.278834  5067 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:24.177717  7788 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:24.214721  8333 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:24.271552  8259 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:24.286069  8136 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.014s	sys 0.006s
I20260709 12:24:24.286760  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:24.287232  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:24.287530  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:24.288167  5943 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:24.288681  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:24.288828  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:24.289383  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:24.290072  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:24.299770  5939 raft_consensus.cc:2393] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 6: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:24.299829  5942 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:24.291436  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:24.250203  8306 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:24.291059  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:24.278347  8035 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:24.302199  5946 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:24.304630  5071 raft_consensus.cc:2410] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:24:24.249598  8162 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 12:24:24.297124  8136 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.015s	sys 0.006s
I20260709 12:24:24.301975  8259 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.269265  8317 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:24.302639  8306 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:24.354769  5220 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:24.362275  5067 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:24.343500  4898 catalog_manager.cc:5697] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 9 to 10, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 10 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:24.375802  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:24.384078  5948 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.039s	sys 0.088s
W20260709 12:24:24.384160  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 97041.7us
W20260709 12:24:24.384708  5948 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.040s	sys 0.089s
I20260709 12:24:24.385665  8343 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:24.385949  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 136837us
I20260709 12:24:24.386412  8343 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:24.386716  5041 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:24.388805  8125 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:24.392494  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:24.343231  8162 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:24.326609  8098 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:24.367998  8306 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:24.358750  8180 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:24.419701  5072 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:24.386919  8343 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:24.435861  8035 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:24.436985  8035 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:24.379967  8016 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:24.438692  5222 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:24.427884  4898 catalog_manager.cc:5697] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 6 to 7, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 7 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:24.448385  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 62362.9us
W20260709 12:24:24.460775  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 348593us
I20260709 12:24:24.462481  4898 catalog_manager.cc:5697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 5 to 7, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 7 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:24.464022  5188 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:24.440029  8016 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:24.467702  4898 catalog_manager.cc:5697] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 5 to 10, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 10 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:24.473584  4898 catalog_manager.cc:5697] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 5 to 10, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 10 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:24.470572  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 411607us
I20260709 12:24:24.472571  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:24.482257  5041 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:24.485778  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:24.487409  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 429692us
I20260709 12:24:24.418947  8372 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:24:24.489158  5115 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:24.495555  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:24.394718  8213 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:24.448904  8180 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:24.494802  5041 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:24.479297  8070 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.013s	sys 0.013s
I20260709 12:24:24.456874  8035 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:24.514487  4898 catalog_manager.cc:5697] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:24.508149  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:24.516817  5069 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:24.478762  8055 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.518537  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:24.518831  5071 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:24.520813  8389 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 12:24:24.521656  8387 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:24:24.524288  8386 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:24:24.529389  8035 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:24.533986  8136 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:24.537915  7924 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:24.538688  7924 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:24.521039  4895 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43234 (request call id 125) took 19657 ms (client timeout 15000 ms). Trace:
I20260709 12:24:24.470264  8230 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:24:24.499159  8162 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:24:24.510262  8070 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.015s	sys 0.015s
I20260709 12:24:24.536913  8030 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:24.561584  5948 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:24.565874  5943 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:24.531113  8035 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:24.572053  4898 catalog_manager.cc:5697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 8 to 9, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 9 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:24.487226  8327 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:24.489203  8016 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:24:24.540613  8216 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:24.551148  8372 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:24.606863  4898 catalog_manager.cc:5697] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:24.569211  8030 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.571509  8390 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.556165  8125 raft_consensus.cc:697] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:24.540423  8055 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:24.630138  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 129514us
I20260709 12:24:24.635376  4898 catalog_manager.cc:5697] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 6 to 8, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 8 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:24.600829  8162 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:24.613689  8389 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:24.638710  5189 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:24.605623  8372 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:24.647078  8396 log.cc:927] Time spent T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:24:24.647908  8372 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.649240  8372 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:24.640643  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:24.557318  7924 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:24.644068  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 148610us
I20260709 12:24:24.645179  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:24.645824  8398 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.628176  8333 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:24.646677  8343 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:24.619426  8216 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:24.660420  5071 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:24.666268  4898 catalog_manager.cc:5697] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 3 to 6, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 6 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:24.623615  8317 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:24.632172  8070 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:24.628403  8407 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:24.687712  8400 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.689075  8327 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:24.688804  8162 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.689481  8125 consensus_queue.cc:237] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:24.651463  8391 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:24.687275  8230 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:24.706135  8216 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:24.706821  8406 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:24:24.681067  8398 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:24.705861  8070 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:24.645172  8098 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:24.713305  8397 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:24:24.716957  8413 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:24.721607  8417 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:24.722466  8414 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:24.750907  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:24.751657  5219 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:24.668788  8389 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.650995  8180 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:24.738740  8391 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:24.729508  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:24.738386  8230 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:24.738243  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:24.634698  8030 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:24.763767  8030 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:24.729782  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:24.764837  5067 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:24.766531  8389 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:24.746938  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:24.631767  8390 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.693394  8407 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.746555  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:24.770593  5222 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
W20260709 12:24:24.753804  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 62249us
I20260709 12:24:24.734552  8317 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.678536  8035 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:24:24.697234  5069 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
W20260709 12:24:24.729040  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.770443  5074 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
W20260709 12:24:24.758809  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.760083  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 122585us
I20260709 12:24:24.761368  8055 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:24.758301  5220 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:24.752837  8414 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.743000  8400 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:24.768088  8391 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:24.768103  8311 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:24.789960  8311 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:24.795367  8398 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:24.777932  8317 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.772989  8407 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:24.777371  8389 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:24.805466  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:24.809010  5219 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:24.781085  8414 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:24.811587  8398 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:24.812674  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:24.815665  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:24.817708  5222 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:24:24.819291  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:24.820297  5222 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:24.821592  8055 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.822265  8055 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.824604  5220 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:24.825094  8055 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:24.754457  8413 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:24.825965  8213 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:24.800212  8317 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:24.811017  8389 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.862267  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:24.862411  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 56043.7us
I20260709 12:24:24.862870  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:24.864260  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:24.865211  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:24.866019  5941 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:24.867714  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:24.833194  8035 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:24.873597  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:24.876576  8311 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:24.878753  8400 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:24.878965  8413 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:24.833653  8415 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:24:24.867924  8389 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:24.879252  8311 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:24.924245  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:24.924907  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 157388us
I20260709 12:24:24.848642  8213 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:24.886106  8035 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:24.933014  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 68893.3us
W20260709 12:24:24.933027  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 58522.7us
I20260709 12:24:24.935248  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:24.918787  8311 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:24.935595  5114 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:24.934381  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:24.935961  5222 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:24.931713  8372 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:24.936895  7924 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:24.949414  8306 consensus_meta.cc:287] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.509s	user 0.013s	sys 0.000s
I20260709 12:24:24.938624  8016 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:24.950067  8016 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.949914  7924 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:24.950024  8306 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:24.950654  8016 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:24.952559  8306 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:24.953016  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:24.953780  5074 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:24.953498  8035 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:24.967837  8425 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:24.969374  8422 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:24:24.943336  8400 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:24.943500  8372 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.963528  8311 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:24.952554  7924 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:24.942647  8180 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:24:24.970450  8070 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:24.993899  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:24.982580  8400 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:24.984681  8372 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:24.984652  8425 raft_consensus.cc:697] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:25.005527  5222 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:25.001821  8400 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:24.990259  8016 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:24.987946  8306 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
W20260709 12:24:25.025863  8423 log.cc:927] Time spent T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:24:25.030028  8425 consensus_queue.cc:237] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:25.016146  7924 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:25.029235  8327 raft_consensus.cc:3055] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:25.042022  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.037964  8213 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:25.043931  8417 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:24:25.061694  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 115136us
I20260709 12:24:24.990700  8311 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:24.990446  8407 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.071974  5114 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:25.073772  7924 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:25.083377  8035 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:25.083560  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:25.084491  7924 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:25.073547  8327 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:25.085083  8410 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:25.086174  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.086006  8327 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:25.087872  8327 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.076581  8417 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:25.090974  8417 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:25.092132  8417 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:25.076344  8016 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:24.697136  4898 catalog_manager.cc:5697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 7 to 10, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 10 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:25.095664  8407 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:25.100175  8035 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:25.069348  8434 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:25.118050  8434 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:25.120869  8434 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:25.103497  8391 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:25.104828  8414 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.106755  8279 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:25.106992  8372 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:25.111497  5074 raft_consensus.cc:1240] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:25.111932  8035 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:25.113300  8381 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.711s	user 0.000s	sys 0.005s
W20260709 12:24:25.132551  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 49049.4us
I20260709 12:24:25.137036  8035 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:25.118052  7788 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:25.152513  8380 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.667s	user 0.003s	sys 0.000s
I20260709 12:24:25.154758  8311 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:25.126307  8327 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.137872  8391 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:25.159497  8400 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:25.181192  8414 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:25.218910  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:25.230273  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:25.232764  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:25.264182  8404 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.615s	user 0.003s	sys 0.000s
I20260709 12:24:25.272264  8258 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.274518  8258 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.277529  8258 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:25.279789  8258 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:25.281754  8258 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:25.282846  8258 raft_consensus.cc:3055] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:25.283277  8258 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:25.284305  8258 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:25.287231  8258 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:25.306137  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.307260  5216 raft_consensus.cc:2393] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:25.309636  5216 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:25.310107  8230 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:25.314230  5219 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:25.315116  8230 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:25.311316  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.314021  8394 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:25.318753  8394 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.321427  8394 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:25.323944  8394 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:25.324513  8394 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.335289  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.336301  5216 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
W20260709 12:24:25.338389  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.339116  8394 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:25.344368  5899 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:25.348497  8425 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:25.350520  5899 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:25.351686  5899 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:25.353751  8258 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:25.349079  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:25.363070  8425 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:25.362406  8258 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.367583  8258 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.362406  8216 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:25.402509  8216 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:25.403867  8258 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:25.404731  8258 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:25.405253  8258 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:25.405751  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.407212  8216 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:25.408079  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.408713  8216 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:25.409471  8216 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:25.405791  8258 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:25.419260  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 66612.6us
I20260709 12:24:25.410068  8216 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:25.424552  8216 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:25.425268  8216 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:25.180707  8372 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:25.432206  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.434162  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.435706  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.436442  5143 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:25.440824  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.444664  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.445645  5216 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:25.446900  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:25.447218  8425 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.447814  5219 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:25.447844  8425 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.448770  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:25.171986  8327 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:25.451100  8327 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:25.451501  8425 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:25.452085  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.452826  5216 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:25.454193  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.455371  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.452222  5141 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:25.458096  5146 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:24:25.458577  5146 pending_rounds.cc:85] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 4
I20260709 12:24:25.458791  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.458832  5146 pending_rounds.cc:107] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:24:25.459826  5146 raft_consensus.cc:2889] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:25.460311  8445 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:25.460549  5221 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:25.461122  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:25.461970  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.462430  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.462658  8425 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.463230  8425 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.459342  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.464013  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.464673  5145 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:25.465327  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.465651  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.465830  8425 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:25.466691  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.466962  5899 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:25.467622  5216 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 12:24:24.548659  4895 rpcz_store.cc:269] 0709 12:24:04.801565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.844185 (+ 42620us) service_pool.cc:224] Handling call
0709 12:24:24.448235 (+19604050us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:20.178306 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:20.420456 (+242150us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:20.430139 (+  9683us) write_op.cc:435] Acquired schema lock
0709 12:24:20.430179 (+    40us) tablet.cc:662] Decoding operations
0709 12:24:20.439560 (+  9381us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:20.439651 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:20.439743 (+    92us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:24:20.441364 (+  1621us) tablet.cc:701] Row locks acquired
0709 12:24:20.441393 (+    29us) write_op.cc:260] PREPARE: finished
0709 12:24:20.484026 (+ 42633us) write_op.cc:270] Start()
0709 12:24:20.508175 (+ 24149us) write_op.cc:276] Timestamp: P: 1783599860476427 usec, L: 0
0709 12:24:20.512425 (+  4250us) op_driver.cc:348] REPLICATION: starting
0709 12:24:20.624803 (+112378us) log.cc:844] Serialized 5154 byte log entry
0709 12:24:21.166723 (+541920us) op_driver.cc:464] REPLICATION: finished
0709 12:24:21.742953 (+576230us) write_op.cc:301] APPLY: starting
0709 12:24:21.756586 (+ 13633us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:21.806035 (+ 49449us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:21.818767 (+ 12732us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:21.975948 (+157181us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:22.035286 (+ 59338us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 12:24:22.056507 (+ 21221us) write_op.cc:312] APPLY: finished
0709 12:24:22.056898 (+   391us) log.cc:844] Serialized 94 byte log entry
0709 12:24:22.057116 (+   218us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:22.057301 (+   185us) write_op.cc:454] Released schema lock
0709 12:24:22.057359 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:22.057837 (+   478us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:22.057868 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2385035,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":535691,"apply.run_cpu_time_us":6254,"apply.run_wall_time_us":330980,"num_ops":13,"prepare.queue_time_us":85973,"prepare.run_cpu_time_us":22305,"prepare.run_wall_time_us":788620,"raft.queue_time_us":111244,"raft.run_cpu_time_us":13253,"raft.run_wall_time_us":380431,"replication_time_us":643509,"thread_start_us":618993,"threads_started":4,"wal-append.queue_time_us":415369,"wal-append.run_cpu_time_us":42252,"wal-append.run_wall_time_us":2060279}]]}
I20260709 12:24:25.432870  8258 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:25.468508  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.469758  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.466682  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.472688  8258 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:25.480875  5146 raft_consensus.cc:2368] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12.
I20260709 12:24:25.481158  8258 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:25.481948  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.481894  8258 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:25.482745  5146 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:25.483700  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.484782  8258 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:25.485512  8258 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:25.487267  8258 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:25.488513  8390 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:25.489068  8258 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:25.470311  5145 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:25.489629  8258 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.490249  8258 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:25.495992  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.498225  8327 consensus_queue.cc:1059] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:25.499301  8327 raft_consensus.cc:3055] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:25.499500  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:25.499693  8327 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:25.500290  5074 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:25.503134  5146 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:25.498560  8372 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.504272  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.500771  8327 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:25.128197  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:25.519435  5042 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:25.504483  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.522508  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.524147  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:25.524849  5944 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:25.525319  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.526036  5946 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:25.526932  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:25.527420  5946 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:24:25.479527  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:25.509476  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:25.529515  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.529783  8372 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:25.529680  8434 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:25.530202  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 50819.3us
I20260709 12:24:25.531042  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.531665  5144 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:25.531926  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:25.532553  8372 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:25.532725  8434 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:25.521734  5222 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:25.534941  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:25.535676  8411 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:25.474169  8425 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:24:25.537405  8343 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:25.537771  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:25.537935  8425 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:25.538429  8425 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:25.539711  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:25.539443  8425 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.479734  8445 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:25.504335  8427 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.540864  8425 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:25.542328  8427 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:25.497470  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.541888  8445 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.546631  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.449347  5219 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:25.476804  8378 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:25.511250  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.532584  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.548722  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:25.479636  8447 raft_consensus.cc:3055] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:25.550338  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:25.551005  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.591553  5143 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
W20260709 12:24:25.594182  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 56529.8us
I20260709 12:24:25.588946  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:25.551760  8035 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:25.554472  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.508105  5216 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:25.563979  5067 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:25.566619  5076 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:24:25.597862  5076 pending_rounds.cc:85] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 6
I20260709 12:24:25.598233  5076 pending_rounds.cc:107] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:24:25.598556  5076 raft_consensus.cc:2889] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:25.567514  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:25.562920  5145 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:25.570396  5073 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:25.572307  5948 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:25.577272  5075 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:25.579699  5944 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:25.584789  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:25.593804  8447 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:25.601449  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:25.601867  8378 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:25.601658  8447 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.518193  5221 raft_consensus.cc:1240] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:24:25.606166  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.607518  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:25.608248  5944 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:25.604636  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:25.610224  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.610585  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:25.610741  8333 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:25.602849  8317 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:25.603678  8372 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:25.611882  8317 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.612047  8372 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.612367  8317 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:25.612528  8372 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:25.613677  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.615012  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.615561  5948 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:25.610374  8447 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:25.611544  8411 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:25.626682  8411 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:25.626775  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:25.627614  8411 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:25.628182  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:25.612573  8237 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:24:25.625335  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.631210  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.631531  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:25.596011  5042 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:25.611554  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.612370  8434 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:25.625749  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.638732  5219 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:25.639142  5144 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:25.639333  5042 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:25.639997  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.634498  5940 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:25.640471  5042 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:25.640913  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:25.641634  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:25.643258  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:25.644004  5940 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:25.634956  5946 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:25.632319  5074 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:24:25.646358  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.647095  5042 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:25.625267  8216 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:25.665936  8030 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.666550  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.666668  8030 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.668423  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.669188  5939 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:25.648236  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.646735  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.587154  8258 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.669325  8030 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:25.694438  8016 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.688794  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:25.695274  8016 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:25.697340  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.697798  8030 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:25.698349  5148 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:25.698531  8162 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:25.699465  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.700881  8016 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:25.705127  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:25.705235  5221 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:25.658130  8378 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.707535  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:25.708288  5221 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:25.658342  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:25.671183  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:25.710322  5074 raft_consensus.cc:2410] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:25.711794  5074 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:25.659446  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.594003  8428 consensus_queue.cc:1059] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:24:25.660255  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.712647  5069 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:25.715611  5222 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:25.706359  8098 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:25.717825  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:25.718708  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.719337  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.720077  5939 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:25.724889  5948 raft_consensus.cc:2368] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10.
W20260709 12:24:25.725123  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 54061.8us
I20260709 12:24:25.717285  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.726508  5222 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:24:25.726624  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:25.645625  8434 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.720932  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:25.648447  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:25.728639  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:25.728948  5900 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:25.721064  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.715034  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.730913  5144 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:25.713891  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:25.731922  5219 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:24:25.656945  5944 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:25.732766  5115 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:25.742302  5939 raft_consensus.cc:2410] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:25.743628  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:25.745290  5190 leader_election.cc:400] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12.
I20260709 12:24:25.745967  5190 leader_election.cc:403] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:24:25.746234  5944 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:25.746476  5115 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:25.753258  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:25.754348  5939 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:25.754468  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:25.767469  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 120438us
I20260709 12:24:25.767109  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.773103  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:25.777379  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.653926  8425 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:25.783234  5939 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:25.784741  5940 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:25.741817  8434 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:25.794938  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.796357  5144 raft_consensus.cc:2368] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:25.727982  8216 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:25.752827  8016 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:25.753239  8428 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:25.744956  8391 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:25.798731  5940 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:25.753677  8162 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:25.752521  8237 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:25.825266  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.826010  5144 raft_consensus.cc:2376] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7. Re-sending same reply.
I20260709 12:24:25.745223  8333 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:25.744035  8258 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:25.759094  7788 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:25.782054  7924 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:25.781575  8445 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.828186  5940 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:25.786209  8163 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12.
I20260709 12:24:25.824568  8415 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.827255  5219 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:25.848212  8333 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:25.848862  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:25.849695  8428 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:25.848863  8333 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:25.850652  8428 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:25.851087  8428 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.851665  8428 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.851020  8333 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:25.853989  7788 raft_consensus.cc:697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:25.854391  8311 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:25.855474  8311 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:25.855281  7788 consensus_queue.cc:237] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:25.820967  5221 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:25.856135  8311 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:25.858006  8415 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:25.854086  5945 raft_consensus.cc:2376] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13. Re-sending same reply.
W20260709 12:24:25.859436  8393 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.001s	sys 0.011s
W20260709 12:24:25.859824  8393 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.001s	sys 0.011s
I20260709 12:24:25.821913  8016 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:25.826804  8411 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:25.860062  5945 raft_consensus.cc:3055] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:25.860642  8415 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:25.814703  8390 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:25.877285  8390 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:25.878702  8411 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:25.879788  8390 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:25.879987  8411 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:25.880407  8411 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:25.880784  8411 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:25.880954  8030 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:24:25.862709  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 119191us
I20260709 12:24:25.882083  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:25.884008  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:25.839207  7924 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:25.885165  7924 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.886516  8311 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:25.887179  8311 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:25.888554  5946 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:25.862901  5221 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:25.861776  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:25.866279  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.873433  5945 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:25.891454  5945 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:25.892715  5945 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:25.872331  8016 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.894420  5144 raft_consensus.cc:2393] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:25.898910  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:25.901263  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 116039us
I20260709 12:24:25.846546  8445 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:25.847424  8372 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:25.911175  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 156765us
I20260709 12:24:25.914213  5115 leader_election.cc:400] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10.
I20260709 12:24:25.914665  5115 leader_election.cc:403] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:25.918674  5221 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:25.918619  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.897771  8415 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:25.927353  5144 raft_consensus.cc:2393] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:25.932099  5945 raft_consensus.cc:1275] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:24:25.933195  5945 pending_rounds.cc:85] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 4
I20260709 12:24:25.933529  5945 pending_rounds.cc:107] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:24:25.934211  5945 raft_consensus.cc:2889] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:25.940340  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 167454us
I20260709 12:24:25.941838  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:25.942953  5067 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 9: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:25.948292  5147 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:24:25.953940  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.955642  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.956964  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.960569  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.961920  5147 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:25.962334  8451 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:24:25.964339  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.965230  5150 raft_consensus.cc:2368] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10.
I20260709 12:24:25.964030  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.966565  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.967257  5150 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:25.967316  5147 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:25.968698  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:25.974444  8445 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:25.975780  8456 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:25.976405  8456 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:25.982246  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:25.986528  5217 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:25.989698  5144 raft_consensus.cc:2393] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:25.897505  8317 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:25.997081  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.999987  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1184) took 1164 ms. Trace:
I20260709 12:24:25.969355  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.001499  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.003760  5147 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:25.931574  8425 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:26.007360  5144 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:26.004307  5220 rpcz_store.cc:276] 0709 12:24:24.835966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.836219 (+   253us) service_pool.cc:224] Handling call
0709 12:24:24.840675 (+  4456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.841021 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:24.841429 (+   408us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.850401 (+1008972us) spinlock_profiling.cc:243] Waited 5.3 ms on lock 0x7b180036ec20. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db23a 00007fd8a8545a2a
0709 12:24:25.998567 (+148166us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.999406 (+   839us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:25.999473 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:25.999550 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.999685 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.999900 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3645,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":2509,"raft.queue_time_us":3838,"spinlock_wait_cycles":26455552,"thread_start_us":146516,"threads_started":1}
I20260709 12:24:26.011934  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:25.910421  8216 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.013356  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.013154  5147 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:26.017040  5144 raft_consensus.cc:2410] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:26.002317  8163 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:26.022926  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.009099  8098 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:26.030174  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:26.032480  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.038264  8317 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:26.039682  8454 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:24:25.989051  8456 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.042706  5147 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:26.018218  8372 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:26.068864  5147 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:26.069036  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 155088us
I20260709 12:24:26.070050  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:26.071450  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:26.058997  7788 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:26.061839  8391 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:26.072543  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 69804.5us
I20260709 12:24:26.076663  5900 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:26.078477  5190 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:26.080143  8393 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:26.082966  8447 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:26.084429  5190 leader_election.cc:400] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:26.085016  5190 leader_election.cc:403] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:26.085932  8393 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:26.089279  7924 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:26.091187  8259 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:26.093034  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:26.094352  8434 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:26.095557  5148 raft_consensus.cc:2368] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:26.078730  8391 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.097440  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:26.098217  5072 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:26.098582  8391 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:26.099186  8391 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:26.093043  7924 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:26.100083  8452 log.cc:927] Time spent T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 12:24:26.074640  7788 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:26.100790  7788 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:26.079475  8372 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.114745  5067 raft_consensus.cc:1275] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:26.137665  8470 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:26.138451  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.139617  5072 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:26.141497  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:26.143653  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:26.145612  5042 leader_election.cc:400] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10.
I20260709 12:24:26.146160  5042 leader_election.cc:403] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:24:26.147045  8317 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10.
I20260709 12:24:26.147573  8411 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:26.148272  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:26.149314  8317 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:26.149590  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:26.107079  8425 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:26.151686  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.154707  5042 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:26.130429  8333 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:26.158982  8472 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:26.123814  8372 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.154507  8470 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:26.149833  8317 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:26.173424  8372 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:26.179553  8461 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:24:26.179840  8317 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:26.185972  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:26.159370  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:26.166860  5222 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:24:26.161152  8016 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:26.162057  8311 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:26.189947  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.167016  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.202026  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:26.156345  8464 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:24:26.207618  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:26.194808  4930 debug-util.cc:398] Leaking SignalData structure 0x7b0800196580 after lost signal to thread 1520
I20260709 12:24:26.206823  8055 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:26.213270  8055 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.213758  8055 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:26.213919  8428 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:26.216343  8016 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:26.229215  8428 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:26.230020  8372 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:26.229017  8410 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.143s	user 0.009s	sys 0.005s
I20260709 12:24:26.234500  8410 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:26.235993  8400 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:26.240257  8472 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:26.242657  5042 leader_election.cc:400] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:26.243188  5042 leader_election.cc:403] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:24:26.201146  8216 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:26.247768  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.252568  5144 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:26.251941  8216 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:26.264376  8216 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:26.266585  8471 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:24:26.270737  8343 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:26.271358  8343 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:26.272562  8343 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:26.262027  8372 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:26.263228  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 50530us
I20260709 12:24:26.284777  5144 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:26.286774  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.288151  5144 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:26.289700  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.294049  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:26.295073  5939 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:26.263660  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 61765.7us
I20260709 12:24:26.277513  8391 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:26.298570  8259 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:26.304522  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.305176  8454 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10.
I20260709 12:24:26.242295  8333 raft_consensus.cc:697] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:26.309978  8055 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:26.311286  8259 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.252496  8317 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:26.310209  8343 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:26.315621  8311 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:26.317399  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:26.317834  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 125675us
I20260709 12:24:26.323927  5145 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:26.318938  5144 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:26.319979  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.325124  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.327656  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:26.328397  5940 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:26.329128  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.330024  5042 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:26.331002  5115 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:26.331295  5900 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:26.332273  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.332926  8017 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:26.333861  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.334585  5142 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:26.336850  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:26.337090  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:26.337524  5222 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:26.339704  8055 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:26.340685  8213 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:26.343757  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.348198  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:26.349200  5149 raft_consensus.cc:2393] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:26.359704  8447 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:26.360236  5149 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:24:26.360481  8259 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:26.360764  5148 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:26.362058  5148 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:26.363207  5148 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:26.365391  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 582) took 1053 ms. Trace:
I20260709 12:24:26.351322  8216 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.333410  8017 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.352331  8407 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:26.323904  7788 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:26.372754  5940 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:26.369948  8259 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
W20260709 12:24:26.375324  8463 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.289s	user 0.002s	sys 0.000s
I20260709 12:24:26.341378  8213 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:26.343058  8055 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.356117  8317 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:26.378057  8055 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.356848  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:26.357591  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.358444  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:26.379534  5943 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:26.379813  5145 raft_consensus.cc:2393] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:26.380156  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 781) took 1127 ms. Trace:
I20260709 12:24:26.380510  5943 rpcz_store.cc:276] 0709 12:24:25.258302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.356546 (+1098244us) service_pool.cc:224] Handling call
0709 12:24:26.380091 (+ 23545us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.366540  8017 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:26.342298  8410 consensus_queue.cc:1048] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:24:26.382186  8259 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:26.383069  8259 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:26.384986  8333 consensus_queue.cc:237] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:26.389050  8213 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:26.368234  5898 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:26.369181  8447 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:24:26.369943  4930 debug-util.cc:398] Leaking SignalData structure 0x7b080032ce00 after lost signal to thread 4933
I20260709 12:24:26.366603  5142 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:26.370282  8391 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.370810  8411 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:26.370967  8373 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:26.395300  8373 raft_consensus.cc:697] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:26.396533  8259 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:26.396286  8373 consensus_queue.cc:237] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:26.396857  5142 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:26.379671  8434 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:26.373158  7788 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:26.390187  8055 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:26.365876  5224 rpcz_store.cc:276] 0709 12:24:25.312087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.364582 (+1052495us) service_pool.cc:224] Handling call
0709 12:24:26.364876 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.365059 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.365084 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.365128 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:26.365158 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.365210 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.365324 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.384940  5145 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:26.410250  8372 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9.
I20260709 12:24:26.391848  5222 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:26.393213  5148 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:26.394021  8265 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:26.393886  8391 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.415110  8391 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:26.366850  5939 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:26.408380  8447 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:24:26.399695  5898 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:26.424118  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 85511.8us
I20260709 12:24:26.425161  5041 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:26.428625  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 75173.8us
I20260709 12:24:26.449079  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:26.449863  5946 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:26.452630  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:26.452939  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:26.454517  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 180504us
I20260709 12:24:26.455927  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:26.417369  8216 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:26.458747  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:26.462656  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:26.438722  8125 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:26.465989  5941 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:26.467952  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:26.468693  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.441658  8265 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:26.468662  5943 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:26.441247  8407 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:26.457754  8391 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:26.461524  8372 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:26.472426  8265 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:26.486207  5946 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:26.454074  8384 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:26.477033  8480 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:26.493930  5943 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
W20260709 12:24:26.485877  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 155935us
I20260709 12:24:26.518796  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:26.520533  5076 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:26.522833  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:26.524811  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:26.482254  8475 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:24:26.527277  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:26.493221  8470 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:26.502905  8265 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.476781  8017 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:26.477404  8476 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:24:26.533447  5072 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
W20260709 12:24:26.545735  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 71425.6us
I20260709 12:24:26.510820  8480 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.491467  8391 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.495015  8372 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:26.548017  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:26.550733  5069 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:26.554708  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:26.556121  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:26.564762  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.559587  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:26.569131  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:26.551739  8480 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:26.574389  5069 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:26.573189  5115 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:26.555996  8476 consensus_queue.cc:1243] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:26.577382  5218 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:26.576959  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:26.576800  5114 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:26.565764  8445 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:26.580080  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 584) took 1267 ms. Trace:
I20260709 12:24:26.582010  5942 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:26.583015  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 784) took 1323 ms. Trace:
I20260709 12:24:26.580643  5218 rpcz_store.cc:276] 0709 12:24:25.312677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.573385 (+1260708us) service_pool.cc:224] Handling call
0709 12:24:26.575727 (+  2342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.578053 (+  2326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.579982 (+  1929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.572742  8384 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:26.583539  5942 rpcz_store.cc:276] 0709 12:24:25.259379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.573299 (+1313920us) service_pool.cc:224] Handling call
0709 12:24:26.581796 (+  8497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.582631 (+   835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.582936 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.549181  8407 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:26.588701  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1153) took 1174 ms. Trace:
W20260709 12:24:26.590142  8481 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:24:26.566375  8464 consensus_queue.cc:1048] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:24:26.585070  8476 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:26.594341  8163 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:26.571785  8427 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:26.601379  5149 rpcz_store.cc:276] 0709 12:24:25.414698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.361122 (+946424us) service_pool.cc:224] Handling call
0709 12:24:26.361438 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:26.444749 (+ 83311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.444785 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:26.466449 (+ 21664us) log.cc:844] Serialized 52 byte log entry
0709 12:24:26.537456 (+ 71007us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:26.538001 (+   545us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.538481 (+   480us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.538721 (+   240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.538839 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.588419 (+ 49580us) raft_consensus.cc:1685] finished
0709 12:24:26.588446 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.588632 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86544,"prepare.queue_time_us":614,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":80,"raft.queue_time_us":80015,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":15408,"spinlock_wait_cycles":4509440,"thread_start_us":43911,"threads_started":1,"wal-append.queue_time_us":103819}
I20260709 12:24:26.618927  8017 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:26.582710  8445 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:26.619794  8445 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:26.579572  8470 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:26.629418  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.613143  8486 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:26.601387  8447 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.602748  8384 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:26.610961  8343 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:26.643327  8384 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:26.611061  8476 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:26.626410  8333 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:26.612498  8163 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:26.650007  5070 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:26.659001  5145 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:26.661954  5143 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:24:26.662173  8488 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:24:26.662626  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1061) took 1110 ms. Trace:
I20260709 12:24:26.663030  5143 rpcz_store.cc:276] 0709 12:24:25.552522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.661146 (+1108624us) service_pool.cc:224] Handling call
0709 12:24:26.661508 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.662364 (+   856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.662568 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.664484  5143 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:26.665174  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1063) took 1085 ms. Trace:
I20260709 12:24:26.665508  5143 rpcz_store.cc:276] 0709 12:24:25.579934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.663836 (+1083902us) service_pool.cc:224] Handling call
0709 12:24:26.664158 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.664827 (+   669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.665102 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.650480  8486 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:26.669215  8035 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:26.669442  8486 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:26.670637  8445 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.671123  8163 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:26.673377  8163 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:26.678639  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:26.678879  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:26.679448  5143 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:24:26.679446  5221 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:24:26.679822  5143 pending_rounds.cc:85] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 5
I20260709 12:24:26.680088  5143 pending_rounds.cc:107] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:24:26.680233  5143 raft_consensus.cc:2889] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:26.681506  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:26.685995  8476 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:26.687484  5222 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:26.687740  5146 raft_consensus.cc:3055] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:26.688107  5146 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:26.688822  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:26.688858  5146 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:26.690708  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1159) took 1274 ms. Trace:
I20260709 12:24:26.655862  8333 raft_consensus.cc:697] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:26.691507  5146 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:26.692399  8333 consensus_queue.cc:237] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:26.681272  5218 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:26.696135  8265 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.697564  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:26.699414  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:26.701187  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:26.702780  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:26.704617  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:26.691104  5148 rpcz_store.cc:276] 0709 12:24:25.416581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.363824 (+947243us) service_pool.cc:224] Handling call
0709 12:24:26.392988 (+ 29164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.411543 (+ 18555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.690496 (+278953us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b54004fae50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:26.690628 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66376064}
I20260709 12:24:26.705863  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:26.707142  5148 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:24:26.707507  5148 pending_rounds.cc:85] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 4
I20260709 12:24:26.707746  5148 pending_rounds.cc:107] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:24:26.707782  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:26.708039  5148 raft_consensus.cc:2889] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:26.709336  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:26.712941  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:26.713841  5222 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:26.642843  8447 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:26.716544  8372 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:26.714404  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1064) took 1098 ms. Trace:
I20260709 12:24:26.720474  5143 rpcz_store.cc:276] 0709 12:24:25.582534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.668362 (+1085828us) service_pool.cc:224] Handling call
0709 12:24:26.679092 (+ 10730us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.680882 (+  1790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.681030 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.724161  5221 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:26.728389  5146 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:26.717991  8447 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:26.711215  8480 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.738471  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 790) took 1288 ms. Trace:
I20260709 12:24:26.739002  5944 rpcz_store.cc:276] 0709 12:24:25.450218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.733819 (+283601us) service_pool.cc:224] Handling call
0709 12:24:25.745731 (+ 11912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.747041 (+  1310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.738199 (+991158us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b54002418d0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:26.738394 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26257664}
I20260709 12:24:26.739811  8480 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:26.722747  5041 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:26.749368  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:26.751242  8391 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:26.751964  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1047) took 1284 ms. Trace:
I20260709 12:24:26.752179  5948 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:26.752547  5145 rpcz_store.cc:276] 0709 12:24:25.467998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.405623 (+937625us) service_pool.cc:224] Handling call
0709 12:24:26.405982 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.659552 (+253570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.751623 (+ 92071us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54004fa954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:26.751873 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581321088}
I20260709 12:24:26.747323  8480 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:26.753760  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:26.754315  8480 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:26.755689  5222 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:26.755366  8480 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:26.760707  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 183801us
I20260709 12:24:26.764231  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:26.766273  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:26.772173  8311 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:26.772917  5216 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:26.775337  5900 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:26.775542  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:26.780310  8098 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:26.788010  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:26.759366  8391 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:26.791925  8373 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:26.804389  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:26.793423  8503 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:24:26.796205  8487 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.212s	user 0.002s	sys 0.000s
I20260709 12:24:26.782984  8311 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:24:26.783401  8447 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:24:26.807286  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.810101  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1043) took 1348 ms. Trace:
I20260709 12:24:26.677395  8445 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:26.807824  8055 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:26.818921  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:26.816713  5142 rpcz_store.cc:276] 0709 12:24:25.461240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.397721 (+936481us) service_pool.cc:224] Handling call
0709 12:24:26.398050 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:26.455794 (+ 57744us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:26.455827 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:26.460658 (+  4831us) log.cc:844] Serialized 26 byte log entry
0709 12:24:26.541771 (+ 81113us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:26.541843 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.541957 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.795573 (+253616us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:26.809633 (+ 14060us) raft_consensus.cc:1685] finished
0709 12:24:26.809679 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.810009 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44882,"raft.queue_time_us":59140,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":7453,"spinlock_wait_cycles":11214720,"thread_start_us":63789,"threads_started":1,"wal-append.queue_time_us":95439}
I20260709 12:24:26.802389  8470 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.827731  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:26.798303  7924 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:24:26.817448  8447 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.831728  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:26.837368  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.837754  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.839131  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.824801  8411 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:26.835255  5041 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:26.843549  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.828240  7788 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:26.806810  8391 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.852262  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:26.847316  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:26.847661  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.847977  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.836426  8447 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:26.852043  5146 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:26.832463  8479 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:26.853744  5143 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:26.850267  5942 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:26.856644  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.834851  8343 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.846644  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:26.836056  8265 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:26.859822  5142 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:26.845300  8216 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:26.846093  8213 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:26.849327  7788 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:26.832777  8125 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
W20260709 12:24:26.867431  8502 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:24:26.881444  7788 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:26.861578  5141 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:26.827875  8470 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:26.840984  8055 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:24:26.860874  8016 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:24:26.862310  8414 consensus_queue.cc:1059] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:24:26.855580  8411 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:24:26.854187  8279 consensus_queue.cc:1059] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:24:26.919339  8279 raft_consensus.cc:3055] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:26.919919  8279 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:26.912127  8213 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:26.923221  8213 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:26.858383  8163 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:24:26.929298  5145 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:26.881068  8216 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:26.938539  5941 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 12:24:26.940825  8400 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:26.942468  8476 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:26.942848  8476 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:26.943774  8476 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:26.945057  8476 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:26.955364  8510 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:26.958520  5216 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:26.961794  5216 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 12:24:26.962180  8510 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:26.865168  8343 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:26.967592  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 143984us
I20260709 12:24:26.969122  8265 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:26.973482  8504 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:26.920992  8279 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:27.009023  8279 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:27.012063  8216 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:26.993808  8391 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:27.033612  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 174864us
W20260709 12:24:27.042090  4930 debug-util.cc:398] Leaking SignalData structure 0x7b08001ab120 after lost signal to thread 5100
W20260709 12:24:27.044073  4930 debug-util.cc:398] Leaking SignalData structure 0x7b0800380680 after lost signal to thread 5174
W20260709 12:24:27.044514  4930 debug-util.cc:398] Leaking SignalData structure 0x7b0800395480 after lost signal to thread 5175
I20260709 12:24:27.047993  5073 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:24:27.048079  4930 debug-util.cc:398] Leaking SignalData structure 0x7b0800387400 after lost signal to thread 5248
I20260709 12:24:27.049837  5071 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:27.054342  5075 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:26.991964  8384 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:27.054404  5069 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:27.057579  5068 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:27.125177  8515 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:24:27.129356  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.134858  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 142172us
W20260709 12:24:27.137575  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 97464us
I20260709 12:24:27.141592  8425 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.142236  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:27.142462  8425 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:27.144659  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:27.145792  8425 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:27.149319  5941 raft_consensus.cc:2410] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:24:27.152215  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:27.153048  5942 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:27.154690  8523 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:27.156898  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:27.160076  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 309415us
I20260709 12:24:27.159245  8518 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:27.167176  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:27.168148  5075 raft_consensus.cc:2368] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:27.209244  5141 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
W20260709 12:24:27.265792  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 104696us
I20260709 12:24:27.055197  8470 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:27.270318  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:27.288074  8509 log.cc:927] Time spent T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.095s	user 0.000s	sys 0.004s
I20260709 12:24:27.289278  5068 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:24:27.289387  8514 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.297s
I20260709 12:24:27.291242  8514 consensus_queue.cc:1243] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:24:27.292447  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:27.295879  5224 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:27.297585  5222 raft_consensus.cc:3055] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:27.298056  5222 raft_consensus.cc:740] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:27.298947  5222 consensus_queue.cc:260] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:27.300420  5222 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:27.301908  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:27.302625  5942 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:27.302510  8055 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:27.303656  8125 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:27.304011  8125 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:27.304832  8125 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:27.306244  8125 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:27.205291  8265 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.361006  5222 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:27.380965  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:27.376607  5069 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:27.293324  8514 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.211685  8425 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
W20260709 12:24:27.378892  8513 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:24:27.311794  8411 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 12:24:27.283334  8317 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.139s
I20260709 12:24:27.513653  5075 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
W20260709 12:24:27.465381  8519 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
W20260709 12:24:27.460613  8016 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.021s	sys 0.050s
I20260709 12:24:27.364951  5942 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
W20260709 12:24:27.535497  5900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.005s	sys 0.009s
I20260709 12:24:27.396274  8265 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:27.538216  5900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.005s	sys 0.009s
W20260709 12:24:27.540154  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 553058us
I20260709 12:24:27.541956  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:27.536684  8016 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.023s	sys 0.050s
I20260709 12:24:27.546015  8484 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:27.547762  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:27.422281  8216 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
W20260709 12:24:27.580549  5143 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.720s	user 0.013s	sys 0.001s
I20260709 12:24:27.581033  5143 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:27.583715  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:27.585747  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.595160  8425 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.596042  8425 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:27.598712  8523 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.599522  8523 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:27.605054  7788 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.721s	user 0.021s	sys 0.000s
I20260709 12:24:27.607180  8384 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:27.608072  8384 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:27.608774  8384 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:27.612128  8384 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:27.612283  8529 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:27.614487  8523 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:27.618100  8447 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.753s	user 0.016s	sys 0.015s
I20260709 12:24:27.620426  8425 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:27.620864  8265 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:27.539587  8343 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:27.605924  7788 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:27.672839  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:27.673978  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:27.674793  5070 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:27.675420  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:27.675838  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:27.676676  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:27.676906  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:27.677574  8523 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:27.677918  8163 consensus_queue.cc:1059] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:27.678210  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:27.678509  8523 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:27.678776  8163 consensus_queue.cc:1059] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:27.679253  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:27.679656  8523 raft_consensus.cc:3055] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:27.680147  8163 raft_consensus.cc:3055] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:27.680691  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:27.668221  8447 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:27.682613  5068 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:27.674000  7788 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:27.428635  8514 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:27.684537  8447 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:27.686591  8514 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:27.680151  8523 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:27.584769  5075 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:24:27.621836  8384 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:27.665696  8529 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:27.680672  8163 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:27.676990  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:27.622864  5148 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 12:24:27.697011  5070 raft_consensus.cc:2368] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
W20260709 12:24:27.728154  5115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.000s	sys 0.007s
W20260709 12:24:27.728849  5115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.000s	sys 0.007s
W20260709 12:24:27.729326  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.729030  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 236150us
I20260709 12:24:27.865744  5222 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:24:27.964537  5190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.015s	sys 0.024s
W20260709 12:24:27.965224  5190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.015s	sys 0.024s
I20260709 12:24:27.865178  5221 pending_rounds.cc:85] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 3
I20260709 12:24:27.990010  5221 pending_rounds.cc:107] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 12:24:27.990428  5221 raft_consensus.cc:2889] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:28.005868  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.579095  8484 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:27.695997  8384 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:28.148058  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.719442  8016 consensus_queue.cc:1059] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.265s
W20260709 12:24:28.163911  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.173225  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.173470  5040 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.009s	sys 0.018s
W20260709 12:24:27.989398  8125 consensus_meta.cc:287] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.683s	user 0.010s	sys 0.003s
W20260709 12:24:28.174011  5040 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.009s	sys 0.018s
W20260709 12:24:28.174041  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.849133  8521 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:28.180588  5116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.051s	sys 0.163s
W20260709 12:24:28.189725  5116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.053s	sys 0.163s
I20260709 12:24:28.222079  5071 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
W20260709 12:24:28.242604  5900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.073s	sys 0.072s
W20260709 12:24:28.243466  5900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.074s	sys 0.072s
I20260709 12:24:28.162710  8530 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:28.260021  5898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.028s	sys 0.043s
W20260709 12:24:28.262441  5898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.028s	sys 0.043s
I20260709 12:24:28.245000  5900 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:28.178812  8544 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.294694  5948 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:28.302745  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:28.306147  5943 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:24:28.260551  8445 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.331110  8539 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:28.331561  8529 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:28.332208  8529 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:28.332018  8539 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:28.332751  8529 raft_consensus.cc:697] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:28.333052  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 807) took 1063 ms. Trace:
I20260709 12:24:28.301705  8411 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:28.333803  8306 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:28.334123  8411 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:28.334431  8306 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:28.313511  8279 consensus_meta.cc:287] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.297s	user 0.015s	sys 0.000s
I20260709 12:24:28.336867  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1212) took 1066 ms. Trace:
I20260709 12:24:27.850448  8523 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:28.339393  5223 rpcz_store.cc:276] 0709 12:24:27.270894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.271023 (+   129us) service_pool.cc:224] Handling call
0709 12:24:27.288211 (+ 17188us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.323223 (+ 35012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.323282 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:27.376303 (+ 53021us) log.cc:844] Serialized 26 byte log entry
0709 12:24:27.718615 (+342312us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:27.718735 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.718999 (+   264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:28.336634 (+617635us) raft_consensus.cc:1685] finished
0709 12:24:28.336659 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.336807 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34432,"raft.queue_time_us":290342,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":38626,"thread_start_us":252542,"threads_started":1,"wal-append.queue_time_us":930952}
I20260709 12:24:28.305923  8343 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:24:28.302886  8544 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:28.233889  8521 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:28.239286  8163 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:28.294528  8530 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:28.336208  8445 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:28.333595  5941 rpcz_store.cc:276] 0709 12:24:27.269385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.269623 (+   238us) service_pool.cc:224] Handling call
0709 12:24:27.270018 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.539348 (+269330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.539382 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:27.540004 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 12:24:27.707380 (+167376us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:27.707438 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.707524 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:28.332773 (+625249us) raft_consensus.cc:1685] finished
0709 12:24:28.332800 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.332953 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":283141,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":38374,"thread_start_us":418684,"threads_started":2,"wal-append.queue_time_us":702340}
I20260709 12:24:28.392664  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:28.361981  8306 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:28.355084  8522 log.cc:927] Time spent T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.699s	user 0.000s	sys 0.002s
I20260709 12:24:28.383301  8384 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:28.346540  8279 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.401891  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:28.387666  8523 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:28.404573  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.386374  8533 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:28.363392  8539 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:28.434783  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:28.382620  8541 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:28.408298  5222 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:28.361289  8528 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:28.384653  8484 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:28.367678  8543 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.446552  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:28.385576  8544 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:28.448277  5223 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:28.454578  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:28.465258  8525 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:28.465709  8545 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.466321  8545 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:28.466204  8525 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:28.468935  8545 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:28.363703  8529 consensus_queue.cc:237] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:28.472605  8530 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:28.458353  8539 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.473804  8539 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:28.474422  8384 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:28.474999  8384 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:28.476357  8539 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:28.435443  8213 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:28.456130  8533 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:28.478013  8213 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:28.459822  8542 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:28.479472  8542 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:28.477691  8539 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:28.480515  8539 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:28.407693  8551 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:28.441802  8163 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:28.477856  8550 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.465339  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:28.479835  8533 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:28.415146  8306 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:28.469080  8546 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.514777  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:28.516531  5114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.996s	user 0.009s	sys 0.019s
I20260709 12:24:28.512501  5215 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
W20260709 12:24:28.516960  5114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.009s	sys 0.019s
I20260709 12:24:28.481254  8213 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:28.522930  5223 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:28.446370  8279 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:28.512223  8384 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:28.549314  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.540102  8213 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:28.564271  8539 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:28.566152  8521 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:28.460879  8528 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:28.566840  8535 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:28.567752  8261 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.544425  8542 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:28.525216  8306 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:28.592895  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1211) took 1322 ms. Trace:
I20260709 12:24:28.550670  8016 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.590221  8261 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:28.591683  8279 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:28.462215  8541 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:28.621167  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:28.622002  5223 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:28.602738  5189 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.65:37253 timed out after 1.523s (SENT) [suppressed 21 similar messages]
I20260709 12:24:28.593439  5218 rpcz_store.cc:276] 0709 12:24:27.270532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.271172 (+   640us) service_pool.cc:224] Handling call
0709 12:24:27.271569 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.694989 (+423420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.695043 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:27.720563 (+ 25520us) log.cc:844] Serialized 26 byte log entry
0709 12:24:28.408171 (+687608us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:28.408211 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.408375 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:28.592646 (+184271us) raft_consensus.cc:1685] finished
0709 12:24:28.592671 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.592827 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39446,"raft.queue_time_us":275066,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":14752,"thread_start_us":722449,"threads_started":2,"wal-append.queue_time_us":680554}
I20260709 12:24:28.623510  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:28.580781  5899 connection.cc:256] RPC call timeout handler was delayed by -1.1600045440000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1834s for a response
I20260709 12:24:28.464838  8543 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:28.624400  5218 raft_consensus.cc:2393] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 7: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:28.624830  5899 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:28.633387  8445 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:28.634308  8445 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:28.637746  8548 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:28.522784  8550 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:28.639086  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:28.586215  8213 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:28.640588  5899 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:28.646430  5223 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:28.647130  5899 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:24:28.648789  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 209698us
I20260709 12:24:28.525566  8546 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:28.653143  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:28.625072  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 202230us
I20260709 12:24:28.641477  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:28.636760  8306 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:28.510306  8551 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:28.645794  5218 raft_consensus.cc:2410] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:24:28.640295  8454 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:24:28.671363  5219 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:28.649993  5115 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:28.641425  8542 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:28.680171  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:28.676954  5897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.093s	user 0.001s	sys 0.013s
I20260709 12:24:28.643672  8539 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:28.638475  8548 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:28.686348  5897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.155s	user 0.002s	sys 0.013s
I20260709 12:24:28.687094  8425 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:28.687731  8425 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:28.690641  5217 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:28.631161  8535 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:28.715760  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:28.674096  8098 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:28.751897  8535 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:28.752841  8535 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:28.739076  5222 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:28.754770  8535 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:28.756196  8535 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:28.739625  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 136920us
I20260709 12:24:28.624609  8016 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:28.753649  5941 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
W20260709 12:24:28.780776  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.669759  8445 raft_consensus.cc:2764] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:28.783247  8445 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:28.788556  8464 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.835s	user 0.020s	sys 0.010s
W20260709 12:24:28.789805  8464 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.836s	user 0.021s	sys 0.010s
I20260709 12:24:28.792023  8464 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.717440  8496 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.727173  8560 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:28.825453  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 153933us
I20260709 12:24:28.785678  8445 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:28.686409  8542 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:28.716980  8539 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:28.849309  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 178617us
I20260709 12:24:28.804896  8163 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:28.804916  8425 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:28.870543  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 220590us
I20260709 12:24:28.872303  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:28.851569  8098 raft_consensus.cc:697] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:28.852671  8557 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:28.893784  8557 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:28.898973  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:28.901809  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 179259us
I20260709 12:24:28.907330  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:28.907769  5941 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:28.908176  5946 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:28.911723  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:28.914268  5946 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:28.909693  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 80049.2us
I20260709 12:24:28.838194  8496 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:28.917901  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 56988.5us
I20260709 12:24:28.845485  8560 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:28.922017  8560 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:28.923317  8479 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:28.923799  8479 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:28.897902  8425 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.927634  5939 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:28.887838  8548 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:28.926050  8425 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:28.940251  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:28.940973  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:28.952859  5141 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:28.953950  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:28.966784  5941 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:28.934226  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:28.967973  5940 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:28.968994  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:28.884972  8163 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:28.903681  8016 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:28.950703  5143 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:24:28.971431  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:28.972611  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:28.974283  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:28.942307  5146 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:28.951000  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:28.976320  8565 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:28.977602  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:28.978679  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:28.977130  8565 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:28.933173  8213 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:28.986665  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.984695  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:28.993741  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.994910  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.004734  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.005952  4930 debug-util.cc:398] Leaking SignalData structure 0x7b0800351100 after lost signal to thread 5973
W20260709 12:24:29.006377  4930 debug-util.cc:398] Leaking SignalData structure 0x7b080033e6a0 after lost signal to thread 5974
W20260709 12:24:29.007481  4930 debug-util.cc:398] Leaking SignalData structure 0x7b080035a200 after lost signal to thread 8375
W20260709 12:24:29.015002  4930 debug-util.cc:398] Leaking SignalData structure 0x7b08003312a0 after lost signal to thread 8409
I20260709 12:24:28.841656  8464 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:28.984375  8575 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:24:28.955214  8560 consensus_queue.cc:1059] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:24:28.982465  8528 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:28.985630  8496 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:28.984517  8550 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:28.983704  8586 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:29.052657  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:29.001803  8543 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:29.006575  8541 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:29.174994  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:29.059656  8569 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.187s	user 0.001s	sys 0.001s
I20260709 12:24:29.041998  8557 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:29.193069  8539 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:29.177116  8576 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 12:24:29.060097  8554 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:29.177989  8588 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:29.178948  8567 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:29.180369  8556 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.215s
I20260709 12:24:29.177168  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:29.402982  5947 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:29.185249  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:29.228945  8565 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:29.055073  8586 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:29.408967  8586 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:29.409392  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:29.410322  5224 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:29.411592  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:29.412358  5224 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
W20260709 12:24:29.229287  8549 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 12:24:29.229383  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:29.414207  5215 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:29.415069  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1210) took 2144 ms. Trace:
I20260709 12:24:29.415621  5216 rpcz_store.cc:276] 0709 12:24:27.270161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.270980 (+   819us) service_pool.cc:224] Handling call
0709 12:24:27.271357 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.696864 (+425507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.696902 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:27.717480 (+ 20578us) log.cc:844] Serialized 26 byte log entry
0709 12:24:28.397032 (+679552us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:28.397091 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.397166 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.414788 (+1017622us) raft_consensus.cc:1685] finished
0709 12:24:29.414817 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.414980 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6350,"raft.queue_time_us":281575,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"thread_start_us":516062,"threads_started":2,"wal-append.queue_time_us":659075}
I20260709 12:24:29.016537  8098 consensus_queue.cc:237] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:29.191332  8456 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:29.298345  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.272871  8539 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:29.272035  8557 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:29.174439  8546 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:29.229964  8554 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:29.261658  8555 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 12:24:29.057731  8528 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.141s
W20260709 12:24:29.286242  8572 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:24:29.463522  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 632) took 1300 ms. Trace:
W20260709 12:24:29.329468  8511 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:24:29.230767  8445 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:29.316680  5141 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:29.199577  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:29.276036  8551 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:29.326673  8542 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:29.413255  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:29.489109  5224 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
W20260709 12:24:29.419130  5942 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.026s	sys 0.045s
W20260709 12:24:29.490291  5942 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.027s	sys 0.046s
I20260709 12:24:29.433507  8554 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:29.433367  8539 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:29.464216  5142 rpcz_store.cc:276] 0709 12:24:28.163201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.179743 (+ 16542us) service_pool.cc:224] Handling call
0709 12:24:28.180278 (+   535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:28.306870 (+126592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.307146 (+   276us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:28.308754 (+  1608us) log.cc:844] Serialized 26 byte log entry
0709 12:24:28.617935 (+309181us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:28.617979 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.618082 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.462517 (+844435us) raft_consensus.cc:1685] finished
0709 12:24:29.462672 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.463427 (+   755us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":163393,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":18660,"thread_start_us":429402,"threads_started":2,"wal-append.queue_time_us":427685}
I20260709 12:24:29.403584  8567 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:29.549244  5071 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 1206) took 1873 ms (client timeout 1523 ms). Trace:
W20260709 12:24:29.548022  8576 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 12:24:29.623451  5071 rpcz_store.cc:269] 0709 12:24:27.672206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.677935 (+  5729us) service_pool.cc:224] Handling call
0709 12:24:29.545049 (+1867114us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b540058af54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:29.545211 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1248806272}
I20260709 12:24:29.624208  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 810) took 1331 ms. Trace:
I20260709 12:24:29.552713  8539 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:29.469471  8557 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:29.576972  8593 raft_consensus.cc:2764] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:24:29.578068  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1195) took 2413 ms. Trace:
I20260709 12:24:29.578361  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1175) took 2396 ms. Trace:
I20260709 12:24:29.633338  5069 rpcz_store.cc:276] 0709 12:24:27.164960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.166382 (+  1422us) service_pool.cc:224] Handling call
0709 12:24:27.166729 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.390416 (+223687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.577670 (+2187254us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54006effd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:29.577959 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481875200}
W20260709 12:24:29.590725  8523 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.144s	user 0.016s	sys 0.000s
W20260709 12:24:29.602031  8573 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.391s	user 0.002s	sys 0.000s
I20260709 12:24:29.334280  8588 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:29.624399  8261 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:29.539755  8425 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:29.618290  8525 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:29.633414  5148 rpcz_store.cc:276] 0709 12:24:27.181814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.182004 (+   190us) service_pool.cc:224] Handling call
0709 12:24:27.582639 (+400635us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:27.697306 (+114667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.578126 (+1880820us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54007aadd0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:29.578264 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":704955648}
I20260709 12:24:29.596496  8017 raft_consensus.cc:3055] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:29.624847  5942 rpcz_store.cc:276] 0709 12:24:28.292660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.292821 (+   161us) service_pool.cc:224] Handling call
0709 12:24:28.293280 (+   459us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:28.597910 (+304630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.597966 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:28.622320 (+ 24354us) log.cc:844] Serialized 52 byte log entry
0709 12:24:28.810788 (+188468us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:28.899799 (+ 89011us) log.cc:844] Serialized 14 byte log entry
0709 12:24:29.490836 (+591037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.490963 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.623927 (+132964us) raft_consensus.cc:1685] finished
0709 12:24:29.623951 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.624138 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14267,"prepare.queue_time_us":94306,"prepare.run_wall_time_us":46,"raft.queue_time_us":208998,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":15877,"spinlock_wait_cycles":14291328,"thread_start_us":1077364,"threads_started":4,"wal-append.queue_time_us":295565}
I20260709 12:24:29.638489  8586 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:29.670958  8017 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:29.675045  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1020) took 1811 ms. Trace:
I20260709 12:24:29.678791  5221 rpcz_store.cc:276] 0709 12:24:27.863764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.864436 (+   672us) service_pool.cc:224] Handling call
0709 12:24:27.864914 (+   478us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:28.526703 (+661789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.526761 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:28.532070 (+  5309us) log.cc:844] Serialized 52 byte log entry
0709 12:24:28.773338 (+241268us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:28.773490 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.773826 (+   336us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.674710 (+900884us) raft_consensus.cc:1685] finished
0709 12:24:29.674738 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.674965 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":221465,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":542612,"threads_started":2,"wal-append.queue_time_us":329955}
I20260709 12:24:29.681623  8391 raft_consensus.cc:3055] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:29.681686  8557 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:29.681375  8017 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:29.682205  8391 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:29.658792  8521 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.682842  8521 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:29.683211  8391 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:29.684407  8391 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:29.685679  5076 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
W20260709 12:24:29.687040  5076 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 698) took 3162 ms (client timeout 1855 ms). Trace:
W20260709 12:24:29.687450  5076 rpcz_store.cc:269] 0709 12:24:26.524367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.524525 (+   158us) service_pool.cc:224] Handling call
0709 12:24:29.686797 (+3162272us) spinlock_profiling.cc:243] Waited 3.16 s on lock 0x7b5400429354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:29.686934 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7266826112}
I20260709 12:24:29.691926  8480 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.762740  8480 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:29.763140  8586 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:29.765462  8557 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:29.766122  8557 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:29.767799  8414 raft_consensus.cc:3055] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:29.768167  8557 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:29.768247  8414 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:29.429771  5899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.024s	sys 0.041s
W20260709 12:24:29.787042  5899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.025s	sys 0.041s
W20260709 12:24:29.787559  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 868490us
I20260709 12:24:29.789819  5942 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:29.769928  8521 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:29.769425  8554 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:29.813942  8554 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:29.771215  8327 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:29.820509  8411 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:29.769789  8414 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:29.827157  8414 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:29.776139  8480 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:29.635356  8567 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:29.841014  5145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 627) took 3003 ms (client timeout 1582 ms). Trace:
W20260709 12:24:29.841749  5145 rpcz_store.cc:269] 0709 12:24:26.835861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.840763 (+  4902us) service_pool.cc:224] Handling call
0709 12:24:29.838564 (+2997801us) spinlock_profiling.cc:243] Waited 76.2 ms on lock 0x7b54001b6554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:29.839275 (+   711us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175158016}
I20260709 12:24:29.857038  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1218) took 1261 ms. Trace:
I20260709 12:24:29.857640  5219 rpcz_store.cc:276] 0709 12:24:28.595453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.595653 (+   200us) service_pool.cc:224] Handling call
0709 12:24:28.627973 (+ 32320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.671828 (+ 43855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.856747 (+1184919us) spinlock_profiling.cc:243] Waited 43 ms on lock 0x7b54004835d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:29.856961 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98791168}
W20260709 12:24:29.874246  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.887003  8588 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:29.902307  8532 raft_consensus.cc:3055] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:29.903097  8532 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:29.903766  5042 connection.cc:256] RPC call timeout handler was delayed by -1.2173572500000773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
W20260709 12:24:29.905184  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 62359.5us
I20260709 12:24:29.946798  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:29.916488  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:29.914701  5943 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:29.904624  8035 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:29.947867  8579 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.003s	user 0.057s	sys 0.072s
W20260709 12:24:29.956223  8529 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.063s	user 0.000s	sys 0.014s
I20260709 12:24:29.924629  8586 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:29.907578  8559 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.968977  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:29.949132  8391 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:29.907927  8583 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:29.969980  5223 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.985s	user 0.009s	sys 0.006s
I20260709 12:24:29.974918  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:29.948293  7788 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.922008  8373 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:29.984045  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:29.904238  8532 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:29.968000  8579 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.057s	sys 0.072s
W20260709 12:24:29.993425  5223 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.008s	user 0.009s	sys 0.006s
I20260709 12:24:29.960448  8535 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:30.012406  5042 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:30.014588  5947 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:30.016096  5042 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:30.016620  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:29.987010  8529 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.000s	sys 0.014s
W20260709 12:24:30.020640  5144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.990s	user 0.000s	sys 0.016s
W20260709 12:24:30.021317  5144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.000s	sys 0.016s
I20260709 12:24:30.022437  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 641) took 1080 ms. Trace:
I20260709 12:24:30.029784  5941 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:30.031445  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:29.994547  8580 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.049s	user 0.025s	sys 0.064s
I20260709 12:24:30.069941  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:29.968900  8480 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
W20260709 12:24:30.013478  8548 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.012s	user 0.008s	sys 0.008s
I20260709 12:24:29.968823  8125 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:24:29.968735  8035 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:29.994062  8456 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
W20260709 12:24:30.069294  8580 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.025s	sys 0.065s
I20260709 12:24:30.029102  8535 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:30.030300  8532 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:30.078364  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.094116  5941 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:24:30.023224  5144 rpcz_store.cc:276] 0709 12:24:28.946629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.979801 (+ 33172us) service_pool.cc:224] Handling call
0709 12:24:29.018165 (+ 38364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:29.018436 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:29.030180 (+ 11744us) log.cc:844] Serialized 14 byte log entry
0709 12:24:30.021940 (+991760us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:30.022021 (+    81us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:30.022094 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.022182 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.022362 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11493,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"thread_start_us":991271,"threads_started":1}
W20260709 12:24:30.066613  8578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.006s	sys 0.048s
I20260709 12:24:29.997385  7788 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:30.089521  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:30.089879  5942 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:30.097673  5940 raft_consensus.cc:2368] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7.
I20260709 12:24:30.010528  8411 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:24:30.094833  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.098634  5947 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:30.009737  8586 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:30.012334  8391 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:30.011149  8373 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:30.069124  8311 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:24:30.107750  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.019655  8559 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:30.087350  8548 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.008s	sys 0.008s
W20260709 12:24:30.096462  8578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.006s	sys 0.048s
I20260709 12:24:30.020422  8583 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:30.140064  5039 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.090s	user 0.002s	sys 0.011s
W20260709 12:24:30.141769  5039 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.280s	user 0.003s	sys 0.011s
I20260709 12:24:30.147554  8411 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:30.148347  8411 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:30.148535  8373 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:30.148806  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.149823  5941 raft_consensus.cc:2410] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:30.149757  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.150508  5947 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:30.150807  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.149889  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.151407  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.152199  5942 raft_consensus.cc:2410] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 13 because replica has last-logged OpId of term: 12 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 12:24:30.152650  5947 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:30.155104  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.156288  5940 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:30.157977  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.150648  7788 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:30.159101  7788 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:24:30.160439  5113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.209s	user 0.011s	sys 0.010s
W20260709 12:24:30.160882  5113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.219s	user 0.011s	sys 0.010s
I20260709 12:24:30.149134  8411 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:30.145002  8311 consensus_queue.cc:1243] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer ceec344dd00f40c18ef58414ce147016 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:30.163847  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.170425  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:30.171012  5040 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.130s	user 0.001s	sys 0.019s
W20260709 12:24:30.171473  5040 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.156s	user 0.002s	sys 0.019s
W20260709 12:24:30.164335  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:30.174178  5188 connection.cc:256] RPC call timeout handler was delayed by -2.1131660420001026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
I20260709 12:24:30.149209  8373 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:30.166399  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:30.140004  5944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.178s	user 0.107s	sys 0.139s
I20260709 12:24:30.168445  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:30.187431  5944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.226s	user 0.107s	sys 0.139s
W20260709 12:24:30.188037  5114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.169s	user 0.007s	sys 0.015s
W20260709 12:24:30.188509  5114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.169s	user 0.007s	sys 0.015s
I20260709 12:24:30.169595  8525 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.192369  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:30.183537  8311 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.197732  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.198693  5940 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
W20260709 12:24:30.199431  5041 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.181s	user 0.017s	sys 0.007s
W20260709 12:24:30.199800  5041 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.181s	user 0.017s	sys 0.007s
W20260709 12:24:30.200183  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 1.18186e+06us
I20260709 12:24:30.185853  8391 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:30.172924  8480 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:24:30.223294  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.224381  5940 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:30.182461  8535 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:30.197072  8525 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:30.223733  8213 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.058s	sys 0.072s
W20260709 12:24:30.218688  8551 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.027s	sys 0.010s
I20260709 12:24:30.202497  8311 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:30.225500  8373 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:30.220547  8560 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.039s	user 0.002s	sys 0.021s
W20260709 12:24:30.234594  5115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.228s	user 0.015s	sys 0.023s
W20260709 12:24:30.235925  5220 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.206s	user 0.011s	sys 0.016s
W20260709 12:24:30.250339  5115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.244s	user 0.015s	sys 0.023s
W20260709 12:24:30.250537  5220 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.222s	user 0.011s	sys 0.016s
W20260709 12:24:30.236474  8213 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.054s	user 0.058s	sys 0.072s
I20260709 12:24:30.239620  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:30.238871  8551 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.027s	sys 0.010s
W20260709 12:24:30.255036  5114 connection.cc:256] RPC call timeout handler was delayed by -1.0051962170000479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
I20260709 12:24:30.259693  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:30.243767  7924 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.009s	sys 0.009s
W20260709 12:24:30.240057  8553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.029s	sys 0.022s
W20260709 12:24:30.266908  5116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.463s	user 0.005s	sys 0.034s
W20260709 12:24:30.310966  5116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.508s	user 0.006s	sys 0.034s
W20260709 12:24:30.311611  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 55560.1us
W20260709 12:24:30.251024  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 1.24498e+06us
I20260709 12:24:30.312613  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:30.248979  8560 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.186s	user 0.002s	sys 0.022s
W20260709 12:24:30.245342  8558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.186s	user 0.012s	sys 0.015s
W20260709 12:24:30.259519  8534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.151s	user 0.005s	sys 0.014s
W20260709 12:24:30.286795  7924 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.009s	sys 0.009s
W20260709 12:24:30.310775  8553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.030s	sys 0.022s
W20260709 12:24:30.321830  5116 connection.cc:256] RPC call timeout handler was delayed by -1.4776015560000815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:24:30.334973  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1033) took 1367 ms. Trace:
I20260709 12:24:30.335561  5218 rpcz_store.cc:276] 0709 12:24:28.967197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.967346 (+   149us) service_pool.cc:224] Handling call
0709 12:24:28.967656 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.985247 (+ 17591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.985288 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:28.985343 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:28.985391 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.334645 (+1349254us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b5400852354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:30.334704 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.334887 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39822592}
I20260709 12:24:30.337581  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:30.338440  5942 raft_consensus.cc:2376] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12. Re-sending same reply.
I20260709 12:24:30.322549  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:30.330858  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:30.339560  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.340226  5940 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:30.259627  8428 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:30.340070  5150 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:30.347257  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:30.249821  8373 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:30.260102  8613 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:24:30.324273  8558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.265s	user 0.012s	sys 0.015s
I20260709 12:24:30.350948  8410 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:30.327939  8610 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:30.334559  8544 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:30.354112  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.328012  8583 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:30.381062  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.356174  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 816) took 1637 ms. Trace:
W20260709 12:24:30.358352  5222 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 1027) took 1905 ms (client timeout 1766 ms). Trace:
I20260709 12:24:30.362797  8525 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:30.363974  8595 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:24:30.384533  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.332088  8476 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.365774  8586 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:30.372644  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:30.382895  5222 rpcz_store.cc:269] 0709 12:24:28.452583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.454173 (+  1590us) service_pool.cc:224] Handling call
0709 12:24:30.358061 (+1903888us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54006f77d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:30.358217 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641890048}
I20260709 12:24:30.386050  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:30.373236  8464 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:30.374285  8098 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.333491  8510 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.401969  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:30.340245  8534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.352s	user 0.006s	sys 0.015s
I20260709 12:24:30.333144  8309 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.379447  5942 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 5.
I20260709 12:24:30.354233  8593 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:30.352881  8327 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:30.384528  5144 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:30.376657  8486 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.012s	sys 0.030s
I20260709 12:24:30.392527  5940 raft_consensus.cc:2368] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7.
I20260709 12:24:30.386766  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.389498  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.411293  5218 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:30.384208  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:30.394554  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.395538  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.413702  5224 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:30.382436  5948 rpcz_store.cc:276] 0709 12:24:28.718928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.719768 (+   840us) service_pool.cc:224] Handling call
0709 12:24:28.720281 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.870339 (+150058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.870375 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:28.870423 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:28.870454 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.355899 (+1485445us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54002d78d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:30.355952 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.356106 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344513792}
I20260709 12:24:30.378288  8343 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:24:30.405581  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.378762  8610 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:30.410295  8486 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.013s	sys 0.031s
I20260709 12:24:30.417492  8610 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:30.413142  8613 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:30.419183  8613 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:30.397043  8594 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.426043  8594 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:30.383715  8544 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:30.428934  8610 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:30.430588  8544 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:30.368449  8428 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:30.432052  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 714) took 1527 ms. Trace:
I20260709 12:24:30.376816  8410 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:30.432958  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 48896.2us
I20260709 12:24:30.382652  8598 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:30.435022  5224 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:30.439164  5944 rpcz_store.cc:276] 0709 12:24:28.904637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.905189 (+   552us) service_pool.cc:224] Handling call
0709 12:24:28.905657 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:28.913156 (+  7499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.913190 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:28.926716 (+ 13526us) log.cc:844] Serialized 26 byte log entry
0709 12:24:30.188880 (+1262164us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:30.188922 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.188996 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:30.431702 (+242706us) raft_consensus.cc:1685] finished
0709 12:24:30.431733 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.431960 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134489,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":12687,"thread_start_us":1226304,"threads_started":1,"wal-append.queue_time_us":1403315}
I20260709 12:24:30.438735  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.445218  5222 raft_consensus.cc:2410] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:24:30.382975  8600 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:30.451371  8428 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:30.462347  7788 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:30.452468  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1220) took 1856 ms. Trace:
I20260709 12:24:30.399417  8476 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:30.463440  5217 rpcz_store.cc:276] 0709 12:24:28.596270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.596417 (+   147us) service_pool.cc:224] Handling call
0709 12:24:28.627716 (+ 31299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.748699 (+120983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.452152 (+1703453us) spinlock_profiling.cc:243] Waited 57.8 ms on lock 0x7b5400318b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:30.452388 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132954880}
I20260709 12:24:30.440296  8410 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:30.469137  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.441097  8213 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:30.471590  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1032) took 1516 ms. Trace:
I20260709 12:24:30.437230  8605 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:30.401634  8098 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:30.475849  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.477062  5946 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:30.413780  8309 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:30.478348  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:30.456370  8600 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:30.480545  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 446881us
I20260709 12:24:30.472139  5223 rpcz_store.cc:276] 0709 12:24:28.954691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.955602 (+   911us) service_pool.cc:224] Handling call
0709 12:24:28.955973 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.956220 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:29.998694 (+1042474us) log.cc:844] Serialized 14 byte log entry
0709 12:24:30.471256 (+472562us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:30.471307 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:30.471354 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.471419 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.471531 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1269732,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":4221,"thread_start_us":1485509,"threads_started":2}
I20260709 12:24:30.481452  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:30.411165  8510 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:30.486676  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.491403  5942 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:30.472640  8213 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:30.497392  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.498082  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:30.502015  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.458365  8598 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:30.543183  8598 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:30.518208  5075 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:30.519348  8373 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:30.520242  8309 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:30.521620  8476 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:30.523597  8510 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:30.525152  5145 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:30.496671  8608 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:30.498662  8213 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:30.527910  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.496295  8311 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:30.490670  8327 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.481135  8605 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:30.573385  5939 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:30.540396  5947 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:30.575212  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.553951  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.483083  8600 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:30.525192  8567 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.637s	user 0.005s	sys 0.010s
I20260709 12:24:30.527057  5069 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:30.498334  8593 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.566684  5070 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:30.568980  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:30.619699  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.622964  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.635231  5076 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:30.636137  5143 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:26.263242  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:30.623092  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.635974  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:30.612859  5188 connection.cc:256] RPC call timeout handler was delayed by -2.0294660419999673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
I20260709 12:24:30.638477  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.639562  5142 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
W20260709 12:24:30.640532  5188 connection.cc:256] RPC call timeout handler was delayed by -1.9023019010001008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
W20260709 12:24:30.640992  5188 connection.cc:256] RPC call timeout handler was delayed by -1.839566042000115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
W20260709 12:24:30.641297  5188 connection.cc:256] RPC call timeout handler was delayed by -1.6653808050000407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260709 12:24:30.641624  5188 connection.cc:256] RPC call timeout handler was delayed by -1.6487660420002612s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
W20260709 12:24:30.641916  5188 connection.cc:256] RPC call timeout handler was delayed by -1.5653092009997636s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
W20260709 12:24:30.642194  5188 connection.cc:256] RPC call timeout handler was delayed by -1.475162599000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
W20260709 12:24:30.642465  5188 connection.cc:256] RPC call timeout handler was delayed by -1.425862767000126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22440000000000004s for a response
W20260709 12:24:30.642736  5188 connection.cc:256] RPC call timeout handler was delayed by -1.3919957590001104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
I20260709 12:24:30.644356  5223 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:30.646662  5223 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:30.650090  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:30.635453  5148 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:30.655321  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.655514  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.655319  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.655900  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.656113  5070 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:30.656703  5072 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:30.653512  5067 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:30.657407  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:30.661495  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.661491  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.665123  5070 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:30.666141  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.669910  5072 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:30.670751  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.671095  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:30.672037  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.672772  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.673396  5072 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:30.674544  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:30.675119  5939 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:30.667117  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.677913  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.639590  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:30.671419  5947 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 4.
I20260709 12:24:30.673414  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.620558  8608 raft_consensus.cc:697] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:30.687667  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:30.688424  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.689945  5072 raft_consensus.cc:2393] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:30.690025  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.660921  8593 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:30.660451  8610 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:30.689230  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 49768.1us
I20260709 12:24:30.619073  8327 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:30.618537  8567 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:30.661783  8599 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:30.728440  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.730028  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:30.731240  5217 raft_consensus.cc:2393] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 5: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:30.711416  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:30.712942  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.735445  5219 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:30.714725  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.716322  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:30.736865  5224 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:30.716792  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.736758  5223 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:30.740020  5218 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:24:30.718164  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.741071  5221 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:30.719013  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.719149  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:30.722963  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 349382us
I20260709 12:24:30.743010  5215 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
W20260709 12:24:30.747038  5898 leader_election.cc:336] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.585s (SENT)
I20260709 12:24:30.698977  5074 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
I20260709 12:24:30.742036  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:30.751392  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.692691  8510 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:30.753955  5188 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:30.742264  5216 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 6.
I20260709 12:24:30.752170  5945 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:30.759279  5188 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:30.760584  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:30.761579  5069 raft_consensus.cc:2376] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5. Re-sending same reply.
I20260709 12:24:30.764639  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.765302  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:30.768890  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.756968  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:30.781622  5217 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:30.778705  5074 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:30.755692  8510 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:30.783617  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.786190  5069 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:30.788226  5072 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:30.786641  5224 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:30.764338  8525 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:24:30.790729  8620 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:24:30.792739  8608 consensus_queue.cc:237] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:30.793645  8414 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:30.794782  8593 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:30.745519  8599 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:30.797773  8618 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:24:30.798563  8599 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:30.799180  8163 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:30.800911  8567 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:30.804459  8510 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:30.810075  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.811074  5069 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 4.
I20260709 12:24:30.812162  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.813042  5069 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:30.687893  8479 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:30.814395  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.816099  5072 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
W20260709 12:24:30.818444  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 53356us
I20260709 12:24:30.786265  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:30.819454  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:30.817106  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.797142  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1174) took 3647 ms. Trace:
I20260709 12:24:30.822557  5149 rpcz_store.cc:276] 0709 12:24:27.149682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.150501 (+   819us) service_pool.cc:224] Handling call
0709 12:24:29.838736 (+2688235us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:29.906705 (+ 67969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:29.906755 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:29.907447 (+   692us) log.cc:844] Serialized 26 byte log entry
0709 12:24:30.392531 (+485084us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:30.654145 (+261614us) log.cc:844] Serialized 14 byte log entry
0709 12:24:30.656732 (+  2587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.656816 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:30.795298 (+138482us) raft_consensus.cc:1685] finished
0709 12:24:30.795330 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.796881 (+  1551us) spinlock_profiling.cc:243] Waited 2.69 s on lock 0x7b54001b6550. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:30.797045 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67347,"prepare.queue_time_us":141190,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":72487,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":13403,"spinlock_wait_cycles":6181211520,"thread_start_us":724486,"threads_started":2,"wal-append.queue_time_us":746674}
I20260709 12:24:30.753247  8532 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:24:30.840758  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.818008  8098 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:30.814451  8479 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:30.841810  5074 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:30.844552  5072 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:30.833189  8581 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:30.836771  8518 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:30.854750  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:30.864562  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 78370.5us
I20260709 12:24:30.865067  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.865823  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:30.870357  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.874348  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.875584  5941 raft_consensus.cc:2410] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:24:30.848565  8551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.018s	sys 0.023s
I20260709 12:24:30.871199  5188 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:30.884562  5188 leader_election.cc:400] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:30.882328  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.889189  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:30.889549  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.890012  5942 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:30.851244  8581 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:30.854629  8518 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.851828  8476 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:30.911813  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.912690  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.882753  8163 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:30.888124  8414 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:30.887667  8607 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
W20260709 12:24:30.883100  8551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.018s	sys 0.023s
I20260709 12:24:30.908984  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:30.919595  5898 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.704s (SENT)
I20260709 12:24:30.921015  8327 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:30.916235  8518 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:30.912009  8581 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:30.952509  5900 connection.cc:256] RPC call timeout handler was delayed by -1.1843594310003027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 12:24:30.953649  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 44797.6us
I20260709 12:24:30.954622  5069 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:30.895082  8525 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:30.963512  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:30.980687  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:30.981383  5074 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:30.987738  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.911703  8035 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:30.990479  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.990311  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.993703  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.994551  5143 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:30.904526  8532 consensus_queue.cc:1059] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
W20260709 12:24:30.984318  8632 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:24:31.012485  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.005280  5149 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:31.019017  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.006186  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.024549  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.010624  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.037595  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.044025  5942 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:24:31.042564  5142 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:24:31.048834  5076 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:31.049990  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.980047  8525 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.956064  8567 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:31.052807  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.929792  8414 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.060415  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.077770  5143 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:31.078083  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.078739  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:30.927731  8605 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:31.066478  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.068064  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.064512  5148 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:31.054111  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:31.066965  5900 connection.cc:256] RPC call timeout handler was delayed by -1.1238359650001257s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
I20260709 12:24:31.074133  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.085065  5076 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:30.962244  8557 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.918113  8476 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:31.091518  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.092444  5074 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:31.084933  5142 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:31.076148  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:31.079396  5143 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:31.086083  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.082934  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.063133  8525 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:31.074193  8532 consensus_queue.cc:1243] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:31.096993  5144 raft_consensus.cc:2393] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:31.088141  5900 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:31.101709  5076 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:24:31.082867  8622 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 12:24:31.081468  8414 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:31.080793  8559 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:31.109519  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.111055  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:31.115361  5074 raft_consensus.cc:2376] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9. Re-sending same reply.
I20260709 12:24:31.104717  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:31.110201  8410 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.005s
I20260709 12:24:31.108850  8605 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:24:31.127034  8591 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.019s	sys 0.014s
I20260709 12:24:31.099098  8557 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:31.134473  8559 raft_consensus.cc:3055] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:24:31.160650  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.127542  8532 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:31.150074  8591 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.020s	sys 0.014s
W20260709 12:24:31.152820  8213 consensus_meta.cc:287] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.002s
I20260709 12:24:31.145596  8605 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.162587  8559 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:31.175442  8629 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:24:31.176412  8372 raft_consensus.cc:3055] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:24:31.176714  8636 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:31.178930  5074 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:24:31.183504  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 290620us
I20260709 12:24:31.141553  8410 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:31.179908  8605 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:31.192341  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:31.219686  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.182951  8372 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:31.229561  5074 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:31.173194  8532 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:31.232192  5147 raft_consensus.cc:2410] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:31.233924  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1031) took 2280 ms. Trace:
I20260709 12:24:31.179746  8213 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:31.220547  8470 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:31.240578  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:31.246858  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.254488  8649 consensus_queue.cc:1059] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:31.262105  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.233049  8417 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:31.281035  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.234483  5220 rpcz_store.cc:276] 0709 12:24:28.953818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.955063 (+  1245us) service_pool.cc:224] Handling call
0709 12:24:28.986016 (+ 30953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.986310 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:30.252794 (+1266484us) log.cc:844] Serialized 14 byte log entry
0709 12:24:30.546220 (+293426us) spinlock_profiling.cc:243] Waited 7.65 ms on lock 0x7b18001dd220. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db23a 00007fd8a8545a2a
0709 12:24:31.233270 (+687050us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.233471 (+   201us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:31.233527 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:31.233635 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.233724 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.233861 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1249903,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"raft.queue_time_us":189197,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":2200,"spinlock_wait_cycles":17595136,"thread_start_us":1906287,"threads_started":3,"wal-append.queue_time_us":884663}
I20260709 12:24:31.323177  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.324002  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.294903  8410 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:31.296561  8557 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:31.298640  8213 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:31.302317  8637 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
W20260709 12:24:31.303350  8628 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
W20260709 12:24:31.304250  8641 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 12:24:31.305101  8638 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:24:31.305770  8476 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:31.306274  8627 raft_consensus.cc:3055] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:31.308910  8510 consensus_meta.cc:287] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.006s
I20260709 12:24:31.278889  8372 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:31.233279  8559 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:31.354938  8017 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:31.311074  8532 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:31.262454  8470 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:31.279791  8417 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:31.316991  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.324416  5149 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:31.357314  8627 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:31.390098  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.391009  5074 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:31.390484  8470 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:31.390604  8627 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:31.374944  8532 raft_consensus.cc:3055] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:31.376086  8479 consensus_meta.cc:287] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.528s	user 0.014s	sys 0.000s
I20260709 12:24:31.357100  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.386590  8557 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:31.389076  8417 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:31.420706  5070 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
W20260709 12:24:31.381548  8600 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.005s
I20260709 12:24:31.366822  8510 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:31.477733  5190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.016s	sys 0.015s
W20260709 12:24:31.590137  5190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.016s	sys 0.025s
W20260709 12:24:31.590631  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.559440  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.591536  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.597026  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:31.603611  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.604329  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.563920  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 824) took 1097 ms. Trace:
W20260709 12:24:31.627364  5943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.010s	sys 0.027s
W20260709 12:24:31.628124  5943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.010s	sys 0.027s
I20260709 12:24:31.398401  8532 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:31.605006  5948 rpcz_store.cc:276] 0709 12:24:30.466704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.467064 (+   360us) service_pool.cc:224] Handling call
0709 12:24:30.467467 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:30.467740 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:30.467767 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:30.476382 (+  8615us) log.cc:844] Serialized 26 byte log entry
0709 12:24:30.899348 (+422966us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:30.899406 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.899489 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:31.563553 (+664064us) raft_consensus.cc:1685] finished
0709 12:24:31.563586 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.563831 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":396911,"threads_started":1,"wal-append.queue_time_us":639955}
I20260709 12:24:31.639196  5141 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:31.642516  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:31.643433  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 727) took 1156 ms. Trace:
W20260709 12:24:31.643610  5900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.023s	sys 0.025s
W20260709 12:24:31.645156  5900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.024s	sys 0.025s
W20260709 12:24:31.646066  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 545218us
W20260709 12:24:31.495046  8645 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:24:31.647639  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:31.576818  8646 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:24:31.643993  5946 rpcz_store.cc:276] 0709 12:24:30.487134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.487290 (+   156us) service_pool.cc:224] Handling call
0709 12:24:30.487599 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.487860 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:30.972179 (+484319us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.633062 (+660883us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.641953 (+  8891us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:31.642995 (+  1042us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:31.643131 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.643212 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.643363 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8635,"prepare.queue_time_us":324379,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":1131550,"threads_started":3,"wal-append.queue_time_us":504871}
W20260709 12:24:31.650583  5150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1052) took 6169 ms (client timeout 2212 ms). Trace:
I20260709 12:24:31.593108  8372 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:31.659194  5150 rpcz_store.cc:269] 0709 12:24:25.469639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.968178 (+498539us) service_pool.cc:224] Handling call
0709 12:24:31.638565 (+5670387us) spinlock_profiling.cc:243] Waited 4.14 s on lock 0x7b5400238a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:31.638794 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9513898752}
I20260709 12:24:31.497915  8647 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.426501  8627 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:31.592615  8559 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:31.661707  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.666599  5068 raft_consensus.cc:2410] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:31.667943  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.668783  5068 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:31.672454  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.419713  8518 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:31.664052  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.678179  5075 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:31.679111  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:31.684501  5041 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:31.573617  8605 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:31.686014  8608 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:31.596000  8600 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:31.687240  8479 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.735271  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 615) took 1015 ms. Trace:
W20260709 12:24:31.680943  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 452538us
I20260709 12:24:31.652992  5898 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:31.711201  8608 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:31.739158  5075 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:31.742266  8650 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:24:31.742985  8654 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:31.743539  8654 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:31.744413  8654 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:31.746987  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 716) took 1084 ms. Trace:
I20260709 12:24:31.747429  5070 rpcz_store.cc:276] 0709 12:24:30.662487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.668511 (+  6024us) service_pool.cc:224] Handling call
0709 12:24:31.746790 (+1078279us) spinlock_profiling.cc:243] Waited 742 ms on lock 0x7b540047ca54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:31.746919 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1706447232}
I20260709 12:24:31.748428  5072 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:31.750830  8518 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:31.737032  5219 rpcz_store.cc:276] 0709 12:24:30.719696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.744438 (+ 24742us) service_pool.cc:224] Handling call
0709 12:24:30.779861 (+ 35423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.780122 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:30.820833 (+ 40711us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.734821 (+913988us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:31.734889 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:31.734948 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.735028 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.735189 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":53085,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":1858,"raft.queue_time_us":127412,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":913451,"threads_started":2,"wal-append.queue_time_us":775089}
I20260709 12:24:31.753726  8600 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:31.754617  8525 consensus_meta.cc:287] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.649s	user 0.013s	sys 0.000s
I20260709 12:24:31.755089  8525 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:31.757617  8479 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:31.738847  8651 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:24:31.760653  8479 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:31.766559  8510 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:31.680444  8647 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:31.768569  8605 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:31.770260  8588 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:31.770448  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1181) took 1398 ms. Trace:
I20260709 12:24:31.691270  8532 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:31.764504  5114 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:31.772589  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 119702us
W20260709 12:24:31.775449  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.788393  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:31.786846  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:31.787879  5147 rpcz_store.cc:276] 0709 12:24:30.372048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.372225 (+   177us) service_pool.cc:224] Handling call
0709 12:24:31.770202 (+1397977us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b540022ea54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:31.770356 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1931098496}
I20260709 12:24:31.790333  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.780622  8525 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:31.793113  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:31.794421  5899 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:31.795495  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:31.779120  8644 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:24:31.784409  8594 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:31.805359  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.806020  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.794611  5147 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:24:31.802070  5941 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.043s	sys 0.017s
W20260709 12:24:31.788239  8581 consensus_meta.cc:287] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.782s	user 0.009s	sys 0.005s
I20260709 12:24:31.760210  8608 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:31.803273  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:31.873739  5939 raft_consensus.cc:2368] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11.
I20260709 12:24:31.734418  8655 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.827934  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:31.834406  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:31.835673  5941 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.044s	sys 0.017s
I20260709 12:24:31.836494  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:31.837018  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1180) took 1465 ms. Trace:
I20260709 12:24:31.804031  8518 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:31.800284  8588 raft_consensus.cc:697] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:31.880730  5145 rpcz_store.cc:276] 0709 12:24:30.371556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.371712 (+   156us) service_pool.cc:224] Handling call
0709 12:24:30.381111 (+  9399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.563785 (+182674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.835072 (+1271287us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54004fae54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:31.836943 (+  1871us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330715008}
I20260709 12:24:31.874627  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:31.881192  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44775us
W20260709 12:24:31.886602  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 58742us
I20260709 12:24:31.804440  8525 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:31.887991  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 1224) took 1313 ms. Trace:
I20260709 12:24:31.895022  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:31.901678  5073 rpcz_store.cc:276] 0709 12:24:30.574784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.574911 (+   127us) service_pool.cc:224] Handling call
0709 12:24:31.887772 (+1312861us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54000307d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:31.887924 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286430080}
I20260709 12:24:31.871387  8532 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:31.883011  8017 consensus_meta.cc:287] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.527s	user 0.013s	sys 0.001s
I20260709 12:24:31.909682  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 735) took 1044 ms. Trace:
I20260709 12:24:31.920747  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:31.831732  8484 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:24:31.917987  5941 rpcz_store.cc:276] 0709 12:24:30.864952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.892323 (+ 27371us) service_pool.cc:224] Handling call
0709 12:24:30.892651 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.892897 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:30.914978 (+ 22081us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.890467 (+975489us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.909137 (+ 18670us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:31.909220 (+    83us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:31.909288 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.909387 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.909597 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18073,"prepare.queue_time_us":27602,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":114,"raft.queue_time_us":394383,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":3753,"spinlock_wait_cycles":536320,"thread_start_us":964484,"threads_started":2,"wal-append.queue_time_us":661496}
I20260709 12:24:31.924403  5944 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:31.836333  8510 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:24:31.882735  8518 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:31.925282  5115 leader_election.cc:400] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11.
I20260709 12:24:31.942346  5115 leader_election.cc:403] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:24:31.888090  8309 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:31.874612  8581 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:31.957414  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 309992us
I20260709 12:24:31.962853  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:31.929325  8594 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.968034  5144 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:31.943746  8496 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:31.939350  8591 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:31.925065  8660 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:31.999897  8660 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:31.946383  8518 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:31.916126  8605 consensus_queue.cc:1059] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:24:31.957415  8526 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:31.926420  8525 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:31.990451  8414 consensus_meta.cc:287] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.866s	user 0.014s	sys 0.000s
W20260709 12:24:31.991310  8652 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 12:24:31.977396  8594 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:32.004210  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:32.015019  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.020859  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:32.021816  5219 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:32.006124  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:31.888908  8655 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:31.923810  8657 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:24:31.964139  8607 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.984757  8591 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.029886  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:32.024014  8653 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:24:32.037912  8607 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:32.047118  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:32.042403  8591 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:32.048720  8098 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:32.055318  5222 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
W20260709 12:24:32.055929  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 320252us
I20260709 12:24:32.056742  5223 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:24:32.057416  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 51445.4us
I20260709 12:24:32.058452  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:32.064328  5220 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:32.066840  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:32.067526  5217 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:32.066653  8657 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:32.071026  8660 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:32.071866  8660 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:32.077179  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:32.084910  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.019804  8414 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:32.095425  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:32.098819  5216 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
W20260709 12:24:32.096659  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 505563us
I20260709 12:24:32.008409  8588 consensus_queue.cc:237] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:32.103497  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 729) took 1245 ms. Trace:
I20260709 12:24:32.075524  8400 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:32.075945  8098 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:32.105042  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:32.105374  5189 connection.cc:256] RPC call timeout handler was delayed by -1.116239503000088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30790000000000006s for a response
I20260709 12:24:32.109910  5071 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:32.107359  5943 rpcz_store.cc:276] 0709 12:24:30.858258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.858401 (+   143us) service_pool.cc:224] Handling call
0709 12:24:30.858767 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:30.859150 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:30.859175 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:30.952512 (+ 93337us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.628704 (+676192us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:31.628766 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.628877 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.103240 (+474363us) raft_consensus.cc:1685] finished
0709 12:24:32.103267 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.103419 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":131,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":19272,"thread_start_us":615647,"threads_started":1,"wal-append.queue_time_us":807657}
I20260709 12:24:32.113353  5220 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:32.080996  5219 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:32.090674  8556 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:32.111673  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:32.149490  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 355979us
W20260709 12:24:32.116261  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 448984us
I20260709 12:24:32.148248  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:32.113703  5189 connection.cc:256] RPC call timeout handler was delayed by -1.0088003250002657s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
W20260709 12:24:32.165810  5042 connection.cc:256] RPC call timeout handler was delayed by -1.6355473829999028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 12:24:32.168231  5042 connection.cc:256] RPC call timeout handler was delayed by -1.2701473829997667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
W20260709 12:24:32.169920  5042 connection.cc:256] RPC call timeout handler was delayed by -1.2134473830001298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
W20260709 12:24:32.172395  5042 connection.cc:256] RPC call timeout handler was delayed by -1.1099473829999624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
W20260709 12:24:32.175606  5042 connection.cc:256] RPC call timeout handler was delayed by -1.0865473829999246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
W20260709 12:24:32.178474  5042 connection.cc:256] RPC call timeout handler was delayed by -1.0622473829998853s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
I20260709 12:24:32.182386  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:32.192067  5217 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:24:32.193711  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:32.164587  5898 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:32.196123  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:32.197903  5898 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:32.186237  5221 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:32.187124  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:32.184396  5218 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:32.209262  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:32.115602  8098 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:32.221262  5220 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:32.209589  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:32.223356  5217 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:32.111704  8647 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:32.165109  8556 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.253592  8666 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:32.221235  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:32.134377  8658 raft_consensus.cc:3055] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:32.204854  8557 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:32.227891  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:32.232969  8594 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:32.234767  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1077) took 1737 ms. Trace:
I20260709 12:24:32.236622  5071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1216) took 1726 ms. Trace:
I20260709 12:24:32.237623  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 731) took 1420 ms. Trace:
I20260709 12:24:32.286373  5069 rpcz_store.cc:276] 0709 12:24:30.807590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.829429 (+ 21839us) service_pool.cc:224] Handling call
0709 12:24:30.829850 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.984688 (+154838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.227293 (+1242605us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b540037f354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:32.227478 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286240000}
I20260709 12:24:32.237740  8525 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:32.239018  8414 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:32.240216  8607 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:32.240653  8581 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:32.244045  8579 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:32.244283  8559 consensus_meta.cc:287] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.000s
I20260709 12:24:32.245250  8534 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:32.245959  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:32.299083  5220 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:32.240875  8098 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:32.246948  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:32.305595  5217 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:32.248540  5189 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:32.250366  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:32.251405  8647 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:32.149106  8480 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:24:32.165344  8213 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260709 12:24:32.218362  8372 consensus_meta.cc:287] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.002s
I20260709 12:24:32.253897  8556 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:32.214986  5042 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:32.260886  5218 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:32.263767  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:24:32.279024  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 393102us
W20260709 12:24:32.339367  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 124485us
I20260709 12:24:32.340432  5042 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:32.283706  5219 raft_consensus.cc:2393] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:32.285918  5071 rpcz_store.cc:276] 0709 12:24:30.510174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.512528 (+  2354us) service_pool.cc:224] Handling call
0709 12:24:30.512984 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:32.115478 (+1602494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.236317 (+120839us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b5400589b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:32.236510 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3669992320}
W20260709 12:24:32.240923  8627 consensus_meta.cc:287] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.560s	user 0.007s	sys 0.002s
W20260709 12:24:32.343842  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 80230.2us
I20260709 12:24:32.344914  5042 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:32.340577  5190 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:32.296048  8594 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
W20260709 12:24:32.306744  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 58272.2us
I20260709 12:24:32.280547  8658 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:32.218255  8662 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:24:32.313223  8414 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:32.286192  8557 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:32.319243  5944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 831) took 1749 ms (client timeout 1818 ms). Trace:
W20260709 12:24:32.375597  5944 rpcz_store.cc:269] 0709 12:24:30.570253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.570563 (+   310us) service_pool.cc:224] Handling call
0709 12:24:32.319005 (+1748442us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400004954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:32.319161 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2960481536}
I20260709 12:24:32.292126  8600 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.342792  8528 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:32.284902  5141 rpcz_store.cc:276] 0709 12:24:30.497182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.497313 (+   131us) service_pool.cc:224] Handling call
0709 12:24:31.638775 (+1141462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.639739 (+   964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.234476 (+594737us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400238a50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:32.234690 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2624204032}
I20260709 12:24:32.296507  8579 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:32.349620  5217 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:32.354511  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:32.305298  8534 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:32.356782  5189 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:32.350364  8526 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11.
I20260709 12:24:32.352298  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:32.358872  5224 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:32.360395  8479 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:32.402074  8608 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.518s	user 0.012s	sys 0.000s
W20260709 12:24:32.388031  5899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.028s	sys 0.038s
W20260709 12:24:32.402951  5899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.029s	sys 0.038s
I20260709 12:24:32.395967  8534 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:32.402917  8608 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:32.403661  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 609262us
I20260709 12:24:32.404929  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:32.337983  8532 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:32.405848  5068 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:32.405869  5899 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:32.406026  8532 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:32.407301  5899 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:32.408516  5899 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:32.408749  8532 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:32.358783  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 391414us
I20260709 12:24:32.362313  5221 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:32.357738  8372 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:32.393031  5189 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:32.395140  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:32.374898  8414 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:32.402199  8669 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:32.351217  8666 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:32.387904  8528 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:32.360042  8559 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:32.317337  8591 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:32.384266  8557 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:32.437793  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 44891.5us
I20260709 12:24:32.439041  5189 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:32.441284  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:32.412210  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1089) took 1073 ms. Trace:
I20260709 12:24:32.350970  8647 raft_consensus.cc:3055] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:32.409735  8594 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:32.428452  5218 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:32.392499  8579 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:32.392354  8098 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:32.363960  8512 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.002s	sys 0.031s
I20260709 12:24:32.433313  8559 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.413398  8372 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:32.433127  8528 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:32.436070  8591 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:32.436712  8608 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:32.442705  5148 rpcz_store.cc:276] 0709 12:24:31.323291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.326709 (+  3418us) service_pool.cc:224] Handling call
0709 12:24:31.646803 (+320094us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.686635 (+ 39832us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.686687 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.691386 (+  4699us) log.cc:844] Serialized 26 byte log entry
0709 12:24:32.049727 (+358341us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:32.389057 (+339330us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.396037 (+  6980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.396409 (+   372us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.396839 (+   430us) raft_consensus.cc:1685] finished
0709 12:24:32.396862 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.397051 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38257,"raft.queue_time_us":53679,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":4010,"thread_start_us":619140,"threads_started":2,"wal-append.queue_time_us":508593}
I20260709 12:24:32.420272  8479 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:32.418386  8654 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:24:32.464016  8512 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.002s	sys 0.032s
W20260709 12:24:32.433298  8410 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.025s	sys 0.067s
I20260709 12:24:32.392521  8600 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:32.420142  8666 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:32.444721  8647 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:32.429558  8673 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:32.498529  8479 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.506482  8627 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:32.502295  8654 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:32.461217  8518 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:32.477929  8098 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:32.467931  8372 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:32.463587  8559 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:32.564538  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:32.564882  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:32.570315  5075 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:32.517688  8666 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:32.509449  8410 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.025s	sys 0.067s
I20260709 12:24:32.448657  8532 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:32.450778  8655 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:32.464897  8480 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:24:32.418179  8414 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:32.508836  8677 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:32.533941  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1088) took 1211 ms. Trace:
I20260709 12:24:32.563375  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:32.564357  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:32.564230  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:32.516388  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 99366.8us
I20260709 12:24:32.429795  8662 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:32.519659  8673 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:32.531826  8479 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:32.503037  8669 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:24:32.568908  8608 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.561558  8654 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:32.571205  8098 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:32.607189  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:32.612538  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:32.608419  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 198414us
W20260709 12:24:32.597922  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 218308us
I20260709 12:24:32.599493  5071 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:32.599802  5069 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:32.609123  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:32.608512  5146 rpcz_store.cc:276] 0709 12:24:31.322931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.325501 (+  2570us) service_pool.cc:224] Handling call
0709 12:24:31.326068 (+   567us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:31.589297 (+263229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.589333 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:31.631019 (+ 41686us) log.cc:844] Serialized 52 byte log entry
0709 12:24:32.159328 (+528309us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:32.524307 (+364979us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.533020 (+  8713us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.533384 (+   364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.533499 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.533581 (+    82us) raft_consensus.cc:1685] finished
0709 12:24:32.533600 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.533853 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44994,"prepare.queue_time_us":373402,"raft.queue_time_us":272079,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":78331,"spinlock_wait_cycles":56206208,"thread_start_us":821263,"threads_started":2,"wal-append.queue_time_us":655856}
I20260709 12:24:32.617827  5067 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:24:32.510079  8658 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:32.571704  8591 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:24:32.624379  5076 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:32.627216  5067 pending_rounds.cc:85] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 5
I20260709 12:24:32.627658  5067 pending_rounds.cc:107] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:24:32.627983  5067 raft_consensus.cc:2889] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:32.628144  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:32.629285  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:32.598984  8655 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:32.646670  5076 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
W20260709 12:24:32.646950  5898 leader_election.cc:336] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.583s (SENT)
I20260709 12:24:32.626366  5114 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:32.642378  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:32.647511  5898 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:32.610436  8673 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:32.662032  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:32.655884  5069 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:32.643707  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:32.596071  8647 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:32.707649  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45767.1us
I20260709 12:24:32.628571  8674 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:32.628763  8098 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:32.623202  8675 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:24:32.665750  8526 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:32.725479  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:32.622843  8608 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:32.667878  8677 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:32.706616  8658 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:32.708470  8607 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:32.759322  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 407074us
I20260709 12:24:32.752300  5114 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:32.760609  5190 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:32.761972  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:32.766451  5114 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:32.746718  8669 raft_consensus.cc:3055] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:32.758101  8306 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:32.781955  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:32.784752  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:32.785435  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:32.786029  5073 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:32.790292  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:32.790725  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:32.790210  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:32.782346  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:32.763639  8675 consensus_queue.cc:1243] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:32.763015  8647 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:32.797627  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:32.802085  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:32.798506  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:32.803272  5149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1092) took 1472 ms (client timeout 1594 ms). Trace:
W20260709 12:24:32.803949  8689 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:24:32.803776  5149 rpcz_store.cc:269] 0709 12:24:31.330421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.379036 (+ 48615us) service_pool.cc:224] Handling call
0709 12:24:32.798323 (+1419287us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:24:32.803194 (+  4871us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42025344}
I20260709 12:24:32.799551  5073 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:24:32.805739  8688 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
W20260709 12:24:32.807364  8698 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:32.807484  8532 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:32.808368  8532 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:32.808888  8690 log.cc:927] Time spent T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:24:32.812412  8532 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.813199  8532 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:32.780541  8662 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:32.774680  8410 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:32.779032  8600 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:32.816418  8532 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:32.779925  8669 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:32.790800  8526 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:32.829221  5075 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:32.822950  5071 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:32.779618  8484 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
W20260709 12:24:32.824689  8683 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
W20260709 12:24:32.825968  8685 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260709 12:24:32.826135  8687 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.111s	user 0.003s	sys 0.004s
I20260709 12:24:32.823755  8662 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:32.862627  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 421422us
I20260709 12:24:32.830561  8675 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:32.864382  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 451660us
I20260709 12:24:32.836243  8682 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.836506  8704 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:32.855751  8518 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:32.873202  5900 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:24:32.972894  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 206329us
I20260709 12:24:33.116276  4899 catalog_manager.cc:5697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:32.975859  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1085) took 1653 ms. Trace:
I20260709 12:24:32.976198  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1090) took 1648 ms. Trace:
I20260709 12:24:33.121966  5143 rpcz_store.cc:276] 0709 12:24:31.321796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.331682 (+  9886us) service_pool.cc:224] Handling call
0709 12:24:32.256475 (+924793us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:24:32.256707 (+   232us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.257170 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.257203 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:32.257743 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:24:32.519974 (+262231us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:32.520020 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.520101 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.974707 (+454606us) raft_consensus.cc:1685] finished
0709 12:24:32.974795 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.975499 (+   704us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22444,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7859,"spinlock_wait_cycles":3951872,"thread_start_us":261755,"threads_started":1,"wal-append.queue_time_us":587780}
I20260709 12:24:32.867105  5189 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:33.002298  5944 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:24:33.005159  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 984) took 1820 ms. Trace:
I20260709 12:24:33.007243  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:33.008368  8259 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:33.011407  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:32.925737  8655 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:24:33.005833  5898 leader_election.cc:336] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.742s (SENT)
I20260709 12:24:33.020429  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 991) took 1720 ms. Trace:
W20260709 12:24:33.024554  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.033854  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:33.034673  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 1044) took 1005 ms. Trace:
I20260709 12:24:32.827018  8534 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:33.132460  5075 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:33.046039  8525 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:24:33.047724  8556 consensus_meta.cc:287] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.696s	user 0.015s	sys 0.000s
I20260709 12:24:33.047871  8531 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:33.133723  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:33.134133  8531 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:33.136019  8531 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:33.050041  8588 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:33.137741  8588 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:33.138536  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 133087us
I20260709 12:24:33.139446  8588 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:32.868798  8608 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:32.863899  8662 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:33.141974  8259 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:33.143025  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:33.131235  5224 rpcz_store.cc:276] 0709 12:24:32.029322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.029489 (+   167us) service_pool.cc:224] Handling call
0709 12:24:33.034071 (+1004582us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b540023b9d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:33.034361 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730404736}
I20260709 12:24:33.144529  5071 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:32.880474  8669 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:32.912551  8667 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.561s	user 0.005s	sys 0.000s
I20260709 12:24:33.149859  8556 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:32.875398  8675 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:33.151721  5072 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:33.152872  8556 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:33.133741  8525 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:33.154158  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:32.911391  8672 log.cc:927] Time spent T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.435s	user 0.002s	sys 0.000s
I20260709 12:24:33.154908  8525 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:33.155885  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:33.157042  8525 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:32.881673  8682 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:33.130636  5068 rpcz_store.cc:276] 0709 12:24:31.300002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.690467 (+390465us) service_pool.cc:224] Handling call
0709 12:24:31.735801 (+ 45334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:32.407784 (+671983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.019757 (+611973us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b5400286ed4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:33.020202 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1540502144}
I20260709 12:24:33.159000  5069 raft_consensus.cc:2410] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:33.159536  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:32.893668  8700 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:24:32.877856  8704 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:33.171439  8704 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:33.171821  8674 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.172720  8674 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:33.173336  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:33.174631  5224 raft_consensus.cc:2410] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:24:33.175063  8660 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:33.176282  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:33.119982  4899 catalog_manager.cc:5697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:24:32.916579  8673 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:33.220124  8673 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:33.077394  5223 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:24:33.224413  8579 consensus_meta.cc:287] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.747s	user 0.011s	sys 0.007s
I20260709 12:24:33.090322  8535 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:24:33.096873  8697 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:33.226133  8697 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:33.227258  8654 consensus_meta.cc:287] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.583s	user 0.013s	sys 0.006s
I20260709 12:24:33.227953  8654 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:33.080963  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:33.012461  5041 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.009s	sys 0.045s
W20260709 12:24:33.230099  5041 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.130s	user 0.010s	sys 0.046s
I20260709 12:24:33.205938  8400 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 12:24:33.230572  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 1.13051e+06us
I20260709 12:24:33.231803  5041 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:33.233089  5075 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
W20260709 12:24:33.123360  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 250194us
I20260709 12:24:33.126289  8598 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:33.233495  8697 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:33.238927  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1233) took 1030 ms. Trace:
I20260709 12:24:33.239543  5219 rpcz_store.cc:276] 0709 12:24:32.208659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.352784 (+144125us) service_pool.cc:224] Handling call
0709 12:24:32.353236 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:32.353515 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:32.985889 (+632374us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b1800048fa0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a80e34bc 00007fd8a543148b 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8abf333c7 00007fd8a85459a7 00007fd8a85550bf
0709 12:24:32.986219 (+   330us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.164106 (+177887us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:33.170050 (+  5944us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:33.238398 (+ 68348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.238526 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.238717 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27071,"prepare.queue_time_us":206287,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":63,"raft.queue_time_us":36075,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":292,"spinlock_wait_cycles":39854848,"thread_start_us":333444,"threads_started":2,"wal-append.queue_time_us":194994}
I20260709 12:24:33.122097  5142 rpcz_store.cc:276] 0709 12:24:31.327392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.337021 (+  9629us) service_pool.cc:224] Handling call
0709 12:24:32.241517 (+904496us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:24:32.241805 (+   288us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:32.242441 (+   636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.242475 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:32.243145 (+   670us) log.cc:844] Serialized 52 byte log entry
0709 12:24:32.544678 (+301533us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:32.565717 (+ 21039us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.591780 (+ 26063us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.592067 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.592182 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.903432 (+311250us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b1800449f20. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:32.975598 (+ 72166us) raft_consensus.cc:1685] finished
0709 12:24:32.975648 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.976022 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30713,"prepare.queue_time_us":79425,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":1119,"raft.queue_time_us":10223,"raft.run_cpu_time_us":535,"raft.run_wall_time_us":17352,"spinlock_wait_cycles":21310464,"thread_start_us":300974,"threads_started":1,"wal-append.queue_time_us":408200}
I20260709 12:24:33.161614  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1091) took 1831 ms. Trace:
I20260709 12:24:33.126931  5074 rpcz_store.cc:276] 0709 12:24:31.184506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.688944 (+504438us) service_pool.cc:224] Handling call
0709 12:24:31.689368 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.689789 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:32.199030 (+509241us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.002872 (+803842us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b180040eb20. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a80e34bc 00007fd8a543148b 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db23a
0709 12:24:33.003294 (+   422us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:33.003392 (+    98us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:33.003499 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.004663 (+  1164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.004888 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":756016,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":4366,"raft.queue_time_us":50081,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":264,"spinlock_wait_cycles":52168576,"thread_start_us":692277,"threads_started":2,"wal-append.queue_time_us":384816}
I20260709 12:24:33.154448  8675 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.147991  8588 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:33.169880  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:33.182237  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 313626us
I20260709 12:24:33.160565  8682 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.184516  8660 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.255297  8660 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:33.256387  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:33.258116  5189 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:33.260100  5069 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:33.260478  8655 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:33.250317  5150 rpcz_store.cc:276] 0709 12:24:31.330072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.661842 (+331770us) service_pool.cc:224] Handling call
0709 12:24:31.662416 (+   574us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:31.738795 (+ 76379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.738830 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:31.790739 (+ 51909us) log.cc:844] Serialized 52 byte log entry
0709 12:24:32.159772 (+369033us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:32.498906 (+339134us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.511081 (+ 12175us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.511447 (+   366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.511558 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.910721 (+399163us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:33.160993 (+250272us) raft_consensus.cc:1685] finished
0709 12:24:33.161041 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.161338 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120165,"prepare.queue_time_us":542187,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":123,"raft.queue_time_us":76759,"raft.run_cpu_time_us":535,"raft.run_wall_time_us":49485,"spinlock_wait_cycles":10486016,"thread_start_us":655307,"threads_started":2,"wal-append.queue_time_us":634107}
W20260709 12:24:33.186046  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.186889  5221 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
W20260709 12:24:33.190356  8712 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.083s	user 0.002s	sys 0.000s
I20260709 12:24:33.185606  8534 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.194607  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:33.194069  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:33.196768  8674 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.201779  8017 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:24:33.202436  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:33.203306  8531 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:33.208597  8669 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:33.273525  5069 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:33.274875  5946 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:33.275913  5189 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:33.212523  8709 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:24:33.277338  8372 consensus_meta.cc:287] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.693s	user 0.023s	sys 0.005s
I20260709 12:24:33.277827  8674 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:33.278439  8674 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:33.278556  8709 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:33.279160  8674 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:32.923574  8666 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:33.204828  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 84199.4us
I20260709 12:24:33.221246  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:33.242620  8704 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:33.217301  8559 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.008s
I20260709 12:24:33.157440  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:33.244472  8673 consensus_queue.cc:1059] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:33.233896  8259 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:33.235951  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:33.233183  5041 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:33.233999  8535 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:33.237241  8598 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:33.248096  8654 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:33.261930  8655 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:24:33.263576  8557 consensus_meta.cc:287] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.810s	user 0.012s	sys 0.010s
I20260709 12:24:33.180677  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:33.225134  8579 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:33.269410  5222 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:33.271976  5218 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
W20260709 12:24:33.273237  8528 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.771s	user 0.009s	sys 0.015s
I20260709 12:24:33.278085  8372 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:33.267599  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:33.214558  8414 consensus_meta.cc:287] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.601s	user 0.019s	sys 0.006s
I20260709 12:24:33.215952  8098 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:33.203701  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:33.301764  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.356053  5221 raft_consensus.cc:2410] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:33.358777  5189 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:33.283965  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:33.363101  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:33.279290  8709 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:33.366739  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.258044  8660 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:33.367921  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:33.370325  5224 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:33.356948  8535 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:33.356614  8598 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:33.371460  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:33.371148  5072 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:33.360188  5114 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:33.371752  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:33.381070  5217 raft_consensus.cc:2368] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6.
I20260709 12:24:33.308065  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:33.379492  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:33.382831  5941 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:33.387523  5067 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:33.387696  5076 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:33.388164  5188 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:33.391034  8627 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:33.391249  8400 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:33.391590  8627 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:33.392117  8627 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:33.393260  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:33.394143  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:33.395120  5150 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:33.395946  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:33.396869  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:33.392045  8400 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:33.398846  5222 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:33.394198  5143 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:33.386183  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:33.397783  8660 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:33.403473  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:33.404687  5071 raft_consensus.cc:2368] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13.
I20260709 12:24:33.405197  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:33.408517  5067 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:33.379377  8682 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:33.381489  8535 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:33.416517  5220 raft_consensus.cc:2368] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13.
I20260709 12:24:33.358493  8414 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:33.418377  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:33.364109  8557 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:33.367471  8559 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:33.363351  8528 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:33.420698  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:33.414323  8682 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:33.419958  8654 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:33.431293  8666 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.438158  5947 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
W20260709 12:24:33.431803  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 198842us
I20260709 12:24:33.450161  5224 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:33.454643  8706 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:33.460232  8658 raft_consensus.cc:3055] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:33.460775  8658 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:33.463349  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:33.463620  5188 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:33.464335  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:33.465657  5144 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:24:33.466125  5149 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
W20260709 12:24:33.493321  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.466861  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:33.467283  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:33.471153  5147 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:24:33.475188  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 204148us
W20260709 12:24:33.475840  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 168201us
I20260709 12:24:33.459757  4899 catalog_manager.cc:5697] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:33.444694  5219 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:24:33.479269  5217 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:24:33.479727  8579 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.480399  5071 raft_consensus.cc:1217] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Deduplicated request from leader. Original: 10.6->[12.7-12.7]   Dedup: 12.7->[]
I20260709 12:24:33.481072  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:33.480945  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:33.484728  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.454023  5221 raft_consensus.cc:2376] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13. Re-sending same reply.
I20260709 12:24:33.486076  8704 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.486176  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:33.491390  8479 consensus_meta.cc:287] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.872s	user 0.013s	sys 0.010s
I20260709 12:24:33.491683  8607 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:33.492717  8510 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:33.466392  8400 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:33.493434  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:33.495667  5149 pending_rounds.cc:85] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 6
I20260709 12:24:33.444934  8682 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:33.502614  5071 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:24:33.514487  5149 pending_rounds.cc:107] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:24:33.516747  5148 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:33.520408  5149 raft_consensus.cc:2889] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:33.512758  5224 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:24:33.510848  5146 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:33.453985  8557 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.513419  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:33.513466  5150 raft_consensus.cc:2376] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11. Re-sending same reply.
W20260709 12:24:33.517202  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 53751.2us
I20260709 12:24:33.493242  8414 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:33.530699  8400 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:33.531570  8400 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:33.532116  8400 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:33.532785  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:33.532229  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:33.534008  5072 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:33.535537  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.539157  5069 raft_consensus.cc:2368] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13.
I20260709 12:24:33.544154  8658 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:33.545955  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:33.547338  8658 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:33.545830  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:33.548738  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 47590.8us
I20260709 12:24:33.523689  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:33.528453  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:33.547942  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:33.545768  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:33.551196  5215 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:33.548113  8655 raft_consensus.cc:2764] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
W20260709 12:24:33.545900  8647 consensus_meta.cc:287] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.723s	user 0.009s	sys 0.009s
I20260709 12:24:33.554710  8510 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:33.556082  8525 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:33.493645  8496 consensus_queue.cc:1059] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:24:33.512336  8607 consensus_queue.cc:1243] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:24:33.561368  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 201327us
I20260709 12:24:33.497679  8627 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:33.562873  5188 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:33.518905  8479 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:33.474032  8709 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:33.562366  8510 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.568746  5147 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:33.557952  8660 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:33.564672  8372 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:33.584617  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.569167  8525 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:33.571830  8559 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.576411  8510 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:33.577037  8647 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:33.578752  8098 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:33.605062  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 73030.9us
I20260709 12:24:33.608775  5070 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:24:33.609889  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 62093.6us
I20260709 12:24:33.613075  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:33.627727  5189 leader_election.cc:400] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7.
I20260709 12:24:33.628299  5189 leader_election.cc:403] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:24:33.628659  5216 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 7.
I20260709 12:24:33.629101  8704 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7.
I20260709 12:24:33.631328  8660 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.632017  8660 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:33.639397  5072 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:33.640830  5218 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:33.610751  8647 raft_consensus.cc:697] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:33.647807  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 96320.5us
I20260709 12:24:33.651592  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:33.633692  8709 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:33.649264  8528 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.667135  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:33.647228  8662 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:33.675027  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 112225us
I20260709 12:24:33.693924  4899 catalog_manager.cc:5697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:33.695392  5222 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
W20260709 12:24:33.699131  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 47541.9us
I20260709 12:24:33.664750  8666 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.696471  5189 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:33.701862  5115 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:33.709431  5898 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:33.709049  8098 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:33.712453  8098 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:33.713652  8581 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:33.713994  8098 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:33.717939  8484 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:33.719488  8484 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:33.721853  8484 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:33.723327  8484 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:33.730113  8479 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.730394  8666 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:33.732542  8535 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:33.760058  5219 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:33.761427  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:33.762221  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 77848.1us
I20260709 12:24:33.762809  5216 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:24:33.762681  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 58124.7us
I20260709 12:24:33.763217  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:33.764199  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 64033.6us
I20260709 12:24:33.764251  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:33.767637  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:33.768357  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 164816us
I20260709 12:24:33.765413  4899 catalog_manager.cc:5697] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:24:33.769632  8559 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:33.770609  5899 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:33.771301  5072 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:33.765431  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:33.758504  8607 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:33.780203  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:33.781713  5898 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:33.781955  5188 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:33.783460  8717 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:33.778360  8496 consensus_queue.cc:1059] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:24:33.783922  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:33.783552  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:33.784822  5114 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:33.785660  5070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 742) took 1171 ms (client timeout 1506 ms). Trace:
I20260709 12:24:33.789458  8706 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:33.776643  8647 consensus_queue.cc:237] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:33.790906  8674 consensus_meta.cc:287] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.000s
I20260709 12:24:33.767328  8654 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:33.791385  8535 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.792109  8531 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:33.793802  8721 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:33.801442  5076 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:33.805723  8714 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.130s	user 0.001s	sys 0.001s
W20260709 12:24:33.806352  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 52411.4us
I20260709 12:24:33.809492  5145 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:33.813884  8660 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:33.816211  8682 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:33.817721  8496 raft_consensus.cc:3055] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:33.818833  5114 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:33.819566  8722 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:33.798673  8717 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.845803  8717 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:33.848268  8721 raft_consensus.cc:697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:33.849113  8579 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:33.825860  8666 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:33.834972  8654 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:33.827064  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 161792us
I20260709 12:24:33.877943  8654 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:33.878448  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:33.800158  8414 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:33.827853  8607 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:33.801664  5070 rpcz_store.cc:269] 0709 12:24:32.603420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.606849 (+  3429us) service_pool.cc:224] Handling call
0709 12:24:33.758262 (+1151413us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b540040f954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:33.774869 (+ 16607us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2208499584}
I20260709 12:24:33.774948  8627 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.776240  8528 consensus_queue.cc:1059] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:24:33.758114  8525 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:33.842859  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 69944.3us
I20260709 12:24:33.839604  8496 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:33.845165  8682 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.849982  8721 consensus_queue.cc:237] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:33.774092  8708 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:33.883880  8414 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.857416  8660 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.858011  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:33.858728  8476 raft_consensus.cc:3055] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:33.859129  5074 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:24:33.826727  8662 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:33.866235  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:33.889430  8715 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:33.868227  5899 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:33.873481  8400 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:33.875037  8510 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:33.881904  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.929378  5898 leader_election.cc:400] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13.
I20260709 12:24:33.888487  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:33.835177  8674 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:33.896258  8531 raft_consensus.cc:3055] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:33.885226  8607 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:33.902489  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 139459us
I20260709 12:24:33.855021  8722 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:33.910472  5069 raft_consensus.cc:2410] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:33.863113  8579 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:33.867151  5076 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:33.909511  8414 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:33.906900  8704 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:33.917963  8476 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:33.953655  5071 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 12:24:33.954806  5223 raft_consensus.cc:2393] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:33.925390  8528 consensus_queue.cc:1059] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
W20260709 12:24:33.958921  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.934357  8531 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:33.937278  8660 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.971485  5188 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:33.944460  8579 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:33.953054  8598 consensus_meta.cc:287] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.002s
I20260709 12:24:33.981583  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:33.954680  8709 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:33.970211  8662 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.982465  5067 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:33.971856  8720 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:33.972487  8722 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:33.938555  8715 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:34.055212  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 100038us
I20260709 12:24:34.055488  8715 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:34.056864  5041 connection.cc:256] RPC call timeout handler was delayed by -1.8356207639999411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.165s for a response
W20260709 12:24:34.057400  5041 connection.cc:256] RPC call timeout handler was delayed by -1.7690207640002882s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 12:24:34.057765  5041 connection.cc:256] RPC call timeout handler was delayed by -1.7213207640002111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 12:24:34.058079  5041 connection.cc:256] RPC call timeout handler was delayed by -1.3136207640000066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
W20260709 12:24:34.058352  5041 connection.cc:256] RPC call timeout handler was delayed by -1.1858207640002547s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2372s for a response
I20260709 12:24:34.059051  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:34.059567  5041 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:34.060590  8682 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:34.061036  4899 catalog_manager.cc:5697] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 4 to 6, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 6 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:34.035750  8717 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.042565  8531 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:34.063076  4899 catalog_manager.cc:5697] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 4 to 5, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 5 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:34.045707  8476 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:34.065002  8476 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:34.025410  8719 log.cc:927] Time spent T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.116s	user 0.000s	sys 0.005s
I20260709 12:24:33.988621  8496 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:34.066457  8717 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:34.049052  8510 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:34.052587  8727 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:24:33.988751  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 62889.2us
I20260709 12:24:34.074966  5898 leader_election.cc:400] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13.
I20260709 12:24:34.063660  8531 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:34.077119  8674 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:34.079470  8718 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.244s	user 0.009s	sys 0.001s
I20260709 12:24:34.043931  8400 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:34.068131  8496 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:34.049110  8725 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.071374  8654 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:34.075276  8720 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:34.098995  8720 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:34.046882  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 178771us
I20260709 12:24:34.101697  8654 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:33.986183  8525 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:34.112594  8525 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:34.113039  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:34.112507  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:34.114946  8531 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:34.115890  5899 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:34.116281  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:34.112443  8722 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.124464  8525 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.083288  5898 leader_election.cc:403] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:24:34.133037  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 58248.7us
I20260709 12:24:34.133627  8531 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:34.072801  4899 catalog_manager.cc:5697] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 5 to 6, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 6 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.090317  5073 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:34.095719  8725 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.096796  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:34.032362  8728 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:24:34.137679  5898 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:34.178002  4899 catalog_manager.cc:5697] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 5 to 7, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 7 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:24:34.092895  8674 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.192716  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:34.198930  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 83148.6us
I20260709 12:24:34.201750  8709 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:34.204623  8098 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:34.204864  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:34.203899  8531 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13.
I20260709 12:24:34.159759  8400 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:34.205246  8098 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.205951  8400 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:34.206382  8400 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:34.207562  8098 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:34.208786  5075 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:24:34.209059  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:34.209656  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:34.210115  5071 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 9: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:34.209944  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:34.211043  5076 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:34.211236  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:34.211982  4898 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49944 (request call id 78) took 19744 ms (client timeout 15000 ms). Trace:
I20260709 12:24:34.213217  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:34.214231  5941 raft_consensus.cc:2410] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:34.201431  5899 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:34.217425  5899 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:34.218000  8531 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:34.217859  8674 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.218890  8531 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:34.217621  8400 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.221873  8400 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:33.987129  8660 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.222980  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:34.231513  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.985401  8598 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:34.244464  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 253105us
I20260709 12:24:34.162189  8720 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:34.091257  8662 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.266453  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:34.267438  5069 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:34.215626  8725 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:34.217494  8588 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:34.222297  8674 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:34.277227  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:34.282434  5899 leader_election.cc:400] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6.
I20260709 12:24:34.280565  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:34.284936  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:34.285616  5219 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:24:34.286036  5219 pending_rounds.cc:85] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 5
I20260709 12:24:34.286235  5219 pending_rounds.cc:107] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:24:34.286487  5219 raft_consensus.cc:2889] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:34.254046  8682 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:34.258400  8720 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:34.292153  5218 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:34.303272  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 187118us
I20260709 12:24:34.305088  5067 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:34.306694  5188 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:34.308832  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45278us
I20260709 12:24:34.283545  8588 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:34.311110  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 220956us
I20260709 12:24:34.315703  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:34.310403  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:34.326857  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:34.302511  8720 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:34.328208  5216 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:34.314842  8525 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:34.308027  8674 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:34.340392  5216 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
W20260709 12:24:34.341267  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 64850.8us
I20260709 12:24:34.342032  5114 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:34.290637  8607 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.344861  5899 leader_election.cc:400] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13.
W20260709 12:24:34.212390  4898 rpcz_store.cc:269] 0709 12:24:14.468040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.468255 (+   215us) service_pool.cc:224] Handling call
0709 12:24:34.211682 (+19743427us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:25.197076 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:25.197904 (+   828us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:25.197967 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:24:25.198006 (+    39us) tablet.cc:662] Decoding operations
0709 12:24:25.199978 (+  1972us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:25.200064 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:25.200083 (+    19us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:24:25.201892 (+  1809us) tablet.cc:701] Row locks acquired
0709 12:24:25.201935 (+    43us) write_op.cc:260] PREPARE: finished
0709 12:24:25.202145 (+   210us) write_op.cc:270] Start()
0709 12:24:25.202263 (+   118us) write_op.cc:276] Timestamp: P: 1783599865202081 usec, L: 0
0709 12:24:25.202277 (+    14us) op_driver.cc:348] REPLICATION: starting
0709 12:24:25.203445 (+  1168us) log.cc:844] Serialized 8230 byte log entry
0709 12:24:25.215630 (+ 12185us) op_driver.cc:464] REPLICATION: finished
0709 12:24:25.253096 (+ 37466us) write_op.cc:301] APPLY: starting
0709 12:24:25.253247 (+   151us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:25.253691 (+   444us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:25.253829 (+   138us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:25.256628 (+  2799us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:25.257136 (+   508us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=21
0709 12:24:25.257204 (+    68us) write_op.cc:312] APPLY: finished
0709 12:24:25.262126 (+  4922us) log.cc:844] Serialized 144 byte log entry
0709 12:24:25.263197 (+  1071us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:25.263458 (+   261us) write_op.cc:454] Released schema lock
0709 12:24:25.263550 (+    92us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:25.269336 (+  5786us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:25.269383 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":860340,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":37052,"apply.run_cpu_time_us":5958,"apply.run_wall_time_us":16196,"num_ops":21,"prepare.queue_time_us":5082,"prepare.run_cpu_time_us":7551,"prepare.run_wall_time_us":7763,"raft.queue_time_us":7354,"raft.run_cpu_time_us":17693,"raft.run_wall_time_us":31197,"replication_time_us":13164,"thread_start_us":40581,"threads_started":4,"wal-append.queue_time_us":1183,"wal-append.run_cpu_time_us":10547,"wal-append.run_wall_time_us":1063691}]]}
W20260709 12:24:34.353413  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.363077  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:34.364073  5943 raft_consensus.cc:2410] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:24:34.364632  8414 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.363116  8476 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:24:34.375198  8660 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:34.356555  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:34.378113  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:34.379525  8669 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:34.339224  8682 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:34.380069  8669 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:34.380671  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:34.380575  8669 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:34.380470  8682 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:34.383859  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:34.384171  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:34.385651  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:34.385895  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:34.386186  5220 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:24:34.386386  5224 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:34.386595  5220 pending_rounds.cc:85] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 4
I20260709 12:24:34.386682  5221 raft_consensus.cc:2368] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8.
I20260709 12:24:34.386909  8607 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:34.385250  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:34.387590  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:34.388451  5215 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:34.388365  5218 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:34.344058  8674 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.389515  8674 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:34.393952  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:34.394778  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:34.394533  8662 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:34.396823  5190 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.586s (SENT) [suppressed 47 similar messages]
I20260709 12:24:34.398401  8414 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:34.398303  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:34.399309  5943 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:34.400633  5943 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:34.347821  8525 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:34.401950  8525 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.403013  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:34.403911  5943 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:34.404343  8557 raft_consensus.cc:3055] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:34.408155  8557 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:34.408277  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:34.408847  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:34.405305  8733 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:34.409394  5942 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:34.409229  8557 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:34.410167  8733 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:34.410704  8557 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:34.410802  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:34.411160  8733 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.378064  5941 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:34.386917  5220 pending_rounds.cc:107] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:24:34.405601  5940 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:34.407212  5939 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:34.407538  5944 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:34.404695  5899 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:34.420389  5220 raft_consensus.cc:2889] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:34.430732  5946 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
W20260709 12:24:34.437347  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 95384.7us
W20260709 12:24:34.433925  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.442121  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:34.443413  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:34.448339  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:34.449211  5068 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:34.452447  5946 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:34.333884  8722 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:34.463560  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.464923  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 155396us
I20260709 12:24:34.387848  8598 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:34.447105  8528 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:34.474758  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:34.475765  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:34.481913  8725 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.482502  5076 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:34.430845  8682 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:34.481941  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:34.492403  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 111842us
W20260709 12:24:34.510138  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 105564us
I20260709 12:24:34.495568  5946 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
W20260709 12:24:34.499126  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 102350us
I20260709 12:24:34.447621  8666 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:34.447314  8662 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:34.514339  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 48349us
I20260709 12:24:34.346818  8579 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:34.470947  8400 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:34.471935  8484 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:34.523159  8647 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.487844  8722 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.524633  8721 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:34.525238  8598 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.527329  8400 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:34.533288  8666 raft_consensus.cc:697] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:34.522673  8662 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.535673  8484 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:34.521489  8725 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.584131  5069 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:24:34.584601  5069 pending_rounds.cc:85] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 4
I20260709 12:24:34.593087  5069 pending_rounds.cc:107] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:24:34.593487  5069 raft_consensus.cc:2889] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:34.550189  8647 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.579685  8697 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:24:34.612977  8732 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:34.613783  8732 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:34.616148  8732 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:34.617614  8741 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:24:34.619163  8722 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:34.627945  5899 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:34.630671  8666 consensus_queue.cc:237] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:34.632357  5073 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:34.639405  5070 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:34.577968  8717 consensus_queue.cc:1059] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:24:34.558954  8598 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:34.650632  5074 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:24:34.653980  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 95515.8us
I20260709 12:24:34.582381  8669 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.555774  8588 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:34.583290  8721 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:34.636129  8414 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.588230  8674 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.664973  5899 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:34.669503  8098 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:34.670197  8484 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.638540  8496 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:24:34.678350  8662 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:34.679720  8721 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:34.648177  8709 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:34.648144  8660 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:24:34.681556  8400 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.682958  8654 consensus_queue.cc:1059] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:34.665519  8708 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:24:34.684650  8557 raft_consensus.cc:3055] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:34.653575  8528 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:24:34.685107  8557 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:34.684935  8709 raft_consensus.cc:697] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:34.687337  5115 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:24:34.701745  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 120934us
I20260709 12:24:34.639063  8535 consensus_queue.cc:1059] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:24:34.716284  5219 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:34.712935  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:34.751756  8598 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:34.712001  8717 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:34.693799  8414 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:34.772059  5068 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:34.772604  5943 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:34.714785  8709 consensus_queue.cc:237] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:34.813130  5188 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:34.773273  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.817211  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.815830  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 152291us
I20260709 12:24:34.718446  8098 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:34.771427  8735 raft_consensus.cc:3055] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:34.821010  8708 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:34.759563  8717 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:34.774446  8697 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:34.849973  8697 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:34.851193  8697 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:34.848582  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:34.775482  8579 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:34.853885  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 146624us
I20260709 12:24:34.773871  8557 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:34.857861  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 84363.5us
I20260709 12:24:34.858865  5041 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:34.852306  8535 consensus_queue.cc:1059] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:24:34.760725  8749 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 12:24:34.834973  8735 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:34.860458  5899 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:34.852610  8717 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:34.870422  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 118366us
I20260709 12:24:34.838647  8484 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:34.895754  8579 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:24:34.898376  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 151858us
I20260709 12:24:34.856415  8660 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:34.854569  8708 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:34.892298  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 43837.6us
I20260709 12:24:34.866878  8752 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:34.866773  8725 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:34.914458  8647 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:34.916299  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:34.918373  5067 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:34.904683  8098 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:34.919096  8763 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.919855  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:34.920846  8674 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:34.921883  8581 raft_consensus.cc:3055] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:34.922315  8581 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:34.923265  8581 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:34.901006  8476 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:24:34.924571  8581 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:34.926232  8735 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.927459  8735 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:34.927729  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:34.892172  8754 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:34.919739  8763 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.929698  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:34.938745  5222 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:34.929723  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.942427  8720 consensus_meta.cc:287] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.596s	user 0.005s	sys 0.009s
I20260709 12:24:34.942950  8720 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:34.943562  8763 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:34.918366  8755 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:34.945910  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:34.947351  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:34.947733  5076 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:34.948149  5071 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:34.936708  8754 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:34.950150  8754 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:34.954133  8715 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:34.904152  8682 raft_consensus.cc:3055] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:34.960520  5220 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:34.972167  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:34.975687  5068 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:34.945286  8720 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:34.985842  5216 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:34.985870  5219 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:34.981215  5943 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:34.979750  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:34.987254  5215 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:34.937678  8557 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:34.989346  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:34.991655  5941 pending_rounds.cc:85] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 4
I20260709 12:24:35.001973  5941 pending_rounds.cc:107] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:24:35.002772  5941 raft_consensus.cc:2889] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:35.003306  5188 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:35.001288  5944 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:35.005513  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:35.006314  5217 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:35.006461  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.007648  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:35.011781  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:35.015343  5219 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:34.938484  8669 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:34.906535  8660 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:34.974064  8682 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:35.037827  5944 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:35.019160  5947 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:35.031144  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:35.031858  8762 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:34.986773  8749 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:34.985280  8510 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:35.028676  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:35.046227  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 66331.2us
I20260709 12:24:35.047189  5068 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:35.055182  5939 raft_consensus.cc:2410] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:24:35.047374  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.055032  5947 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:35.032711  8414 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:35.061348  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:35.063421  5218 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:35.048326  5071 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:34.987620  8535 consensus_queue.cc:1243] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:34.987149  8752 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:34.978395  8725 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:24:34.979254  8715 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:35.058878  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:35.028579  8525 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:35.067488  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:35.018515  8720 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:35.070103  5219 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:35.072778  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:35.068128  5944 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:35.074169  5947 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:35.082165  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:35.083104  5943 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:35.037936  8598 consensus_queue.cc:1059] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:24:35.069288  8525 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.094475  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:35.090636  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.091138  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:35.091807  5147 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:35.090054  8400 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:35.085767  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:35.096812  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:35.086051  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 73335.7us
I20260709 12:24:35.091614  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.069036  8721 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:35.133469  5218 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:35.126567  5141 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:35.141959  5144 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:24:35.129086  5219 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:24:35.129379  8674 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:35.085062  8484 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.062659  8510 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:35.057657  8749 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:35.132833  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:35.138379  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:35.157961  5943 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:35.018559  5940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:35.158941  5940 raft_consensus.cc:606] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:35.159412  5940 raft_consensus.cc:614] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:35.141891  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:35.126104  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:35.146510  5146 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:35.097286  8525 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:35.151252  5143 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:35.151372  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:35.157788  5148 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:35.128437  8754 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:35.049289  8708 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:35.164526  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:35.162956  5223 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:35.171790  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.093947  8720 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:35.244401  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:35.360692  5041 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.017s	sys 0.037s
I20260709 12:24:35.149943  8414 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.149494  8733 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:35.130689  8400 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:35.111301  8682 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:35.156817  8721 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:35.649076  5041 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.025s	sys 0.060s
I20260709 12:24:35.649519  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.443728  5148 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
W20260709 12:24:35.649693  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 790821us
W20260709 12:24:35.402587  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.561364  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:35.362996  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.650987  5041 leader_election.cc:400] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8.
I20260709 12:24:35.651721  5074 raft_consensus.cc:1217] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Deduplicated request from leader. Original: 7.5->[8.6-10.7]   Dedup: 10.7->[]
I20260709 12:24:35.657518  5041 leader_election.cc:403] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 12:24:35.655202  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.445874  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:35.540776  5948 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:35.658353  5074 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 12:24:35.661736  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.652357  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:35.362119  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:35.636250  5142 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.541s	user 0.008s	sys 0.000s
W20260709 12:24:35.606875  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.681826  5142 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
W20260709 12:24:35.443266  8581 consensus_meta.cc:287] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.518s	user 0.033s	sys 0.025s
I20260709 12:24:35.683530  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:35.665119  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 181168us
I20260709 12:24:35.613771  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:35.669086  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:35.685837  5218 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:24:35.664705  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 203704us
I20260709 12:24:35.669085  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:35.666894  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:35.444725  8772 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.348s	user 0.000s	sys 0.000s
I20260709 12:24:35.708359  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:35.709371  5216 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:35.614984  8510 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:35.649971  8733 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:35.727253  8733 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:35.728353  8733 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:35.730489  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.732100  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:35.733650  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.734210  5142 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:35.736238  8662 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.736766  8662 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:35.740078  5150 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:35.739352  8715 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:35.648494  5219 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:35.743073  8721 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:35.748346  8662 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:35.752085  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.753329  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.754503  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.755302  5150 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:35.756032  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:35.756669  5150 raft_consensus.cc:2393] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 10: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:35.758381  8774 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.612s	user 0.000s	sys 0.001s
I20260709 12:24:35.363512  8777 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:35.164351  8669 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.509s
I20260709 12:24:35.761082  8777 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:35.715867  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.721143  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:35.764622  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:35.663578  8768 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 12:24:35.723567  8773 log.cc:927] Time spent T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 12:24:35.765563  8660 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:35.724004  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:35.725962  8720 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:35.376945  8752 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:35.767884  8749 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:35.768191  8535 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8.
I20260709 12:24:35.769673  8765 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:35.713045  5940 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:35.754174  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:35.362807  8484 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:35.866026  5042 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.016s	sys 0.019s
I20260709 12:24:35.770332  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:36.410835  5042 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.276s	user 0.025s	sys 0.037s
I20260709 12:24:36.411375  5939 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:36.411352  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 1.27895e+06us
I20260709 12:24:36.398151  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:36.413321  5944 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:35.786536  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.417475  5150 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:36.417810  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 777) took 1043 ms. Trace:
W20260709 12:24:35.866278  5188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.004s	sys 0.031s
W20260709 12:24:36.418648  5188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.358s	user 0.010s	sys 0.052s
I20260709 12:24:36.418560  5940 rpcz_store.cc:276] 0709 12:24:35.374603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.601149 (+226546us) service_pool.cc:224] Handling call
0709 12:24:36.417726 (+816577us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:36.177754  5898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.000s	sys 0.010s
W20260709 12:24:36.301431  5148 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.000s
W20260709 12:24:36.419493  5898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.003s	sys 0.013s
I20260709 12:24:36.419709  5148 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
W20260709 12:24:35.853221  5141 consensus_meta.cc:287] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.708s	user 0.012s	sys 0.001s
W20260709 12:24:36.131843  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.420569  5141 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
W20260709 12:24:36.416175  5144 consensus_meta.cc:287] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.244s	user 0.011s	sys 0.000s
W20260709 12:24:36.308728  5897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.007s	sys 0.038s
I20260709 12:24:36.421289  5144 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
W20260709 12:24:36.421546  5897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.007s	sys 0.038s
W20260709 12:24:36.416775  5215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.006s	sys 0.007s
W20260709 12:24:36.421450  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 1.36057e+06us
I20260709 12:24:36.424255  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:36.004281  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 335230us
I20260709 12:24:36.438606  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 660) took 1347 ms. Trace:
W20260709 12:24:36.139273  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:36.440613  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:35.783660  5217 consensus_meta.cc:287] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.769s	user 0.010s	sys 0.000s
I20260709 12:24:36.441184  5217 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:24:36.441366  5150 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:36.426584  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:35.825683  5143 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5.
W20260709 12:24:36.374768  5942 consensus_meta.cc:287] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.298s	user 0.012s	sys 0.001s
I20260709 12:24:36.442456  5942 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
W20260709 12:24:36.033929  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.427562  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:35.810647  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:35.764667  8666 raft_consensus.cc:3055] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:36.443347  5071 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:36.428292  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:36.442129  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 672) took 1265 ms. Trace:
I20260709 12:24:36.444119  5068 raft_consensus.cc:2393] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:35.811414  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:36.426589  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:36.444181  5143 rpcz_store.cc:276] 0709 12:24:35.177192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.445476 (+268284us) service_pool.cc:224] Handling call
0709 12:24:36.442068 (+996592us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:36.430279  8647 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.009s	sys 0.014s
I20260709 12:24:35.664165  8414 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:35.885838  5946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.008s	sys 0.028s
W20260709 12:24:36.452693  5946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.375s	user 0.024s	sys 0.044s
I20260709 12:24:36.453510  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:36.425637  5215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.007s	sys 0.007s
I20260709 12:24:36.454797  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 882) took 1387 ms. Trace:
I20260709 12:24:35.792464  5142 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
W20260709 12:24:36.000494  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.829325  5146 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:36.429165  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 668) took 1333 ms. Trace:
I20260709 12:24:36.457042  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:36.456960  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.456099  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.448325  5948 raft_consensus.cc:2393] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 7: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:36.448758  5189 leader_election.cc:336] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Timed out: RequestConsensusVote RPC to 127.1.123.65:37253 timed out after 1.756s (SENT)
W20260709 12:24:36.399526  8780 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:24:36.458877  5940 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:24:36.342510  8778 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
W20260709 12:24:35.839831  8766 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.003s	sys 0.017s
I20260709 12:24:36.459117  5069 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:24:36.447467  8647 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.010s	sys 0.014s
W20260709 12:24:36.469156  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.455595  5942 rpcz_store.cc:276] 0709 12:24:35.055288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.055459 (+   171us) service_pool.cc:224] Handling call
0709 12:24:36.442901 (+1387442us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:36.453771  8557 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.442s	user 0.009s	sys 0.005s
I20260709 12:24:36.460572  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:36.406523  8717 consensus_meta.cc:287] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.507s	user 0.010s	sys 0.002s
I20260709 12:24:36.459316  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.457494  5144 rpcz_store.cc:276] 0709 12:24:35.095491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.156536 (+ 61045us) service_pool.cc:224] Handling call
0709 12:24:36.429053 (+1272517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.439334  5141 rpcz_store.cc:276] 0709 12:24:35.090842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.091225 (+   383us) service_pool.cc:224] Handling call
0709 12:24:36.438500 (+1347275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.465667  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:35.769956  8588 raft_consensus.cc:3055] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:36.479529  8588 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:36.480589  8588 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:36.400826  8779 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:36.469172  8766 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.309s	user 0.005s	sys 0.024s
W20260709 12:24:36.426653  8752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.025s	sys 0.008s
I20260709 12:24:36.466915  5945 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:24:36.517280  5945 pending_rounds.cc:85] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 3
I20260709 12:24:36.517649  5945 pending_rounds.cc:107] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:24:36.485451  8588 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:36.228928  8776 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:36.518554  8776 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:36.520607  8557 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:36.521147  8776 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:36.521813  8776 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:36.522581  8776 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:36.523087  8776 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:36.524991  8776 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:36.517668  8735 consensus_meta.cc:287] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.590s	user 0.012s	sys 0.002s
I20260709 12:24:36.526907  8496 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.527384  8496 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:36.530068  8787 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:36.530632  8787 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:36.532661  8787 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:36.532769  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 57279.6us
I20260709 12:24:36.458114  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 1066) took 1456 ms. Trace:
I20260709 12:24:36.472678  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.474085  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.535629  5143 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:36.535918  5147 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:36.478996  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:36.534902  5217 rpcz_store.cc:276] 0709 12:24:34.985814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.986253 (+   439us) service_pool.cc:224] Handling call
0709 12:24:36.441749 (+1455496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.521157  8557 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:36.525436  8776 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.463619  5145 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:36.448108  8666 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:36.488255  5947 raft_consensus.cc:2410] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:24:36.489827  8484 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:36.490072  5075 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1255) took 1511 ms. Trace:
I20260709 12:24:36.490460  8791 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:36.490893  8779 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:36.494854  8733 consensus_meta.cc:287] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.766s	user 0.010s	sys 0.000s
I20260709 12:24:36.554420  5075 rpcz_store.cc:276] 0709 12:24:34.978947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.979090 (+   143us) service_pool.cc:224] Handling call
0709 12:24:35.004367 (+ 25277us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:35.029599 (+ 25232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.029638 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:35.056994 (+ 27356us) log.cc:844] Serialized 52 byte log entry
0709 12:24:35.489238 (+432244us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:35.489307 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.489419 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.489842 (+1000423us) raft_consensus.cc:1685] finished
0709 12:24:36.489864 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.490014 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13136,"raft.queue_time_us":52531,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":15084,"thread_start_us":351624,"threads_started":1,"wal-append.queue_time_us":919135}
I20260709 12:24:36.496997  8674 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.498239  8783 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.500720  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:36.559201  8783 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:36.484941  8717 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:36.502410  8682 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.560844  8666 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:36.502565  8784 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:24:36.504465  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:36.505218  8777 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:36.500741  8414 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:36.512213  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 769) took 1521 ms. Trace:
I20260709 12:24:36.513664  8400 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:36.514232  8525 consensus_meta.cc:287] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.338s	user 0.011s	sys 0.000s
I20260709 12:24:36.514297  8098 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.515182  8788 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.515877  8786 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:36.516824  8792 log.cc:927] Time spent T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.099s	user 0.001s	sys 0.001s
I20260709 12:24:35.770203  8765 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.517935  5945 raft_consensus.cc:2889] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:35.847123  8754 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:36.540109  8752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.025s	sys 0.008s
I20260709 12:24:36.551450  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:36.484447  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:36.588591  5144 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:36.554811  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.590271  5150 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:36.496742  8771 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 1.437s	user 0.004s	sys 0.025s
I20260709 12:24:36.558336  8674 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:36.559406  5148 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:36.502441  8789 raft_consensus.cc:2764] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:24:36.566680  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 62312.8us
I20260709 12:24:36.567621  5142 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:36.568944  5941 rpcz_store.cc:276] 0709 12:24:34.990654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.990897 (+   243us) service_pool.cc:224] Handling call
0709 12:24:34.991312 (+   415us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:35.047152 (+ 55840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.047190 (+    38us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:35.050598 (+  3408us) log.cc:844] Serialized 78 byte log entry
0709 12:24:35.490413 (+439815us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:35.490459 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.490546 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.511941 (+1021395us) raft_consensus.cc:1685] finished
0709 12:24:36.511965 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.512128 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45922,"raft.queue_time_us":44067,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":8132,"thread_start_us":412558,"threads_started":1,"wal-append.queue_time_us":809907}
I20260709 12:24:36.580598  8788 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:36.581411  8717 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:36.582221  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:36.600176  5143 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:36.583101  5076 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:36.576723  8484 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:36.602185  8484 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:36.602810  8484 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:36.603542  5947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 888) took 1452 ms (client timeout 1663 ms). Trace:
W20260709 12:24:36.604034  5947 rpcz_store.cc:269] 0709 12:24:35.150945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.151109 (+   164us) service_pool.cc:224] Handling call
0709 12:24:36.603321 (+1452212us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400007654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:36.603482 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3042508672}
I20260709 12:24:36.605587  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 671) took 1446 ms. Trace:
I20260709 12:24:36.606012  5146 rpcz_store.cc:276] 0709 12:24:35.159189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.163252 (+  4063us) service_pool.cc:224] Handling call
0709 12:24:36.605403 (+1442151us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b540045ae54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:36.605503 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126216064}
I20260709 12:24:36.606680  8400 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:36.607070  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:36.608330  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:36.608908  5146 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
W20260709 12:24:36.609650  8794 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.191s	user 0.006s	sys 0.000s
I20260709 12:24:36.609750  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.611090  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.611747  5146 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:36.612504  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:36.583520  8098 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:36.613766  8788 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:36.615177  5145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 659) took 1524 ms (client timeout 1506 ms). Trace:
W20260709 12:24:36.615594  5145 rpcz_store.cc:269] 0709 12:24:35.090529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.090731 (+   202us) service_pool.cc:224] Handling call
0709 12:24:36.614997 (+1524266us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540019ee54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:36.615107 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3144716032}
I20260709 12:24:36.616097  8098 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:36.609215  8735 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:24:36.617072  5145 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:36.556237  8776 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:36.619484  8776 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:36.579213  8525 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:36.621047  5218 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:36.585484  8765 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:36.613833  5146 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:24:36.585340  8598 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.626784  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.627523  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1079) took 1012 ms. Trace:
I20260709 12:24:36.625968  5147 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:36.577694  8783 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:36.628165  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.585731  8557 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:36.629702  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:36.631109  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.635782  5148 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:36.631925  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:36.627944  5218 rpcz_store.cc:276] 0709 12:24:35.615354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.715463 (+100109us) service_pool.cc:224] Handling call
0709 12:24:35.715978 (+   515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.621419 (+905441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.627281 (+  5862us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54001635d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:36.627462 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081008000}
I20260709 12:24:36.636965  5143 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:36.596446  8779 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:36.633693  5147 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:36.574077  8791 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:36.641966  5150 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:36.646242  5144 raft_consensus.cc:3055] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:36.646828  5144 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:36.651466  5190 leader_election.cc:336] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.586s (SENT)
I20260709 12:24:36.652145  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:36.595767  8786 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:36.645257  8525 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:36.654825  5144 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:36.680198  5144 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:36.633342  8598 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:36.655495  8783 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:36.682881  5143 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
W20260709 12:24:36.682626  5190 leader_election.cc:336] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.704s (SENT)
I20260709 12:24:36.709646  5190 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:36.661219  8496 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:36.724905  5144 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:24:36.726935  5144 pending_rounds.cc:85] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 5
I20260709 12:24:36.727237  5144 pending_rounds.cc:107] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:24:36.727970  5144 raft_consensus.cc:2889] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:36.729784  5190 leader_election.cc:336] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.755s (SENT)
W20260709 12:24:36.730203  5041 connection.cc:256] RPC call timeout handler was delayed by -1.3473211369996534s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
I20260709 12:24:36.730391  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:36.730783  5041 connection.cc:256] RPC call timeout handler was delayed by -1.3023211369995806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 12:24:36.731096  5041 connection.cc:256] RPC call timeout handler was delayed by -1.285221136999553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 12:24:36.731392  5041 connection.cc:256] RPC call timeout handler was delayed by -1.2591211369995108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 12:24:36.731763  5041 connection.cc:256] RPC call timeout handler was delayed by -1.1556211369997982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 12:24:36.732117  5041 connection.cc:256] RPC call timeout handler was delayed by -1.0755211369996687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 12:24:36.732497  5041 connection.cc:256] RPC call timeout handler was delayed by -1.0125211369995668s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
I20260709 12:24:36.738071  8784 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:36.738739  8784 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:36.741178  8784 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:36.747017  8791 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:36.759413  8535 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.760246  8535 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:36.763818  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:36.768716  8786 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:36.713887  8697 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:36.784075  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 54328us
W20260709 12:24:36.785158  5190 leader_election.cc:336] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.770s (SENT)
I20260709 12:24:36.711530  8717 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:24:36.765628  8776 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:36.831996  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:36.835680  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 71978.7us
I20260709 12:24:36.836959  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:36.841271  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 56154.7us
W20260709 12:24:36.849332  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.838696  8790 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:36.828228  8697 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:36.840868  8666 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:36.855226  8674 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:36.895998  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 767) took 1921 ms. Trace:
I20260709 12:24:36.897701  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:36.901312  5899 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:36.902791  8098 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:36.899012  8682 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.904424  8525 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:36.904062  8682 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:36.901743  8790 consensus_queue.cc:1059] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:24:36.905579  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1076) took 1732 ms. Trace:
I20260709 12:24:36.907001  8682 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:36.917694  8098 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.923100  8598 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:36.909900  8789 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:36.910620  8779 raft_consensus.cc:3055] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:24:36.889469  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.906051  5215 rpcz_store.cc:276] 0709 12:24:35.172926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.444641 (+271715us) service_pool.cc:224] Handling call
0709 12:24:35.444976 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:35.545264 (+100288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.545314 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:35.650333 (+105019us) log.cc:844] Serialized 26 byte log entry
0709 12:24:36.455568 (+805235us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:36.473253 (+ 17685us) log.cc:844] Serialized 14 byte log entry
0709 12:24:36.902492 (+429239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.902615 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.902671 (+    56us) raft_consensus.cc:1685] finished
0709 12:24:36.902687 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.905465 (+  2778us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19879,"prepare.queue_time_us":14156,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":158019,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":83288,"thread_start_us":1049542,"threads_started":2,"wal-append.queue_time_us":878588}
I20260709 12:24:36.909125  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:36.925959  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:36.927645  5190 leader_election.cc:336] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.945s (SENT)
I20260709 12:24:36.928309  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:36.929057  5942 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:36.898999  5946 rpcz_store.cc:276] 0709 12:24:34.974984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.975192 (+   208us) service_pool.cc:224] Handling call
0709 12:24:34.996977 (+ 21785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.997323 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:35.037984 (+ 40661us) log.cc:844] Serialized 14 byte log entry
0709 12:24:36.868781 (+1830797us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:36.868837 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:36.895692 (+ 26855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.895790 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.895925 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3334,"prepare.queue_time_us":31591,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":61693,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"thread_start_us":1787067,"threads_started":2,"wal-append.queue_time_us":1433157}
I20260709 12:24:36.938501  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:36.938619  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:36.939802  5941 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 5: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:36.937268  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:36.924512  8779 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:36.954169  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:36.955344  5218 raft_consensus.cc:2410] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 14 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:24:36.956336  5945 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:36.952123  5219 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:24:36.961891  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:36.964624  5149 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.006s	sys 0.011s
W20260709 12:24:36.965090  5149 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.006s	sys 0.011s
I20260709 12:24:36.946290  5899 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:36.967312  8733 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:36.967918  8733 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:36.968555  8733 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:36.983644  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 51183.9us
I20260709 12:24:36.968441  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:36.934476  8784 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:36.991216  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 530896us
I20260709 12:24:36.991935  5215 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:36.940054  8789 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:37.006243  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:36.955996  8788 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:37.006727  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:37.007166  5221 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:24:37.011050  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 562248us
I20260709 12:24:37.016638  5188 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:37.011466  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 663) took 1918 ms. Trace:
W20260709 12:24:37.021694  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 414255us
I20260709 12:24:37.030099  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:36.950222  8598 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:24:37.018198  5149 rpcz_store.cc:276] 0709 12:24:35.093347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.093459 (+   112us) service_pool.cc:224] Handling call
0709 12:24:35.134928 (+ 41469us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:35.135487 (+   559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.135530 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:35.750303 (+614773us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b180030a600. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a541c2a6 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a
0709 12:24:35.750702 (+   399us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b2c00193f58. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:24:35.751278 (+   576us) log.cc:844] Serialized 52 byte log entry
0709 12:24:36.063023 (+311745us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:36.965602 (+902579us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.011007 (+ 45405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.011108 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.011185 (+    77us) raft_consensus.cc:1685] finished
0709 12:24:37.011203 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.011390 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":7672,"spinlock_wait_cycles":33644160,"thread_start_us":1072614,"threads_started":2,"wal-append.queue_time_us":834960}
I20260709 12:24:37.039243  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.006211  8789 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.041587  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:37.042814  5942 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:37.015076  8788 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.047341  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.052469  5940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5deada35125f4c008544b2ff246b039c"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:37.053336  5940 raft_consensus.cc:606] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:24:37.053776  5940 raft_consensus.cc:614] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:37.015909  8708 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:37.051173  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 44639.8us
I20260709 12:24:37.059971  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.021814  8682 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:37.016642  8535 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:37.068600  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:37.069049  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:37.070020  5218 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:37.070163  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:37.071229  8790 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:37.069628  5219 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:37.069187  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:37.078620  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:37.079499  5215 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:37.079963  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:37.080719  5217 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
W20260709 12:24:37.080972  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 70409.9us
I20260709 12:24:37.040556  8674 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:37.081640  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:37.095569  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.059134  8786 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:37.095402  5221 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:37.051399  8789 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:37.058423  8788 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:37.115294  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:37.116978  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 77194us
I20260709 12:24:37.117442  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:37.122763  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.083747  8735 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:37.112217  5189 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:37.087903  8754 raft_consensus.cc:3055] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:37.083073  8708 raft_consensus.cc:697] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:37.085472  8674 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:37.127740  5074 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:37.143625  5189 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:37.129185  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 60128.6us
I20260709 12:24:37.144013  5042 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:37.107355  8798 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.041855  8779 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:37.132225  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.150625  5068 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:37.124708  8786 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.137079  8799 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:37.137758  8414 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.128970  8735 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:37.145867  5947 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:37.130000  5115 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:37.085927  8557 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:24:37.135864  8733 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:37.150501  8802 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:37.142720  8754 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:37.181152  8754 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:37.159675  8708 consensus_queue.cc:237] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:37.181985  5041 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:37.124046  8535 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:37.123793  8682 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:37.179903  8802 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:37.184509  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:37.149938  8098 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:37.150652  8674 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:37.191053  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 61133.9us
I20260709 12:24:37.191967  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:37.166962  8786 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:37.194743  5189 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:37.180087  8801 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:37.150126  8790 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:24:37.184329  8779 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:37.204960  5941 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:37.126784  8807 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:37.178049  8800 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:37.196004  8776 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:37.178113  8697 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:37.154624  8798 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:37.185658  8802 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:37.202473  8400 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:37.227267  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:37.183156  8414 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:37.214954  8754 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:37.181284  8735 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:37.182132  8803 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:24:37.212116  8801 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:37.190474  8557 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:24:37.206189  8810 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:37.258808  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:37.261656  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:37.318059  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:37.260903  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 82043.7us
I20260709 12:24:37.315886  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:37.329339  5218 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:37.303856  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.330200  5222 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:37.331562  5942 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:37.263265  5940 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:37.227144  8813 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:37.334672  5946 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:37.342483  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.343713  5947 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:37.229491  8805 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.338681  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.254022  8535 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:37.257555  8800 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:37.356989  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:37.351576  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:37.354153  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.228343  8776 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:37.360657  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.361692  5940 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:37.257359  8697 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:37.325582  8810 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.264986  8414 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:37.288256  8400 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:37.367027  5067 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:37.225907  8807 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:37.358916  5942 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:37.265353  8821 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:37.294848  8733 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:37.328465  8801 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:37.383512  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.383612  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:37.372869  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 178255us
I20260709 12:24:37.366259  8735 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:37.387684  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:37.377995  8484 consensus_meta.cc:287] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.775s	user 0.017s	sys 0.002s
I20260709 12:24:37.342732  8823 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:37.366328  5071 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:37.257791  8812 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.111s
I20260709 12:24:37.373339  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.319015  8682 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:37.385819  5073 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:37.388520  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:37.397254  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:37.414404  5219 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:37.414865  5222 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:24:37.393520  5042 connection.cc:256] RPC call timeout handler was delayed by -2.3749287669998012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 12:24:37.376031  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 148901us
I20260709 12:24:37.389122  5076 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:37.407378  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:37.416683  5899 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:37.417487  5042 connection.cc:256] RPC call timeout handler was delayed by -2.2048140720003175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
W20260709 12:24:37.418236  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.377655  8810 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:37.388072  8806 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:24:37.418989  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:37.404456  8666 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.001s
I20260709 12:24:37.381685  8400 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:37.419286  5070 raft_consensus.cc:2368] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8.
I20260709 12:24:37.385628  8799 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:37.399188  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 103651us
W20260709 12:24:37.423863  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.382974  8697 raft_consensus.cc:697] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:37.425203  5071 raft_consensus.cc:2410] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:24:37.374878  8525 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:24:37.399549  8647 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:37.391578  8598 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:37.395028  8776 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:37.400776  8752 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:37.426867  8557 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:37.381268  8824 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:24:37.439993  8557 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:37.441304  8807 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:37.443424  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.438313  5899 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:37.450536  5219 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
W20260709 12:24:37.451128  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.368541  8805 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:37.452491  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 269868us
I20260709 12:24:37.414302  8484 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:37.436205  8598 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:37.464323  5074 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:37.465438  5072 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 12:24:37.466656  5042 connection.cc:256] RPC call timeout handler was delayed by -1.9681287670000529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
I20260709 12:24:37.437526  8752 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:37.467993  5042 connection.cc:256] RPC call timeout handler was delayed by -1.9125581319999583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
W20260709 12:24:37.468405  5042 connection.cc:256] RPC call timeout handler was delayed by -1.85222541200028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 12:24:37.468694  5042 connection.cc:256] RPC call timeout handler was delayed by -1.6465434490000916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260709 12:24:37.469084  5042 connection.cc:256] RPC call timeout handler was delayed by -1.0632431680000991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
I20260709 12:24:37.408166  8821 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:37.475342  8821 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:37.389433  8535 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:37.495530  5073 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:37.498493  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.461432  8733 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:37.494383  5041 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:37.509084  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.512764  5073 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
W20260709 12:24:37.481709  8765 consensus_meta.cc:287] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.840s	user 0.010s	sys 0.003s
I20260709 12:24:37.438433  8647 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:37.510890  5070 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:24:37.514901  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.516498  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.467197  8598 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:37.455919  8827 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:37.466990  8682 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:37.469977  8832 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:37.538044  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.442489  8799 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:37.550247  5073 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:24:37.554955  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 86169.2us
I20260709 12:24:37.514875  8733 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:37.513178  8798 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:37.562314  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.470930  8829 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:24:37.534931  8826 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:37.555877  8833 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:24:37.566861  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:37.527068  8827 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:37.497080  8802 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:37.566437  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:37.577845  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 83565.6us
I20260709 12:24:37.547232  8682 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:37.579275  8834 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:37.580343  8800 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:37.560185  8557 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:37.581385  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.563887  8733 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:37.590507  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:37.593467  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 140260us
I20260709 12:24:37.592840  5114 leader_election.cc:400] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8.
I20260709 12:24:37.596901  5114 leader_election.cc:403] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:24:37.598832  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:37.522676  8765 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:37.602454  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.603240  5946 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:37.580607  8682 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:37.601945  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.576174  8827 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:37.611150  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.569809  8807 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:24:37.555239  8832 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:37.620330  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.615790  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.621879  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:37.622218  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.622097  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:37.622777  5219 raft_consensus.cc:2393] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 13: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:37.551915  8697 consensus_queue.cc:237] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:37.623538  5075 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 8: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:37.623669  5940 raft_consensus.cc:2410] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:37.626505  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.635818  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.644865  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:37.643868  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:37.618021  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.648870  5942 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:37.646150  5222 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:37.651374  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:37.652539  5943 raft_consensus.cc:2410] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 12:24:37.653149  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 73867.1us
I20260709 12:24:37.600625  8837 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:24:37.654175  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.654729  5041 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:37.655103  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.615022  8840 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.575974  8752 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:37.660679  5041 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:37.665120  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.666608  5940 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:37.671783  5041 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:37.615911  8838 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:24:37.673075  8484 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:37.674916  5041 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:37.676290  8834 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:37.618872  8807 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 12:24:37.681211  8835 consensus_queue.cc:1175] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" responder_term: 12 status { last_received { term: 12 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 12 index: 7 } }
I20260709 12:24:37.681349  8839 raft_consensus.cc:3055] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:37.713696  8839 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:37.669924  8666 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:37.685438  5189 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:37.686193  8814 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:37.686890  8818 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:37.687358  8832 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:37.686209  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:37.688925  5190 leader_election.cc:336] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.966s (SENT)
I20260709 12:24:37.689901  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.755798  5941 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:37.691000  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:37.757203  5940 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:37.676795  8647 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:37.691267  5947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:37.765936  5947 raft_consensus.cc:606] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:24:37.766353  5947 raft_consensus.cc:614] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:37.696552  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.767218  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.710664  5223 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:37.683683  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:37.704021  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:37.710315  5188 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:37.757375  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.770171  5942 raft_consensus.cc:2376] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13. Re-sending same reply.
I20260709 12:24:37.770665  5948 raft_consensus.cc:2410] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:24:37.755357  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 66485.6us
I20260709 12:24:37.680380  8826 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:37.699184  8834 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:37.773226  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 69292.2us
W20260709 12:24:37.724872  8833 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 12:24:37.780855  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 95516us
W20260709 12:24:37.783154  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 73044.2us
I20260709 12:24:37.674903  8674 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:37.703773  8805 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:37.787424  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.788282  5070 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
W20260709 12:24:37.793889  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 201277us
W20260709 12:24:37.795818  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.758663  8484 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:37.801092  5898 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:37.810679  5943 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 6.
I20260709 12:24:37.767366  8666 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:37.665117  8840 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:37.755148  8847 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:37.778591  8834 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:37.816399  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:37.781448  8798 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:37.821671  5070 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:24:37.820760  8789 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.002s
W20260709 12:24:37.826083  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 204097us
I20260709 12:24:37.840749  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:37.829406  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 150517us
I20260709 12:24:37.803938  8484 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:37.822609  5188 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:37.816828  8779 consensus_meta.cc:287] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.593s	user 0.012s	sys 0.001s
I20260709 12:24:37.795047  8647 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:37.841888  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:37.822940  8800 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:37.848083  5076 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:37.837370  8799 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:37.814664  8805 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:24:37.911331  5943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:37.880743  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.859130  5188 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:37.919073  5943 raft_consensus.cc:606] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:37.858914  5189 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:37.914914  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:37.860450  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 59456.7us
I20260709 12:24:37.920305  5943 raft_consensus.cc:614] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:37.847854  8647 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:37.920138  8802 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:37.836937  8798 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:37.919772  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 79121.2us
I20260709 12:24:37.922147  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.939692  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:37.946785  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 87984us
I20260709 12:24:37.946941  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:37.947770  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:37.945479  5898 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:37.954813  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:37.955461  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.841485  8844 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:37.979097  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:37.782280  8839 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:37.912000  8779 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:37.915194  8855 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:24:37.987039  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 60005.2us
I20260709 12:24:37.848598  8800 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.935456  8647 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:37.947983  8798 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:37.919445  8830 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:37.920089  8765 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:38.013180  8800 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:37.990618  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:38.022195  5940 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:38.023277  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:38.024168  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:37.999552  8535 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:37.939951  8829 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:38.005206  8697 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.843150  8789 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:38.006997  8807 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:24:38.007638  8842 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 12:24:37.963522  8838 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:38.010107  8802 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:24:38.010277  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 151258us
I20260709 12:24:37.989647  5041 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:37.937100  8850 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:24:38.014233  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 66564us
I20260709 12:24:38.023226  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:38.047338  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 57822.8us
I20260709 12:24:38.035130  5070 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:24:37.988731  8855 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.043845  5943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:38.056190  5943 raft_consensus.cc:606] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:24:38.017643  8779 raft_consensus.cc:697] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:38.048628  8851 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:24:38.050925  5940 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:38.034942  8829 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.036069  8777 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:38.050297  5189 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:38.044521  8839 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:37.964475  8484 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:38.042560  8807 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:38.068672  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 113973us
I20260709 12:24:38.070763  5073 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:38.055430  8845 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:38.046424  8802 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:38.087513  8802 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:38.087637  8856 log.cc:927] Time spent T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:24:38.067860  8777 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:38.081696  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.116196  8777 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:38.084438  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:38.117738  5945 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:38.057128  8852 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:38.039590  8697 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:38.069427  8829 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:38.077318  8807 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:38.067242  8850 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:38.094323  8858 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 12:24:38.094664  8848 log.cc:927] Time spent T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:24:38.063555  8855 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:38.084534  8845 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:38.074914  8862 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 12:24:38.069434  8525 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:24:38.213284  8814 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.204324  8850 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:38.214118  8814 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:38.214712  8525 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:38.215232  8525 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:38.215626  8525 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:38.216910  8814 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:38.099611  8789 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:38.100297  8674 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:38.101256  8598 consensus_meta.cc:287] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.000s
I20260709 12:24:38.088519  8854 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:38.100404  8840 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:38.102836  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:38.218817  8598 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:38.219669  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 116940us
I20260709 12:24:38.219969  8825 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:38.220484  8825 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:38.103531  8786 consensus_meta.cc:287] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.896s	user 0.009s	sys 0.002s
I20260709 12:24:38.220880  8825 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:38.105690  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:24:38.107187  8400 consensus_meta.cc:287] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.657s	user 0.013s	sys 0.000s
I20260709 12:24:38.178550  8557 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.108453  8484 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:38.105262  8827 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:38.221618  8400 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:38.111521  8733 consensus_meta.cc:287] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.000s
I20260709 12:24:38.223511  8733 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:38.223699  8827 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:38.224797  5939 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:38.113068  8830 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:38.114884  8800 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:38.077762  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:38.226718  8840 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:38.227464  8800 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:38.228350  5944 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:38.228843  8735 consensus_meta.cc:287] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.818s	user 0.009s	sys 0.000s
W20260709 12:24:38.169297  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 119015us
W20260709 12:24:38.183765  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 172544us
W20260709 12:24:38.205438  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.199985  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:38.187937  8788 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.050s	user 0.009s	sys 0.007s
W20260709 12:24:38.193740  8810 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.758s	user 0.011s	sys 0.004s
I20260709 12:24:38.207690  8845 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:38.217677  8838 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.104454  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:38.221120  8786 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:38.223788  8400 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:38.263628  5221 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:38.094558  8779 consensus_queue.cc:237] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:38.011667  8666 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:38.267105  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 667) took 1050 ms. Trace:
W20260709 12:24:38.105186  8823 consensus_meta.cc:287] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.681s	user 0.006s	sys 0.002s
I20260709 12:24:38.229816  8496 raft_consensus.cc:3055] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:38.230005  8733 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:38.226663  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 148997us
I20260709 12:24:38.187630  8852 raft_consensus.cc:697] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:38.243784  5945 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:38.221128  8598 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:38.248947  5189 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:38.196537  8802 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:38.268220  8098 raft_consensus.cc:3055] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:38.287971  8098 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:38.113554  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.257150  8786 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:38.273394  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 215540us
I20260709 12:24:38.270287  5221 rpcz_store.cc:276] 0709 12:24:37.217242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.217441 (+   199us) service_pool.cc:224] Handling call
0709 12:24:37.217911 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:38.264414 (+1046503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.266745 (+  2331us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54004f2ed4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:38.267015 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2404120960}
I20260709 12:24:38.247520  8844 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:38.297230  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48356.3us
I20260709 12:24:38.289804  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:38.280256  8801 consensus_meta.cc:287] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.867s	user 0.011s	sys 0.002s
I20260709 12:24:38.306306  8812 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:38.278012  8496 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:38.312739  8496 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:38.313880  8496 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:38.315954  8647 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:38.318049  8801 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:38.320387  8801 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:38.321038  8839 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:38.321707  8839 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:38.321563  8798 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:38.321877  5074 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:38.322721  8839 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:38.323793  8798 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:38.323408  8879 consensus_queue.cc:1059] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:38.325047  8098 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:38.326148  5189 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:38.326781  8735 raft_consensus.cc:3055] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:38.327281  8735 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:38.328186  8735 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:38.332568  8525 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:38.321141  5115 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:24:38.353773  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 50657.6us
W20260709 12:24:38.288703  8818 consensus_meta.cc:287] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.003s
I20260709 12:24:38.250819  8788 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:38.250710  8810 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:38.340723  8852 consensus_queue.cc:237] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:38.284915  8823 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:38.266163  8838 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:38.342708  8697 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:38.346511  8844 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:38.332962  8829 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:38.431228  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 268729us
I20260709 12:24:38.433038  5947 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: []
W20260709 12:24:38.433332  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.370604  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 49568.9us
W20260709 12:24:38.434257  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.371255  5945 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:38.369632  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:38.341169  8834 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:38.416950  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 147643us
W20260709 12:24:38.400427  5190 leader_election.cc:336] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.553s (SENT)
W20260709 12:24:38.417501  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 47722.6us
I20260709 12:24:38.417887  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:38.372133  8818 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8.
I20260709 12:24:38.421993  8525 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:38.367146  8777 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:38.329473  8735 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:38.428745  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 102777us
I20260709 12:24:38.436775  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:38.434651  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:38.392275  8400 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:38.472160  5946 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:38.472857  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 72473.8us
I20260709 12:24:38.426031  8786 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:38.369414  8821 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
W20260709 12:24:38.482153  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 47617.9us
I20260709 12:24:38.486295  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:38.492055  5900 connection.cc:256] RPC call timeout handler was delayed by -1.0564507980002418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
W20260709 12:24:38.492499  5900 connection.cc:256] RPC call timeout handler was delayed by -1.0383638080002129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
I20260709 12:24:38.493319  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:38.493989  5143 raft_consensus.cc:2376] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10. Re-sending same reply.
I20260709 12:24:38.498977  5220 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:24:38.507328  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:38.510396  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:38.511955  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:38.513164  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:38.516498  5218 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:38.525628  5142 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:38.519356  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:38.479471  8400 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:38.392349  8845 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:38.479228  8855 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:38.487134  8818 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:38.565742  5221 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:24:38.554994  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.562382  5216 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
W20260709 12:24:38.799923  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.527814  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:38.766940  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:38.528949  5217 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
W20260709 12:24:38.644132  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.486740  8779 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:24:38.519855  5190 leader_election.cc:336] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.663s (SENT)
I20260709 12:24:38.803562  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:38.804246  5190 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:38.804561  5147 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:38.805910  5223 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:38.505923  8827 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.803494  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:38.810616  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 291827us
W20260709 12:24:38.746459  8889 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:24:38.621135  8881 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 12:24:38.620121  8882 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:24:38.814445  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:38.747202  8839 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.622328  8834 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:38.740808  8777 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:38.820170  5219 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:38.820616  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:38.797134  8400 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:38.821707  5217 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:38.613984  8868 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.814661  8814 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.739116  8829 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:38.826804  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:38.513954  8484 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:38.808269  8818 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:38.830503  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:38.831046  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:38.855304  5221 raft_consensus.cc:2393] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:38.744503  8838 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:38.855891  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:38.834055  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.596916  8647 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:38.836803  5219 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:38.838174  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1255) took 1441 ms. Trace:
W20260709 12:24:38.839129  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.814500  8783 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:38.529034  8807 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:38.834885  8779 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:38.740386  8825 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:38.817639  8827 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:38.833348  8839 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:38.929831  8496 consensus_meta.cc:287] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.004s
I20260709 12:24:38.898805  8798 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:39.049801  5219 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:39.040943  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:38.931005  5071 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:38.864282  5945 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:38.929840  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.049974  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.054240  5147 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:39.009140  5943 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:39.057765  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:39.008718  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 911) took 1381 ms. Trace:
W20260709 12:24:38.954038  5075 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 1265) took 1847 ms (client timeout 1533 ms). Trace:
W20260709 12:24:39.059331  5075 rpcz_store.cc:269] 0709 12:24:37.083525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.083803 (+   278us) service_pool.cc:224] Handling call
0709 12:24:38.930518 (+1846715us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b5400589654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:38.930707 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1139208576}
W20260709 12:24:38.982833  5190 leader_election.cc:336] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.664s (SENT)
I20260709 12:24:39.060920  5190 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:39.062641  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:39.063462  5217 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.064999  5224 rpcz_store.cc:276] 0709 12:24:37.397077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.397208 (+   131us) service_pool.cc:224] Handling call
0709 12:24:37.399608 (+  2400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.555938 (+156330us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:37.607621 (+ 51683us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.648266 (+ 40645us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:37.648337 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:37.648381 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.837818 (+1189437us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54004835d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:38.837892 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.838098 (+   206us) inbound_call.cc:177] Queueing success response
0709 12:24:39.051946 (+213848us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b18001770a0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
Metrics: {"prepare.queue_time_us":132367,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":182,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"spinlock_wait_cycles":382542976,"thread_start_us":88228,"threads_started":2,"wal-append.queue_time_us":69182}
I20260709 12:24:39.058676  5939 rpcz_store.cc:276] 0709 12:24:37.627195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.654792 (+ 27597us) service_pool.cc:224] Handling call
0709 12:24:39.008463 (+1353671us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b5400006254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.008647 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1308193664}
I20260709 12:24:39.073390  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:39.073910  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 91134.2us
W20260709 12:24:39.075063  5190 leader_election.cc:336] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.711s (SENT)
W20260709 12:24:39.076083  5190 leader_election.cc:336] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.589s (SENT)
W20260709 12:24:39.077050  5190 leader_election.cc:336] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.954s (SENT)
I20260709 12:24:39.077502  5190 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:38.954222  8895 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 12:24:38.976789  8890 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.327s	user 0.002s	sys 0.000s
W20260709 12:24:38.962329  8882 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:24:39.083983  8647 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:39.085041  8838 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:39.079366  8733 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.095099  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 771) took 1625 ms. Trace:
I20260709 12:24:39.105885  5074 rpcz_store.cc:276] 0709 12:24:37.469825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.470011 (+   186us) service_pool.cc:224] Handling call
0709 12:24:37.470389 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.322676 (+852287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.094825 (+772149us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b5400328a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:39.095032 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957460224}
I20260709 12:24:39.107599  5219 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:39.086401  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.085486  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:39.100930  5144 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:39.093313  8735 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:24:39.103457  5221 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:39.111557  5945 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:39.114933  5939 raft_consensus.cc:2393] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:39.103988  5217 raft_consensus.cc:2410] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:24:39.052516  8825 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:39.005266  8855 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:39.033520  8788 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:38.865392  8818 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.101706  8496 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:39.123360  5150 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:39.124202  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:39.083957  8863 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:24:39.129724  5221 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:38.847155  8814 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:39.112766  8900 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.161s	user 0.000s	sys 0.009s
I20260709 12:24:39.133656  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.131453  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 872) took 1032 ms. Trace:
I20260709 12:24:38.843999  8868 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:39.135355  5190 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.014s (SENT)
I20260709 12:24:39.136165  5939 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:39.137861  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:39.070484  8798 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:39.136891  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 871) took 1053 ms. Trace:
I20260709 12:24:39.140501  5939 rpcz_store.cc:276] 0709 12:24:38.083879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.128362 (+1044483us) service_pool.cc:224] Handling call
0709 12:24:39.128689 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.136663 (+  7974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.136807 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:39.134265  5945 rpcz_store.cc:276] 0709 12:24:38.098939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.130786 (+1031847us) service_pool.cc:224] Handling call
0709 12:24:39.131088 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.131219 (+   131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.131233 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:39.131261 (+    28us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:39.131279 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.131310 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.131399 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:39.114773  8525 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:24:39.145346  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 339472us
I20260709 12:24:39.148726  8779 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:39.152765  8884 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:39.154434  8852 raft_consensus.cc:3055] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:39.154588  8839 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:39.155040  8852 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:39.156081  8852 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:39.105165  8791 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:24:39.158547  8791 raft_consensus.cc:3055] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:39.159014  8791 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:39.159348  5944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 907) took 1534 ms (client timeout 1525 ms). Trace:
I20260709 12:24:39.160020  8791 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:39.164085  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.165223  5945 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:39.168852  5939 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
W20260709 12:24:39.170008  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.159847  5944 rpcz_store.cc:269] 0709 12:24:37.624796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.625079 (+   283us) service_pool.cc:224] Handling call
0709 12:24:39.159080 (+1534001us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b5400003554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.159260 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1318376704}
I20260709 12:24:39.124456  8862 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:24:39.177675  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 873) took 1078 ms. Trace:
I20260709 12:24:39.178181  5939 rpcz_store.cc:276] 0709 12:24:38.099260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.141446 (+1042186us) service_pool.cc:224] Handling call
0709 12:24:39.165149 (+ 23703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.177287 (+ 12138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.177597 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:39.166551  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:39.180742  5223 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:39.185401  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.187135  5147 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:39.119709  8733 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:39.187654  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.197209  5219 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:39.188664  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.199561  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 918) took 1515 ms. Trace:
I20260709 12:24:39.199491  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:39.200991  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:39.201745  5219 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:39.204787  8841 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 1.544s	user 0.005s	sys 0.000s
I20260709 12:24:39.207463  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1082) took 1892 ms. Trace:
I20260709 12:24:39.207904  5215 rpcz_store.cc:276] 0709 12:24:37.314990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.315222 (+   232us) service_pool.cc:224] Handling call
0709 12:24:37.342069 (+ 26847us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:37.364428 (+ 22359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.364459 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:37.367292 (+  2833us) log.cc:844] Serialized 52 byte log entry
0709 12:24:37.537983 (+170691us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:37.538055 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.538170 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.204042 (+1665872us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:39.207099 (+  3057us) raft_consensus.cc:1685] finished
0709 12:24:39.207123 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.207390 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7869,"raft.queue_time_us":23972,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":24638,"spinlock_wait_cycles":3699968,"thread_start_us":123945,"threads_started":1,"wal-append.queue_time_us":190575}
I20260709 12:24:39.211556  5215 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:39.214363  5221 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:39.200096  5941 rpcz_store.cc:276] 0709 12:24:37.683815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.777601 (+ 93786us) service_pool.cc:224] Handling call
0709 12:24:37.802736 (+ 25135us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.803075 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.803099 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:38.602517 (+799418us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b2c00152dc8. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:24:38.602939 (+   422us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.810273 (+207334us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:38.820273 (+ 10000us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.827709 (+  7436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.827821 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.195010 (+367189us) raft_consensus.cc:1685] finished
0709 12:24:39.199305 (+  4295us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.199481 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12427,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"raft.queue_time_us":141755,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":46869,"spinlock_wait_cycles":2779520,"thread_start_us":206868,"threads_started":1,"wal-append.queue_time_us":344414}
I20260709 12:24:39.218173  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.219197  5141 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:39.176936  8791 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:39.219857  5219 raft_consensus.cc:2410] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:24:39.147926  8788 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:24:39.225065  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.184137  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:39.232183  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 66096.2us
I20260709 12:24:39.232831  5215 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:39.234383  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.241745  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.225818  8717 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.244192  8717 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:39.245692  5218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35624 (request call id 1253) took 1849 ms (client timeout 2190 ms). Trace:
W20260709 12:24:39.246243  5218 rpcz_store.cc:269] 0709 12:24:37.396515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.407138 (+ 10623us) service_pool.cc:224] Handling call
0709 12:24:39.245471 (+1838333us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54003fe8d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.245621 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2357818240}
I20260709 12:24:39.246762  5071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 773) took 1775 ms. Trace:
I20260709 12:24:39.247256  5071 rpcz_store.cc:276] 0709 12:24:37.471725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.471868 (+   143us) service_pool.cc:224] Handling call
0709 12:24:38.930803 (+1458935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.053326 (+122523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.246531 (+193205us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400589650. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.246681 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3354414976}
W20260709 12:24:39.249650  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.251297  5189 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:39.251852  8829 consensus_queue.cc:1059] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:39.252542  5189 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:39.252777  8829 raft_consensus.cc:3055] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:39.253178  8829 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:39.253723  8647 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:39.253588  8400 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:39.254006  8829 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:39.254899  5223 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:39.255422  8829 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:39.257289  8733 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:39.257867  8525 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:39.258687  8525 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:39.259302  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:39.259979  8525 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:39.260025  5945 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.262468  8798 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:39.264376  5944 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:39.269166  8682 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:39.269739  8682 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:39.274171  5189 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:39.275689  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:39.276309  5944 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:39.278265  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.278966  5141 raft_consensus.cc:2376] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11. Re-sending same reply.
I20260709 12:24:39.276271  5945 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
W20260709 12:24:39.293118  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 175198us
I20260709 12:24:39.295442  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:39.295894  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:39.296847  5939 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.236521  8666 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:39.300309  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:39.300545  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.301173  5941 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:39.301855  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:39.299583  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.304121  5944 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:39.306838  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:39.307574  5216 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:39.308071  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:39.306685  5189 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:39.272970  8845 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:39.288515  8777 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:39.270565  8682 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:39.299747  8844 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.323654  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:39.324216  5073 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 784) took 1525 ms (client timeout 1717 ms). Trace:
I20260709 12:24:39.324576  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.331260  5141 raft_consensus.cc:2376] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11. Re-sending same reply.
I20260709 12:24:39.332365  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.333068  5141 raft_consensus.cc:2376] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8. Re-sending same reply.
W20260709 12:24:39.325790  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.313095  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:39.335297  5074 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:39.330897  5073 rpcz_store.cc:269] 0709 12:24:37.799174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.010687 (+211513us) service_pool.cc:224] Handling call
0709 12:24:39.323989 (+1313302us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b5400327654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.324140 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76725120}
I20260709 12:24:39.336163  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:39.348467  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.351332  5141 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:39.352365  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.291433  8839 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:24:39.353446  5141 raft_consensus.cc:2410] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:24:39.347718  5941 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:39.363842  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.365265  5141 raft_consensus.cc:2410] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:24:39.362768  5216 raft_consensus.cc:2410] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:24:39.309131  8733 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
W20260709 12:24:39.369956  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 126398us
I20260709 12:24:39.371742  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:39.376752  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 70173.4us
W20260709 12:24:39.377609  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.378768  5189 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:39.329399  8525 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:39.373986  5141 raft_consensus.cc:2376] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8. Re-sending same reply.
I20260709 12:24:39.338470  8807 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:39.339316  8827 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:39.398792  8892 log.cc:927] Time spent T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.908s	user 0.002s	sys 0.000s
I20260709 12:24:39.399065  8818 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:39.368357  8682 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.392529  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:39.406317  8825 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:39.408457  8825 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.412561  8752 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:39.382055  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.383867  8910 consensus_queue.cc:1059] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:39.352043  8862 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:39.362782  8598 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:39.388314  5074 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
W20260709 12:24:39.391215  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.391863  5900 leader_election.cc:336] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.506s (SENT)
I20260709 12:24:39.417207  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:39.418649  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:39.430104  8912 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:39.431063  8912 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:39.432180  8852 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:39.432693  8647 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:39.435199  8868 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:39.438314  8098 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:39.414161  8845 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:39.442168  8837 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:39.442862  8837 raft_consensus.cc:697] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:39.443218  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:39.443922  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52095.5us
I20260709 12:24:39.444758  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.446146  8907 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.447378  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.448668  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.446046  8827 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:39.446606  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 904) took 1831 ms. Trace:
I20260709 12:24:39.450377  5946 rpcz_store.cc:276] 0709 12:24:37.615270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.615429 (+   159us) service_pool.cc:224] Handling call
0709 12:24:39.446384 (+1830955us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b54002427d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.446530 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1945871744}
I20260709 12:24:39.447145  8907 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:39.451862  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:39.461917  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.456204  5940 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:39.467432  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.350386  8844 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:39.409174  8825 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:39.477632  5939 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:39.466480  8845 raft_consensus.cc:697] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:39.479236  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.491320  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:39.493510  5148 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:39.480948  8647 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:39.482920  5190 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.763s (SENT) [suppressed 53 similar messages]
I20260709 12:24:39.483944  8717 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:39.457067  8862 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:39.417893  8525 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:39.486066  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.499733  5141 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:39.509789  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.512580  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.518512  5142 raft_consensus.cc:2410] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:24:39.512681  8779 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:39.525899  8525 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:39.526360  8779 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:39.528899  8496 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.529922  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.530115  8818 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:39.530676  5148 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:39.531498  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.532018  5148 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:39.508249  8803 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:39.534292  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.539067  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.542275  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.546631  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.543999  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:39.571990  5900 leader_election.cc:336] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.848s (SENT)
I20260709 12:24:39.572718  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:39.672298  5148 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:39.676047  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:39.677825  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 105891us
W20260709 12:24:39.679085  5900 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.877s (SENT)
W20260709 12:24:39.685058  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.692108  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.693708  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:39.688112  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 272212us
I20260709 12:24:39.523720  8717 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.443956  8837 consensus_queue.cc:237] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:39.696815  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 213923us
W20260709 12:24:39.701442  5190 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.199s (SENT)
I20260709 12:24:39.494951  8845 consensus_queue.cc:237] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:39.687763  8803 raft_consensus.cc:697] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:39.702687  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:39.671097  8839 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.723690  5216 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:39.529475  8496 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:39.689291  8879 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.708346  5142 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
W20260709 12:24:39.721197  5076 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 1041) took 2326 ms (client timeout 2744 ms). Trace:
I20260709 12:24:39.698793  8844 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:39.690374  8791 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:39.722224  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:39.727967  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:39.728821  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.715282  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 920) took 2030 ms. Trace:
W20260709 12:24:39.727425  5076 rpcz_store.cc:269] 0709 12:24:37.386843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.426559 (+ 39716us) service_pool.cc:224] Handling call
0709 12:24:39.713217 (+2286658us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400328554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.713339 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":923823488}
I20260709 12:24:39.711781  8824 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.723527  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:39.728881  5142 raft_consensus.cc:2376] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12. Re-sending same reply.
I20260709 12:24:39.726141  5074 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
W20260709 12:24:39.729969  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 218873us
I20260709 12:24:39.725677  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:39.737102  5073 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:39.737133  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.738222  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:39.729244  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:39.732852  5190 leader_election.cc:336] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.018s (SENT)
I20260709 12:24:39.733924  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:39.734616  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:39.739614  5071 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:39.739672  5224 raft_consensus.cc:2376] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10. Re-sending same reply.
I20260709 12:24:39.729432  5942 rpcz_store.cc:276] 0709 12:24:37.684416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.772264 (+ 87848us) service_pool.cc:224] Handling call
0709 12:24:37.824327 (+ 52063us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.940790 (+116463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.940820 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:37.946770 (+  5950us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.168701 (+221931us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:38.169523 (+   822us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.170169 (+   646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.170276 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:38.224709 (+ 54433us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b4800408258. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ad464aa1 00007fd8ad45bb37 00007fd8ad45b9ca 00007fd8ad55326f 00007fd8ad552f37 00007fd8ad56755c 00007fd8ad567508 00007fd8ad567488 00007fd8ad567443 00007fd8ad5666ec 00007fd8ac490a96
0709 12:24:39.133521 (+908812us) raft_consensus.cc:1685] finished
0709 12:24:39.138609 (+  5088us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.715097 (+576488us) spinlock_profiling.cc:243] Waited 51.7 ms on lock 0x7b54002422d0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.715210 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":166,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":121827,"raft.run_cpu_time_us":4336,"raft.run_wall_time_us":273207,"spinlock_wait_cycles":145870336,"thread_start_us":298629,"threads_started":2,"wal-append.queue_time_us":316701}
I20260709 12:24:39.738226  5142 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:39.743202  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:39.746729  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:39.747049  5946 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:24:39.747072  5141 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:39.750087  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:39.751281  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.752113  5142 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 14: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:39.723870  8735 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.752521  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1242) took 1239 ms. Trace:
I20260709 12:24:39.752837  5142 rpcz_store.cc:276] 0709 12:24:38.512673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.751007 (+1238334us) service_pool.cc:224] Handling call
0709 12:24:39.752468 (+  1461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:39.754297  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.755322  5142 raft_consensus.cc:2376] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9. Re-sending same reply.
I20260709 12:24:39.756214  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.758967  5948 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: []
W20260709 12:24:39.756991  5190 leader_election.cc:336] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.102s (SENT)
W20260709 12:24:39.760320  5190 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.194s (SENT)
I20260709 12:24:39.752959  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:39.761293  5190 leader_election.cc:336] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.280s (SENT)
W20260709 12:24:39.761966  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.763267  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:39.763630  8829 consensus_meta.cc:287] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.508s	user 0.014s	sys 0.004s
I20260709 12:24:39.766404  5942 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:39.743528  8827 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:24:39.730870  8839 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:39.731349  8879 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:39.772907  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:39.709502  8717 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:39.735878  8708 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:39.721632  8862 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:39.764192  8913 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:39.785035  8783 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.810782  5142 raft_consensus.cc:2410] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:24:39.821235  5148 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:39.743744  8824 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:39.829687  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.764430  8735 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:39.824960  8907 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:39.834944  8825 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:39.850971  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:39.876371  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.876601  5146 raft_consensus.cc:2376] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10. Re-sending same reply.
I20260709 12:24:39.877149  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 686) took 1384 ms. Trace:
I20260709 12:24:39.877493  5146 rpcz_store.cc:276] 0709 12:24:38.492339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.492471 (+   132us) service_pool.cc:224] Handling call
0709 12:24:39.850807 (+1358336us) spinlock_profiling.cc:243] Waited 56.6 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:24:39.877091 (+ 26284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130099072}
I20260709 12:24:39.878495  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.851996  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 869) took 1778 ms. Trace:
I20260709 12:24:39.854043  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 688) took 1361 ms. Trace:
I20260709 12:24:39.879597  5150 rpcz_store.cc:276] 0709 12:24:38.492942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.497755 (+  4813us) service_pool.cc:224] Handling call
0709 12:24:39.853861 (+1356106us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400265054. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.853964 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554974464}
I20260709 12:24:39.854873  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.881052  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 689) took 1375 ms. Trace:
I20260709 12:24:39.881402  5143 rpcz_store.cc:276] 0709 12:24:38.505897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.509437 (+  3540us) service_pool.cc:224] Handling call
0709 12:24:39.854728 (+1345291us) spinlock_profiling.cc:243] Waited 9.23 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:24:39.880989 (+ 26261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21222656}
I20260709 12:24:39.882483  5190 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:39.885386  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:39.855446  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.879022  5146 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:39.856735  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.879534  5943 rpcz_store.cc:276] 0709 12:24:38.073256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.469347 (+396091us) service_pool.cc:224] Handling call
0709 12:24:39.008938 (+539591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.058174 (+ 49236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.851789 (+793615us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400006250. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.851935 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239704960}
I20260709 12:24:39.861922  8814 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:39.863547  8735 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:39.863790  8810 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:39.864977  5900 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:39.865231  8901 raft_consensus.cc:3055] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:39.911062  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:39.866132  8802 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:39.866437  5217 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:24:39.867609  8777 consensus_meta.cc:287] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.000s
I20260709 12:24:39.823339  8783 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:39.858489  8879 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:39.873131  5944 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:39.966856  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 862) took 1978 ms. Trace:
I20260709 12:24:39.967406  5947 rpcz_store.cc:276] 0709 12:24:37.989066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.989385 (+   319us) service_pool.cc:224] Handling call
0709 12:24:38.009980 (+ 20595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.433747 (+423767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.966480 (+1532733us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b540000a354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:39.966757 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972296448}
I20260709 12:24:39.968025  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:39.968701  5944 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:39.873769  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:39.874495  8557 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:39.875299  8824 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:39.888473  5145 raft_consensus.cc:2393] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:39.971654  5223 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:24:39.972038  5145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1249) took 1457 ms (client timeout 1580 ms). Trace:
W20260709 12:24:39.972391  5145 rpcz_store.cc:269] 0709 12:24:38.514947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.610258 (+ 95311us) service_pool.cc:224] Handling call
0709 12:24:39.856573 (+1246315us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:24:39.971978 (+115405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44139264}
W20260709 12:24:39.858508  5067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 1039) took 2472 ms (client timeout 1658 ms). Trace:
W20260709 12:24:39.973721  5067 rpcz_store.cc:269] 0709 12:24:37.386072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.387389 (+  1317us) service_pool.cc:224] Handling call
0709 12:24:39.858304 (+2470915us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540040f954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.858435 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376974592}
I20260709 12:24:39.900969  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:39.975230  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 74409.9us
I20260709 12:24:39.970980  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:39.976681  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:39.822378  8803 consensus_queue.cc:237] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:39.898747  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:39.977401  5947 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:24:39.978370  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 79789.1us
I20260709 12:24:39.841917  8647 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:39.903108  5946 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:39.980513  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:39.981566  5944 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:39.981735  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:39.889627  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:39.865087  8717 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:39.907156  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.911064  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 46175.9us
I20260709 12:24:39.985700  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:39.987191  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:39.919324  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:39.919339  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:39.946816  8783 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:39.872355  5941 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:39.925015  8913 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:39.918876  8799 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:39.953622  8912 raft_consensus.cc:3055] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:39.902612  8525 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:39.907047  8905 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.941985  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:39.961580  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:39.910285  8901 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:39.928196  5217 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:24:39.960407  8852 consensus_meta.cc:287] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.508s	user 0.004s	sys 0.008s
I20260709 12:24:39.971086  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.013386  5945 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:39.970973  8557 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:39.982839  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.001068  8834 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.987846  5948 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
W20260709 12:24:40.007613  8666 consensus_meta.cc:287] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.701s	user 0.007s	sys 0.006s
I20260709 12:24:39.985704  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.020521  5947 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:39.991037  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:39.992827  5222 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:39.960186  8777 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:39.993230  8098 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:39.993579  5900 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:24:39.996135  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.996790  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:40.044839  8098 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:39.997180  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1229) took 1488 ms. Trace:
I20260709 12:24:39.999073  5943 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:39.998546  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:39.971109  8802 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:39.984880  8824 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:39.985127  8879 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:40.008041  5143 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:40.000272  8912 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:40.002590  8783 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:39.973454  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:40.000399  8868 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.026247  5945 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:40.000792  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:40.055981  5069 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:40.045560  5142 rpcz_store.cc:276] 0709 12:24:38.508911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.755975 (+1247064us) service_pool.cc:224] Handling call
0709 12:24:39.996997 (+241022us) spinlock_profiling.cc:243] Waited 52.9 ms on lock 0x7b5400239954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:39.997125 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121756544}
I20260709 12:24:40.057214  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:40.004693  8930 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:40.050338  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 56850.9us
I20260709 12:24:40.052285  8098 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:40.059190  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:40.055538  5145 raft_consensus.cc:2376] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10. Re-sending same reply.
I20260709 12:24:40.060613  5900 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:40.010004  8839 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:40.044659  8666 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:40.058754  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:40.070250  5142 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 13: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:40.060990  5945 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:40.069427  5948 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:40.069589  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:40.072741  5143 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:40.060956  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:40.075048  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:40.077451  5142 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:40.015295  8905 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:40.081418  5217 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:40.081831  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:40.082451  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:40.056475  8810 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:40.082106  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:40.085686  5219 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:40.085940  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:40.045936  8834 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:40.090996  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:40.091516  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1123) took 1285 ms. Trace:
I20260709 12:24:40.092067  5141 rpcz_store.cc:276] 0709 12:24:38.806076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.543674 (+737598us) service_pool.cc:224] Handling call
0709 12:24:40.091333 (+547659us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54003ae3d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:40.091453 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452870528}
I20260709 12:24:40.093052  5221 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:40.093780  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 683) took 1602 ms. Trace:
I20260709 12:24:40.094211  5144 rpcz_store.cc:276] 0709 12:24:38.491490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.491634 (+   144us) service_pool.cc:224] Handling call
0709 12:24:38.491997 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.109665 (+617668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.093516 (+983851us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b540022ea54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:40.093722 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1399975808}
I20260709 12:24:40.094556  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 865) took 2067 ms. Trace:
I20260709 12:24:40.095343  5940 rpcz_store.cc:276] 0709 12:24:38.026713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.070040 (+ 43327us) service_pool.cc:224] Handling call
0709 12:24:39.449632 (+1379592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.466794 (+ 17162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.094282 (+627488us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002427d0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:40.094471 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3171719040}
I20260709 12:24:40.093118  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:40.087598  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:40.087900  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:40.101928  5142 raft_consensus.cc:2376] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 5. Re-sending same reply.
I20260709 12:24:40.105396  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:40.104270  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:40.076264  8827 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:40.054131  8802 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:40.107436  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:40.070415  8779 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:40.112920  5217 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:40.109342  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:40.254518  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1228) took 1745 ms. Trace:
I20260709 12:24:40.254730  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:40.254906  5141 rpcz_store.cc:276] 0709 12:24:38.508668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.109127 (+1600459us) service_pool.cc:224] Handling call
0709 12:24:40.254445 (+145318us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:40.256547  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.104380  5068 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:40.106918  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:40.110692  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 104298us
I20260709 12:24:40.260469  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:40.260703  5075 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:40.113771  5148 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:40.109058  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:40.144539  5188 connection.cc:256] RPC call timeout handler was delayed by -1.6823804730001939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
I20260709 12:24:40.082006  8845 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.066005  8868 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:40.155566  5143 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:40.263998  5188 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:40.155033  5219 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:40.156133  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:40.155110  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:40.266198  5220 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:40.266630  5069 raft_consensus.cc:2410] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:40.160089  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:40.161772  5073 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:40.142189  8098 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:40.165019  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:40.161322  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.087121  8496 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:40.077653  8852 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:24:40.182507  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:40.107920  8810 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:40.069120  8930 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:40.188391  8935 log.cc:927] Time spent T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:24:40.282481  5220 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:40.192289  8905 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:40.198735  8557 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:40.199470  8802 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:40.154999  8837 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:24:40.302381  8868 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:40.180559  8926 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:24:40.174214  8932 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:24:40.205928  8834 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:40.165180  8910 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:24:40.306654  8930 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:40.306926  8910 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:24:40.278409  8943 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:24:40.307629  8910 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:40.065744  8901 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:40.310392  8945 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:40.311111  8945 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:40.312327  8945 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:40.279999  8647 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:40.277815  8916 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.314955  8916 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:40.088038  8907 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 12:24:40.317404  8907 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:40.317941  8907 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:40.318955  8907 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:40.320420  8907 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:40.087711  8917 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:24:40.166889  8843 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.324271  8843 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:40.270372  8845 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:40.327044  8843 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:40.220160  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:40.328230  5149 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:24:40.328783  5149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 682) took 1837 ms (client timeout 1506 ms). Trace:
I20260709 12:24:40.230448  8863 raft_consensus.cc:3055] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:40.232121  8791 raft_consensus.cc:3055] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:40.237820  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.238471  5900 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:40.242451  5215 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:40.246248  8862 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:40.308741  8910 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:40.275641  8934 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.178s	user 0.000s	sys 0.005s
I20260709 12:24:40.251113  8879 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:40.329370  8845 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:40.329785  8791 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:40.352295  8791 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:40.118402  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:40.077814  8829 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:24:40.263628  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.158494  8779 consensus_queue.cc:1059] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:40.266239  5219 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:40.267268  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1225) took 1753 ms. Trace:
I20260709 12:24:40.267750  5143 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
W20260709 12:24:40.263839  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.274423  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1252) took 1758 ms. Trace:
I20260709 12:24:40.277264  5142 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: [7.4-10.5]
W20260709 12:24:40.271992  8938 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.051s	user 0.003s	sys 0.000s
W20260709 12:24:40.271889  8942 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.052s	user 0.003s	sys 0.000s
I20260709 12:24:40.291779  8810 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:40.269608  8815 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:40.267372  8819 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:24:40.301708  8708 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.271119  8788 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:24:40.271726  8939 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.065s	user 0.000s	sys 0.004s
I20260709 12:24:40.250761  5222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1269) took 1387 ms. Trace:
I20260709 12:24:40.329639  8863 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:40.270920  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:40.353094  8827 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:24:40.380861  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 110013us
I20260709 12:24:40.360138  8799 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:40.349789  5150 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:40.382948  5143 pending_rounds.cc:85] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 6
I20260709 12:24:40.383304  5143 pending_rounds.cc:107] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:24:40.383558  5143 raft_consensus.cc:2889] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:40.383764  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:40.384368  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1248) took 1869 ms. Trace:
I20260709 12:24:40.384805  5143 rpcz_store.cc:276] 0709 12:24:38.514678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.264720 (+1750042us) service_pool.cc:224] Handling call
0709 12:24:40.265031 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.384052 (+119021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.384302 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.385351  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:40.342053  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.386281  5143 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:40.391295  5144 rpcz_store.cc:276] 0709 12:24:38.515936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.108762 (+1592826us) service_pool.cc:224] Handling call
0709 12:24:40.274335 (+165573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.392247  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1121) took 1603 ms. Trace:
I20260709 12:24:40.392750  5144 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:24:40.358151  8912 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:40.350687  8945 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:40.394727  8912 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:40.396996  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1118) took 1609 ms. Trace:
I20260709 12:24:40.382103  8799 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:40.399715  8819 raft_consensus.cc:3055] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:40.399503  8799 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:40.400167  8819 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:40.388175  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.388789  5221 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:40.389309  8829 raft_consensus.cc:697] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:40.402365  8829 consensus_queue.cc:237] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:40.403474  8815 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.403869  8815 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:40.404314  8484 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:40.404806  8484 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:40.390062  8708 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:40.355494  5147 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:40.405798  8484 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:40.406213  8863 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:40.406805  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1119) took 1618 ms. Trace:
I20260709 12:24:40.407347  8863 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:40.387307  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:38.491210  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:40.390784  5222 rpcz_store.cc:276] 0709 12:24:38.863749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.908324 (+1044575us) service_pool.cc:224] Handling call
0709 12:24:39.908640 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.025397 (+116757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.250289 (+224892us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b5400273dd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:40.250643 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185908096}
I20260709 12:24:40.410491  8916 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:40.388015  8901 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:40.411626  8708 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:40.354842  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 116526us
I20260709 12:24:40.412330  8834 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:40.412918  5900 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:40.386708  5148 rpcz_store.cc:276] 0709 12:24:38.507821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.113279 (+1605458us) service_pool.cc:224] Handling call
0709 12:24:40.113596 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.261285 (+147689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.261483 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.413926  8484 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:40.414242  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:40.414475  5148 raft_consensus.cc:1240] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:40.415102  8834 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:24:40.329205  5149 rpcz_store.cc:269] 0709 12:24:38.491175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.491410 (+   235us) service_pool.cc:224] Handling call
0709 12:24:40.219904 (+1728494us) spinlock_profiling.cc:243] Waited 56.7 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:24:40.328692 (+108788us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130320640}
I20260709 12:24:40.366393  8400 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:40.416771  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:40.415190  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1126) took 1589 ms. Trace:
I20260709 12:24:40.417670  8930 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:40.397368  5142 rpcz_store.cc:276] 0709 12:24:38.787899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.276496 (+1488597us) service_pool.cc:224] Handling call
0709 12:24:40.276934 (+   438us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:40.396729 (+119795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.396941 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.434806  5142 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:40.435570  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1128) took 1590 ms. Trace:
I20260709 12:24:40.436175  5142 rpcz_store.cc:276] 0709 12:24:38.844946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.434231 (+1589285us) service_pool.cc:224] Handling call
0709 12:24:40.434601 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.435296 (+   695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.435509 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.387231  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:40.418351  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.329838  5224 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:40.415723  8834 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.392640  5143 rpcz_store.cc:276] 0709 12:24:38.788714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.385693 (+1596979us) service_pool.cc:224] Handling call
0709 12:24:40.386061 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.391969 (+  5908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.392186 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.418854  5900 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:40.453603  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.393479  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1122) took 1595 ms. Trace:
I20260709 12:24:40.454758  5144 rpcz_store.cc:276] 0709 12:24:38.798296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.392126 (+1593830us) service_pool.cc:224] Handling call
0709 12:24:40.392498 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.393211 (+   713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.393426 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.398252  8790 raft_consensus.cc:3055] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:40.418982  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.330133  5939 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:24:40.468154  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:40.420603  5071 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:40.420923  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.468359  5143 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:40.469358  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 709) took 1009 ms. Trace:
I20260709 12:24:40.469509  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.420785  5148 rpcz_store.cc:276] 0709 12:24:38.825394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.413954 (+1588560us) service_pool.cc:224] Handling call
0709 12:24:40.414296 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.414919 (+   623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.415131 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.470669  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:40.407145  5147 rpcz_store.cc:276] 0709 12:24:38.788243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.354818 (+1566575us) service_pool.cc:224] Handling call
0709 12:24:40.355252 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.406557 (+ 51305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.406750 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.472048  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.472093  5148 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:24:40.474032  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 55257us
I20260709 12:24:40.457638  5142 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:40.417785  8788 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:40.475104  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.355305  8905 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:40.353437  8791 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:40.331570  8910 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:40.470603  5074 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:40.483238  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.477865  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.469750  5143 rpcz_store.cc:276] 0709 12:24:39.459939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.467694 (+1007755us) service_pool.cc:224] Handling call
0709 12:24:40.468139 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.469071 (+   932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.469293 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.483395  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.484915  5143 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:24:40.479262  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.485417  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.442395  5150 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:40.345880  8824 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:40.476531  5071 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:40.483541  5148 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:24:40.474907  5900 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:40.520128  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 45314.6us
I20260709 12:24:40.479592  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1135) took 1186 ms. Trace:
I20260709 12:24:40.521222  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.521227  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:40.522127  5071 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:40.521399  5142 rpcz_store.cc:276] 0709 12:24:39.292713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.457091 (+1164378us) service_pool.cc:224] Handling call
0709 12:24:40.457429 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.479314 (+ 21885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.479531 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.486260  5069 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:40.380955  8827 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:40.489761  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.467173  8790 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:40.492973  8735 consensus_queue.cc:1059] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:40.494063  5147 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:40.482369  8944 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:40.480247  8905 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:40.495342  8815 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:40.456216  8834 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:40.492969  5068 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:24:40.541599  5068 pending_rounds.cc:85] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 6
I20260709 12:24:40.541921  5068 pending_rounds.cc:107] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:24:40.542171  5068 raft_consensus.cc:2889] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:40.505811  8937 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.308s	user 0.001s	sys 0.004s
I20260709 12:24:40.443562  8930 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.509936  5145 raft_consensus.cc:2410] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:40.484771  5072 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 5.
I20260709 12:24:40.512603  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.524495  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:40.525242  5216 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:40.525811  8098 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:40.525759  5900 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:40.525997  5074 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:40.401049  8819 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:40.530427  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 48508us
I20260709 12:24:40.483842  8838 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:24:40.533476  8905 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:40.524852  8843 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:40.524806  8557 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:24:40.527311  8788 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:40.547338  8752 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.548583  5076 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:40.557016  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:40.560520  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.561897  8098 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:40.545214  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 279705us
I20260709 12:24:40.578533  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.591427  5071 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:40.585996  5073 raft_consensus.cc:2410] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:24:40.586632  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1231) took 2077 ms. Trace:
I20260709 12:24:40.593642  5145 rpcz_store.cc:276] 0709 12:24:38.509384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.108821 (+1599437us) service_pool.cc:224] Handling call
0709 12:24:40.578275 (+469454us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400751cd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:40.586550 (+  8275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":569814912}
I20260709 12:24:40.581679  5898 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:40.533888  8496 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:24:40.579344  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 698) took 1191 ms. Trace:
I20260709 12:24:40.599022  5898 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:40.599426  5150 rpcz_store.cc:276] 0709 12:24:39.388512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.902402 (+513890us) service_pool.cc:224] Handling call
0709 12:24:40.579134 (+676732us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54000ec254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:40.579261 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832299264}
I20260709 12:24:40.581800  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:40.553581  8930 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:40.589484  5188 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:40.604031  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:40.566485  8843 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.556738  8879 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
W20260709 12:24:40.611835  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 498412us
W20260709 12:24:40.610698  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.626523  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:40.628217  5188 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:40.633086  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:40.637410  5899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.026s	sys 0.039s
I20260709 12:24:40.554698  8790 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:40.638177  5899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.026s	sys 0.039s
I20260709 12:24:40.605566  8682 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.601066  8557 raft_consensus.cc:3055] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:24:40.645440  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 512057us
I20260709 12:24:40.622905  8843 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:40.577167  8400 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:24:40.620379  8708 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:40.598699  8752 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:40.635311  8777 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:40.637401  8802 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:24:40.672526  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.661096  5216 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:24:40.677337  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.679193  5216 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:40.682637  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:40.712548  5224 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:40.720188  5224 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:40.647710  8557 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:40.746318  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 788) took 1023 ms. Trace:
W20260709 12:24:40.751009  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.747087  5070 rpcz_store.cc:276] 0709 12:24:39.723200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.747956 (+ 24756us) service_pool.cc:224] Handling call
0709 12:24:39.763294 (+ 15338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:39.831441 (+ 68147us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:39.844021 (+ 12580us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.055610 (+211589us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:40.066206 (+ 10596us) log.cc:844] Serialized 52 byte log entry
0709 12:24:40.066377 (+   171us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:40.066411 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.066481 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.223796 (+157315us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b4800001a58. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ad464aa1 00007fd8ad45bb37 00007fd8ad45b9ca 00007fd8ad55326f 00007fd8ad552f37 00007fd8ad56755c 00007fd8ad567508 00007fd8ad567488 00007fd8ad567443 00007fd8ad5666ec 00007fd8ac490a96
0709 12:24:40.361284 (+137488us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:40.745872 (+384588us) raft_consensus.cc:1685] finished
0709 12:24:40.745910 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.746146 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33652,"prepare.queue_time_us":6349,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":71659,"raft.run_cpu_time_us":3554,"raft.run_wall_time_us":169559,"spinlock_wait_cycles":63464960,"thread_start_us":191111,"threads_started":2,"wal-append.queue_time_us":326164}
W20260709 12:24:40.782414  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:40.788112  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.661350  8829 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:40.806216  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.664319  8916 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 12:24:40.675093  8708 raft_consensus.cc:697] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:40.821775  5067 raft_consensus.cc:2410] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:24:40.664047  8917 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 12:24:40.664517  8682 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:40.825131  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 230085us
I20260709 12:24:40.661973  8948 consensus_queue.cc:1059] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.158s
I20260709 12:24:40.793740  8777 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:40.671015  8803 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.151s
I20260709 12:24:40.862022  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:40.805446  8845 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:40.848886  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 249909us
W20260709 12:24:40.871554  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.873039  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.873682  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.874598  5942 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:40.877485  5946 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:24:40.866585  8098 raft_consensus.cc:697] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:40.864486  5224 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:40.825659  8946 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:40.805044  8496 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:24:40.885083  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.887012  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.882620  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:40.888717  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.888561  5224 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:40.674098  8788 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:40.897032  5939 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:40.891486  8098 consensus_queue.cc:237] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:40.893927  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.901018  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.901866  5944 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:40.903046  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.906209  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:40.906984  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.907770  5941 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:40.911612  5946 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:40.867094  8400 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:40.864169  8815 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:40.918224  5944 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:40.925917  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.926129  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:40.933164  5939 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
W20260709 12:24:40.910354  8931 log.cc:927] Time spent T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:24:40.891637  8946 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:40.891770  8912 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:40.959327  8901 raft_consensus.cc:3055] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:40.960132  8901 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:40.891016  8838 consensus_queue.cc:1059] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:24:40.890362  8845 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:40.941584  8917 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:40.941335  8810 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:40.985718  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:41.014896  5939 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:40.951356  8912 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:40.950541  8946 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:40.972689  8496 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
W20260709 12:24:41.010895  8863 consensus_meta.cc:287] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.603s	user 0.003s	sys 0.009s
I20260709 12:24:40.908437  8557 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:40.999812  8917 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:41.034789  8752 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:41.025437  8960 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:41.019316  8829 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:40.970361  8777 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:41.062223  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 76666us
I20260709 12:24:41.063735  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:40.974357  8708 consensus_queue.cc:237] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:41.074373  5149 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.006s	sys 0.006s
W20260709 12:24:41.075018  5149 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.006s	sys 0.006s
I20260709 12:24:41.047107  8960 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:41.020206  8810 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:41.053391  8496 consensus_queue.cc:1243] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:24:41.055845  8557 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:41.066713  8949 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
W20260709 12:24:41.079901  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.080766  8959 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.084913  8955 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 12:24:41.086654  8791 consensus_meta.cc:287] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.000s
W20260709 12:24:41.087828  8952 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
W20260709 12:24:41.089752  8945 consensus_meta.cc:287] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.695s	user 0.011s	sys 0.003s
W20260709 12:24:41.090018  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.092604  8484 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.678s	user 0.012s	sys 0.000s
W20260709 12:24:41.101429  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 218075us
I20260709 12:24:41.055182  8829 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:41.121090  8905 consensus_meta.cc:287] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.004s
I20260709 12:24:41.122491  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:41.122882  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:41.123095  8905 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:41.124781  5222 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 12:24:41.132588  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 302288us
I20260709 12:24:41.139983  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:41.141240  5150 raft_consensus.cc:2393] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:41.142196  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:41.143260  5145 raft_consensus.cc:2410] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:24:41.146364  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:41.146762  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:41.147202  5145 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:41.148138  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:41.147535  5150 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:41.151971  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:41.152542  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.153009  5145 raft_consensus.cc:2410] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:41.155900  5150 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:24:41.156505  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:41.158420  5145 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:41.158907  5142 raft_consensus.cc:3055] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:41.159543  5142 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:41.160279  5150 pending_rounds.cc:85] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 5
I20260709 12:24:41.160777  5150 pending_rounds.cc:107] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:24:41.160642  5142 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.164588  5150 raft_consensus.cc:2889] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:41.164592  8783 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:41.170187  8783 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:41.165998  5142 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:41.168196  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:41.173588  5145 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:41.104837  8682 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:41.057245  8912 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:41.181521  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:41.190374  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.179390  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:41.217227  5148 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:41.218856  5148 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:41.219246  5145 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:41.211701  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1246) took 2697 ms. Trace:
I20260709 12:24:41.219920  5143 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:41.220392  5141 rpcz_store.cc:276] 0709 12:24:38.513885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.255689 (+1741804us) service_pool.cc:224] Handling call
0709 12:24:40.256059 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.256481 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.256513 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.257054 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.566597 (+309543us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:40.566646 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.566730 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.211423 (+644693us) raft_consensus.cc:1685] finished
0709 12:24:41.211462 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.211622 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1709,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":9165,"thread_start_us":309152,"threads_started":1,"wal-append.queue_time_us":417320}
I20260709 12:24:41.219681  5148 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.130625  8905 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:41.229280  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:41.233635  5148 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:41.238903  8845 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:41.239711  8963 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:24:41.241905  8966 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:24:41.245198  8930 consensus_meta.cc:287] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.627s	user 0.008s	sys 0.001s
I20260709 12:24:41.076018  8917 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:41.106140  8959 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.212316  8976 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:24:41.262423  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.253556  8919 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.254432  8953 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 12:24:41.256032  8682 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:41.248649  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 710) took 1770 ms. Trace:
I20260709 12:24:41.264379  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1247) took 2750 ms. Trace:
I20260709 12:24:41.265220  5144 rpcz_store.cc:276] 0709 12:24:39.478559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.455739 (+977180us) service_pool.cc:224] Handling call
0709 12:24:40.478947 (+ 23208us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.480056 (+  1109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.480083 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.480558 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.672872 (+192314us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:40.672921 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.672999 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.248411 (+575412us) raft_consensus.cc:1685] finished
0709 12:24:41.248434 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.248585 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52625,"raft.queue_time_us":1644,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":10351,"thread_start_us":191978,"threads_started":1,"wal-append.queue_time_us":399868}
W20260709 12:24:41.267277  5114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.016s	sys 0.024s
I20260709 12:24:41.265673  5148 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:41.265924  5146 rpcz_store.cc:276] 0709 12:24:38.514230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.256762 (+1742532us) service_pool.cc:224] Handling call
0709 12:24:40.257054 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.257453 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:40.294823 (+ 37370us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b480004c758. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ad464aa1 00007fd8ad45bb37 00007fd8ad45b9ca 00007fd8ad55326f 00007fd8ad552f37 00007fd8ad56755c 00007fd8ad567508 00007fd8ad567488 00007fd8ad567443 00007fd8ad5666ec 00007fd8ac490a96
0709 12:24:40.599341 (+304518us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.825145 (+225804us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.825263 (+   118us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.825770 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.825956 (+   186us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:40.826177 (+   221us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.826371 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.848503 (+ 22132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.264034 (+415531us) raft_consensus.cc:1685] finished
0709 12:24:41.264066 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.264291 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70432,"prepare.queue_time_us":598854,"prepare.run_cpu_time_us":145,"prepare.run_wall_time_us":12340,"raft.queue_time_us":129601,"raft.run_cpu_time_us":1288,"raft.run_wall_time_us":50215,"spinlock_wait_cycles":8909056,"thread_start_us":485429,"threads_started":2,"wal-append.queue_time_us":402918}
W20260709 12:24:41.282212  5114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.019s	sys 0.024s
W20260709 12:24:41.282862  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 680083us
I20260709 12:24:41.218035  8916 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:24:41.238114  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:41.286391  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.249269  8912 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:41.285830  5142 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:24:41.263741  8910 consensus_meta.cc:287] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.762s	user 0.008s	sys 0.004s
I20260709 12:24:41.248461  8970 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:41.263082  8977 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:41.287161  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 49123.1us
I20260709 12:24:41.283736  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:41.293319  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:41.294595  5115 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:41.265713  8783 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:41.282174  8525 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.300066  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.301821  5114 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:41.302708  5115 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:41.250494  8930 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:41.216987  8960 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.309082  5114 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:41.306779  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 243168us
I20260709 12:24:41.290781  8967 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:41.314239  8967 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.314714  8967 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:41.318173  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1127) took 2484 ms. Trace:
I20260709 12:24:41.318910  8960 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:41.268280  8919 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:41.318679  5149 rpcz_store.cc:276] 0709 12:24:38.834128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.416290 (+1582162us) service_pool.cc:224] Handling call
0709 12:24:40.416673 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.417063 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.417094 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.417639 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 12:24:41.075787 (+658148us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:41.103058 (+ 27271us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.103400 (+   342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.103509 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.317848 (+214339us) raft_consensus.cc:1685] finished
0709 12:24:41.317879 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.318101 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25171,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":54,"raft.queue_time_us":659,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":3794,"spinlock_wait_cycles":1505024,"thread_start_us":657417,"threads_started":1,"wal-append.queue_time_us":816061}
I20260709 12:24:41.320442  8959 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:41.321621  8810 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:41.321892  8919 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:41.298489  8970 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:41.323063  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:41.324153  8810 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:41.324589  8970 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:41.325112  8843 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.656s	user 0.012s	sys 0.000s
I20260709 12:24:41.298055  8484 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:41.325807  5899 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:41.326188  8484 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:41.326924  8484 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.327428  8484 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:41.300129  8977 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:41.330031  8834 consensus_meta.cc:287] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.783s	user 0.011s	sys 0.003s
I20260709 12:24:41.330564  8834 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:41.331820  8977 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:41.332647  8834 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:41.341245  8810 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:41.342058  8810 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:41.343235  8810 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.345207  5075 raft_consensus.cc:2393] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:41.348453  8790 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:41.300696  8839 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:41.348551  5899 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:41.351701  8959 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:41.353466  5899 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:41.303282  8752 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:41.373688  5899 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:24:41.304044  8824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.010s	sys 0.024s
I20260709 12:24:41.352205  8959 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:41.368863  8970 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.367403  8971 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:24:41.391170  8985 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.419535  8985 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.422760  8930 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:41.426744  8901 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.428046  8901 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:41.318753  8829 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:41.384765  5899 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:41.364836  8843 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.435998  8843 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:41.393311  8970 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:41.440335  8970 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:41.307173  8525 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:41.395515  8946 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:41.443095  5943 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:41.443712  8525 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:41.392896  8959 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:41.384943  8967 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:41.444597  8946 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:41.446617  8967 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:41.450825  8827 consensus_meta.cc:287] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.909s	user 0.012s	sys 0.003s
I20260709 12:24:41.451316  8827 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:41.391773  8484 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:41.450268  8985 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:41.456132  8959 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.433348  8829 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.460209  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 75565.5us
I20260709 12:24:41.472575  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:41.388834  8824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.011s	sys 0.025s
W20260709 12:24:41.413021  8954 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.527s	user 0.003s	sys 0.000s
W20260709 12:24:41.417208  8944 consensus_meta.cc:287] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.889s	user 0.010s	sys 0.002s
I20260709 12:24:41.504024  8944 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:41.418210  8969 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:41.394505  8975 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:41.481240  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:41.488904  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:41.490882  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:41.475457  8827 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:41.509706  5948 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:41.500116  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:41.501056  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:41.502846  8799 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:41.508911  5944 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:41.509114  8098 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.474570  8829 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.475703  8946 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:41.506002  8969 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:41.511166  5189 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:41.507251  8951 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.020s	sys 0.012s
I20260709 12:24:41.513177  8098 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.510051  8827 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:41.517823  8708 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:41.513136  8982 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:41.538308  8708 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.538405  8982 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:41.538810  8708 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:41.521126  5941 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:41.513319  8975 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:41.525130  8829 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:41.542502  8975 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:41.521391  8985 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:41.523955  5189 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:41.525142  8969 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:41.515703  8946 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.553018  8946 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:41.558653  8947 raft_consensus.cc:3055] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:41.559283  8947 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:41.537694  8827 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:41.538884  8982 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:41.561053  5947 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:41.560567  8862 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:41.548187  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.548652  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.524403  8951 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.021s	sys 0.012s
I20260709 12:24:41.552314  8969 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:41.567171  8985 raft_consensus.cc:3055] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:41.567699  8985 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:41.568557  8985 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:41.569576  5190 leader_election.cc:336] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.580s (SENT)
I20260709 12:24:41.569772  8525 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:41.570183  8525 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.570005  8802 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:41.570787  8525 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:41.568879  5189 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:41.570098  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:41.573212  8983 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:41.573711  8977 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:41.573774  8983 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.575534  8983 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:41.562929  8959 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:41.628770  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.569515  5147 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:41.741573  8905 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.742381  8905 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:41.746999  8708 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:41.560524  8777 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.162s
I20260709 12:24:41.745316  8976 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.850422  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.745658  8829 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.872161  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.930487  5900 leader_election.cc:336] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.889s (SENT)
W20260709 12:24:42.042230  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 111807us
W20260709 12:24:42.045171  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.784374  8947 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:41.953019  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 650321us
I20260709 12:24:42.109314  5115 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:42.082466  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.084281  5898 leader_election.cc:336] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.814s (SENT)
W20260709 12:24:42.093004  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.171926  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 87721.7us
W20260709 12:24:42.184273  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.192373  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 83153.1us
W20260709 12:24:42.192575  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.193683  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:42.192281  8967 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:42.206238  8777 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.206493  8901 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:42.206916  8777 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:42.208372  8940 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.207320  8901 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.209103  8940 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.209412  8777 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:41.988198  8958 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:42.015975  8913 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.019423  8985 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:42.223645  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:41.888334  8829 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:41.879434  8976 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:42.178843  8972 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:42.258033  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.219456  8901 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:42.210038  8905 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:42.225693  8752 raft_consensus.cc:2764] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:24:42.049356  8827 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.264714  8972 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.220541  8958 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.226630  8947 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:42.227249  8777 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:42.272675  8752 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:42.206214  8967 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:42.294137  8972 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:42.294314  5070 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.162s	user 0.015s	sys 0.007s
W20260709 12:24:42.294848  5070 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.163s	user 0.015s	sys 0.008s
I20260709 12:24:42.295636  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1289) took 1168 ms. Trace:
I20260709 12:24:42.295987  5070 rpcz_store.cc:276] 0709 12:24:41.127261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.129718 (+  2457us) service_pool.cc:224] Handling call
0709 12:24:41.130215 (+   497us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.130721 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.130765 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.131502 (+   737us) log.cc:844] Serialized 26 byte log entry
0709 12:24:42.295221 (+1163719us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:42.295273 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.295356 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.295401 (+    45us) raft_consensus.cc:1685] finished
0709 12:24:42.295413 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.295575 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5452,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":134,"thread_start_us":1163145,"threads_started":1,"wal-append.queue_time_us":347492}
I20260709 12:24:42.298678  8940 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:42.300789  8967 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:42.301105  8946 consensus_meta.cc:287] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.747s	user 0.003s	sys 0.004s
W20260709 12:24:42.302305  8525 consensus_meta.cc:287] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.731s	user 0.005s	sys 0.006s
W20260709 12:24:42.305985  8799 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.793s	user 0.008s	sys 0.005s
I20260709 12:24:42.270306  8708 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:42.279908  8958 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:42.260764  8913 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:42.283392  8777 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:42.342921  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.316236  8799 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.353140  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 935) took 1446 ms. Trace:
I20260709 12:24:42.353785  5943 rpcz_store.cc:276] 0709 12:24:40.906846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.908211 (+  1365us) service_pool.cc:224] Handling call
0709 12:24:40.908551 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:41.443503 (+534952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.339539 (+896036us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b54001a0254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:42.353047 (+ 13508us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1228781952}
W20260709 12:24:42.338469  8982 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.774s	user 0.008s	sys 0.004s
I20260709 12:24:42.311699  8946 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:42.363637  5942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.017s	sys 0.017s
I20260709 12:24:42.310844  8525 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:42.371446  5942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.017s	sys 0.017s
W20260709 12:24:42.364202  8969 consensus_meta.cc:287] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.786s	user 0.010s	sys 0.002s
I20260709 12:24:42.349951  8779 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.377301  8779 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:42.380215  8779 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:42.380898  8983 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.805s	user 0.007s	sys 0.006s
I20260709 12:24:42.382479  8913 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:42.380893  8779 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.384052  8779 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:42.386758  8779 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:42.391357  8946 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:42.393890  8777 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:42.363692  8799 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.399442  8777 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.371518  8982 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.427500  8708 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:42.429446  8708 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:42.430487  8708 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:42.430970  8969 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.431667  8969 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.431996  8708 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:42.433967  8969 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:42.399818  8983 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:42.436455  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 212902us
W20260709 12:24:42.434882  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.440428  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:42.438097  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:42.441404  5945 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:42.443461  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:42.445734  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:42.439750  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:42.448312  5943 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:42.449649  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:42.450438  5943 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:42.426383  8913 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.449786  5189 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:42.434257  8708 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:42.454309  8940 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:42.444437  8969 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.456760  8972 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:42.454684  8967 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:24:42.455454  5189 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:42.458493  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 893) took 1566 ms. Trace:
I20260709 12:24:42.456789  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:42.418906  8901 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:42.461635  5939 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.009s	sys 0.029s
I20260709 12:24:42.446966  8879 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:42.467953  5115 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:42.424415  8777 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:42.470073  5948 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:42.469776  5942 rpcz_store.cc:276] 0709 12:24:40.892153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.892359 (+   206us) service_pool.cc:224] Handling call
0709 12:24:40.892806 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.035466 (+142660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.035520 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.528928 (+493408us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b2c00152dc8. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:24:41.529579 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 12:24:42.373806 (+844227us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:42.373873 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.373982 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.458083 (+ 84101us) raft_consensus.cc:1685] finished
0709 12:24:42.458116 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.458419 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39412,"raft.queue_time_us":213223,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":8096,"spinlock_wait_cycles":5994752,"thread_start_us":977499,"threads_started":2,"wal-append.queue_time_us":926674}
I20260709 12:24:42.436162  8829 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:42.469133  8972 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:42.473950  5939 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.009s	sys 0.030s
I20260709 12:24:42.473105  8985 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:42.458940  8976 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:42.458745  8827 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:42.461462  8940 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:24:42.497943  8951 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.020s	sys 0.055s
W20260709 12:24:42.499202  5190 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.013s	sys 0.025s
W20260709 12:24:42.500694  5190 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.013s	sys 0.025s
W20260709 12:24:42.501031  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 906246us
I20260709 12:24:42.493508  8985 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.493594  8972 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:42.498601  8947 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:42.460268  8913 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.472224  8969 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:42.500048  8951 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.021s	sys 0.055s
I20260709 12:24:42.512926  5216 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:42.522368  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:42.523566  5943 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:42.517642  5221 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:24:42.517845  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:42.518249  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:42.525223  5219 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:42.518540  5215 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:42.499490  8940 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:42.504127  8976 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.532227  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:42.509341  8879 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:42.536628  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.535620  5945 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:42.535032  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:42.537478  5148 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:42.510391  8985 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:42.531423  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:42.539074  5141 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:42.541805  5215 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:42.541872  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.539944  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.542564  5143 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:42.543756  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:42.532641  8484 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.544432  5141 raft_consensus.cc:2376] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9. Re-sending same reply.
I20260709 12:24:42.544584  8484 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.545897  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.546703  5144 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 6: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:42.546974  8484 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:42.540551  8827 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:24:42.519335  8947 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:42.552074  8947 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:42.552929  8982 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:42.553755  9000 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:42.554471  9000 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:42.526866  8525 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:42.544441  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.561079  5943 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:24:42.564450  5943 pending_rounds.cc:85] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 5
I20260709 12:24:42.562440  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:42.564710  5148 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 11: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:42.565023  5943 pending_rounds.cc:107] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:24:42.565372  5943 raft_consensus.cc:2889] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:42.561740  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:42.569468  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.534700  8752 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.575153  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.576836  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.538911  8879 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.577505  5144 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:42.580724  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.581178  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.581481  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.552120  8969 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:42.584954  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:42.587765  5141 raft_consensus.cc:2410] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:42.588807  5145 raft_consensus.cc:2393] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:42.584826  5942 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:24:42.591069  5215 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:42.587709  5143 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:42.552959  9001 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:24:42.591993  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:42.536321  8976 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:42.595718  5942 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:42.596980  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:42.598526  5144 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:42.598431  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:42.599972  8983 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:42.602101  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.603546  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.604267  5146 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:42.578680  8752 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:42.605165  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.585635  8799 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:42.588651  8879 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:42.606494  8777 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:42.606843  8752 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:42.608860  8976 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:42.611709  8985 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:42.645587  8752 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:42.646205  8752 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.646610  8752 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:42.647267  8985 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:42.615826  5146 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 8: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:42.617019  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.650132  5940 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:42.589253  9000 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:42.620199  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:42.641376  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.642299  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.651891  5946 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:42.576458  8982 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:24:42.652779  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:42.652817  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:42.654577  5943 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:42.654651  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.655409  5146 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
W20260709 12:24:42.664106  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.701133  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:42.701987  5942 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:42.710460  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.614149  9003 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:24:42.712451  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:42.717453  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.718017  8999 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:24:42.619485  9009 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 12:24:42.720723  9009 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.723248  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 945) took 1093 ms. Trace:
I20260709 12:24:42.723697  5939 rpcz_store.cc:276] 0709 12:24:41.629983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.630163 (+   180us) service_pool.cc:224] Handling call
0709 12:24:41.634205 (+  4042us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.634733 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.634777 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.719808 (+ 85031us) log.cc:844] Serialized 26 byte log entry
0709 12:24:42.490678 (+770870us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:42.490737 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.490835 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.722994 (+232159us) raft_consensus.cc:1685] finished
0709 12:24:42.723015 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.723179 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21064,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":20900,"thread_start_us":760557,"threads_started":1,"wal-append.queue_time_us":811939}
I20260709 12:24:42.727942  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:42.659294  8525 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:42.731886  8525 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:42.732928  8525 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.735086  8819 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:42.735353  5943 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:42.738272  8983 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.738938  8983 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:42.656685  8972 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:42.744606  8972 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:42.654006  8985 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.746026  8985 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:42.746891  8735 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:24:42.748782  8985 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:42.756536  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 158153us
I20260709 12:24:42.761014  5115 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:42.765507  5948 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:42.766957  5946 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:42.613840  8958 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:42.772433  5144 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
W20260709 12:24:42.780000  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 91065.2us
I20260709 12:24:42.782401  5142 raft_consensus.cc:2368] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12.
I20260709 12:24:42.783665  5190 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:42.784147  8958 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:42.780790  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 53021.2us
I20260709 12:24:42.794152  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:42.764433  8947 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.796396  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:42.802176  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:42.791733  8484 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:24:42.805100  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:42.799319  5189 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:42.806975  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:42.807639  5215 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:42.810060  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:42.810550  8951 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:42.795783  9002 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:42.812265  8777 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:42.813120  8982 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:42.819089  8960 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.822192  8960 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:42.822165  8982 raft_consensus.cc:697] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:42.813297  5189 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:42.721351  9009 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.789075  8913 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:42.819012  9000 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.808666  9003 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:42.792863  8983 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:42.829077  5144 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:24:42.829942  5947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 932) took 1929 ms (client timeout 2102 ms). Trace:
W20260709 12:24:42.846623  5947 rpcz_store.cc:269] 0709 12:24:40.900570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.902741 (+  2171us) service_pool.cc:224] Handling call
0709 12:24:42.829667 (+1926926us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b540000ad54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:42.829846 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511233408}
I20260709 12:24:42.799518  8972 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:24:42.800493  8958 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:42.799151  8985 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:24:42.829011  9002 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.835129  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:42.829618  5189 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:42.804792  9012 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:42.838847  8951 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:42.859496  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:42.805362  8947 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:42.863448  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52693us
I20260709 12:24:42.846069  9003 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.870272  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:42.866863  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:42.871098  5943 raft_consensus.cc:2393] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:42.868273  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:42.825237  8982 consensus_queue.cc:237] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:42.851531  8958 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.845719  8819 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:24:42.854879  9002 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:42.855670  9012 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.872972  5947 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:42.880561  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.866122  8913 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:42.860504  9014 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:42.849687  9000 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:42.871920  9003 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:42.869387  8951 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:42.904425  5947 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:24:42.865190  9011 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:24:42.867360  8940 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.875927  8958 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:42.895017  9012 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:42.897639  8983 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:42.915575  5222 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:42.913803  5948 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:42.898540  9014 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.906927  5189 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:42.922643  9009 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:42.918375  8976 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.936088  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:42.938540  5219 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:42.939896  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:42.920153  8983 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.914006  8947 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:42.944617  8983 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:42.945564  8400 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:24:42.945169  8976 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:42.955885  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:42.957526  8400 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:42.957973  8400 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.958356  8400 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:42.934151  9014 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:42.948524  8976 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:42.961452  5939 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:42.964890  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 942) took 1475 ms. Trace:
I20260709 12:24:42.967417  5941 rpcz_store.cc:276] 0709 12:24:41.489229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.489343 (+   114us) service_pool.cc:224] Handling call
0709 12:24:41.489715 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:41.539393 (+ 49678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.964458 (+1425065us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b5400008f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:42.964791 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71367424}
I20260709 12:24:42.950187  8708 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:42.940836  8752 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:42.990270  8976 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.000675  8708 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:43.013375  8708 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:43.018743  5146 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:42.994745  8972 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:24:43.005806  8913 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:43.007642  8879 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:43.018481  8976 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:43.026382  5939 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:24:43.029384  8838 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.030731  8400 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:42.996629  8951 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:43.050825  5148 raft_consensus.cc:2368] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:24:43.051422  8879 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:43.039469  5041 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.824s	user 0.046s	sys 0.052s
I20260709 12:24:43.040206  9003 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:43.054347  8917 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:43.055648  8400 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:43.055428  9002 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:43.054016  9003 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:43.046301  8913 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:43.058647  9002 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:43.059195  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:43.060348  5947 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:43.062275  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:43.062486  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:43.063167  5219 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:43.059577  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:43.065984  5941 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:43.067535  8901 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:43.067835  8838 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:43.063230  5943 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:43.070108  8838 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:43.070279  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:43.071040  5143 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:43.071791  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:43.072005  8976 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:43.072332  5143 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:43.072614  5947 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:43.074070  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:43.074120  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:43.074415  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:43.075114  9002 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:43.075170  5141 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:43.076020  9002 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:43.076247  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:43.076946  5141 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:43.077601  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:43.079327  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:43.080245  5215 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:43.080307  8965 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:43.086370  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:43.089591  5067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 1281) took 2930 ms (client timeout 1892 ms). Trace:
I20260709 12:24:43.089877  8983 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:43.090118  5067 rpcz_store.cc:269] 0709 12:24:40.159257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.159715 (+   458us) service_pool.cc:224] Handling call
0709 12:24:43.089306 (+2929591us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b5400327b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:43.089477 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1261405568}
I20260709 12:24:43.092849  5942 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
W20260709 12:24:43.093683  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.094208  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:43.098344  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:43.100488  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:43.100749  5149 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:43.102499  8958 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:43.104828  5939 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:43.106675  8735 raft_consensus.cc:3055] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:43.106444  9012 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:43.107118  8735 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:43.107944  8735 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:43.108697  9012 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:43.109143  8735 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:43.077083  9002 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:43.245886  5041 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.031s	user 0.047s	sys 0.053s
W20260709 12:24:43.246625  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 2.03143e+06us
I20260709 12:24:43.267318  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:43.267446  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 189679us
I20260709 12:24:43.284896  5141 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:43.290113  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:43.086364  8965 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.110988  9014 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:43.332049  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:43.470279  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:43.471863  5141 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:43.473050  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:43.415352  5143 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:43.247641  8983 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:43.527344  9014 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:43.539881  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:43.540738  5143 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:43.541880  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:43.542764  5143 raft_consensus.cc:2376] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12. Re-sending same reply.
I20260709 12:24:43.543855  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:43.843204  9034 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
W20260709 12:24:43.340695  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.162247  5041 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.002s	sys 0.011s
W20260709 12:24:44.163266  5041 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.003s	sys 0.012s
W20260709 12:24:44.163810  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 915733us
I20260709 12:24:44.181452  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1279) took 1599 ms. Trace:
I20260709 12:24:44.223141  5144 rpcz_store.cc:276] 0709 12:24:42.582133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.803367 (+221234us) service_pool.cc:224] Handling call
0709 12:24:42.803672 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.846332 (+ 42660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.180655 (+1334323us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b54000ec754. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:44.181154 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57698176}
I20260709 12:24:44.185846  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1276) took 1604 ms. Trace:
I20260709 12:24:44.202776  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:44.225174  5146 rpcz_store.cc:276] 0709 12:24:42.581283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.712150 (+130867us) service_pool.cc:224] Handling call
0709 12:24:44.185207 (+1473057us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b540036f454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:44.185525 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693080960}
I20260709 12:24:44.191633  5073 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:24:44.229410  5073 pending_rounds.cc:85] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 5
I20260709 12:24:44.229853  5073 pending_rounds.cc:107] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:24:44.216745  9002 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.231382  9002 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:44.232558  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:44.209862  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:44.234028  5144 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:44.230336  5073 raft_consensus.cc:2889] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:44.234818  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 733) took 1402 ms. Trace:
I20260709 12:24:44.235432  5144 rpcz_store.cc:276] 0709 12:24:42.832211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.231881 (+1399670us) service_pool.cc:224] Handling call
0709 12:24:44.234690 (+  2809us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.235499  9002 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:44.234119  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 68823.8us
I20260709 12:24:44.237668  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:44.238653  5188 connection.cc:256] RPC call timeout handler was delayed by -2.3639240669999708s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
I20260709 12:24:44.238835  5144 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:44.239653  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 731) took 1412 ms. Trace:
I20260709 12:24:44.240229  5144 rpcz_store.cc:276] 0709 12:24:42.827180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.236918 (+1409738us) service_pool.cc:224] Handling call
0709 12:24:44.239575 (+  2657us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.241663  8982 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.243182  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:44.242864  8982 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:44.228509  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:44.246213  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:44.247413  5074 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:44.248333  8982 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:44.249022  9033 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:44.249503  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:44.250506  5074 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:44.251627  5144 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:44.252569  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 736) took 1297 ms. Trace:
I20260709 12:24:44.253046  5144 rpcz_store.cc:276] 0709 12:24:42.955002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.241235 (+1286233us) service_pool.cc:224] Handling call
0709 12:24:44.252468 (+ 11233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.246330  5146 raft_consensus.cc:2368] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12.
I20260709 12:24:44.254527  8752 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:44.255046  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1286) took 1443 ms. Trace:
I20260709 12:24:44.255867  5074 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:44.254608  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:44.258189  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:44.255605  5146 rpcz_store.cc:276] 0709 12:24:42.811147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.227707 (+1416560us) service_pool.cc:224] Handling call
0709 12:24:44.254953 (+ 27246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.262270  5144 raft_consensus.cc:2368] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10.
I20260709 12:24:44.266255  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1285) took 1456 ms. Trace:
I20260709 12:24:44.266669  5144 rpcz_store.cc:276] 0709 12:24:42.810837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.254057 (+1443220us) service_pool.cc:224] Handling call
0709 12:24:44.266188 (+ 12131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.268819  5144 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:44.269619  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 717) took 1457 ms. Trace:
I20260709 12:24:44.270092  5144 rpcz_store.cc:276] 0709 12:24:42.812101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.267520 (+1455419us) service_pool.cc:224] Handling call
0709 12:24:44.268279 (+   759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.269252 (+   973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.269517 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.265489  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:44.250022  9033 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:44.279652  8958 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:44.281618  5069 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:44.282747  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:44.279773  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:44.285394  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:44.273685  5070 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:44.292660  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:44.293592  5070 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:44.294639  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:44.303105  5145 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:44.303936  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 725) took 1486 ms. Trace:
I20260709 12:24:44.304246  5145 rpcz_store.cc:276] 0709 12:24:42.817277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.296762 (+1479485us) service_pool.cc:224] Handling call
0709 12:24:44.297225 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.303647 (+  6422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.303867 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.305920  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:44.306854  5900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.207s	user 0.017s	sys 0.009s
W20260709 12:24:44.307245  5900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.208s	user 0.017s	sys 0.009s
I20260709 12:24:44.307597  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:44.309095  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:44.311965  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:44.312590  9036 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.313482  9036 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:44.314531  8975 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:44.315752  8975 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:44.319410  9001 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:44.297711  9033 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:44.320250  9001 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:44.321521  8975 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:44.322382  8975 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.319140  9043 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:44.324446  5146 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:44.337900  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 98912.8us
W20260709 12:24:44.358558  5171 debug-util.cc:398] Leaking SignalData structure 0x7b08000b0e60 after lost signal to thread 1520
I20260709 12:24:44.360651  9046 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:44.361819  9046 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:44.366353  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.366798  9001 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:44.367549  9036 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:44.368547  9042 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.370365  9001 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.383584  9001 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:44.385838  9042 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:44.385910  8975 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:44.388576  9042 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:44.398131  5144 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:44.399639  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 718) took 1587 ms. Trace:
I20260709 12:24:44.400241  5144 rpcz_store.cc:276] 0709 12:24:42.812439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.271967 (+1459528us) service_pool.cc:224] Handling call
0709 12:24:44.330619 (+ 58652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.399113 (+ 68494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.399500 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.405858  9001 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:44.432377  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:44.376981  5900 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:44.435058  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 112048us
I20260709 12:24:44.436900  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:44.412997  8484 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.437952  5188 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:44.438175  8484 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:44.439992  5143 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:44.440315  8815 raft_consensus.cc:3055] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:44.440860  8815 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:44.438601  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:44.437150  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:44.441519  8484 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:44.452123  9011 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:44.447350  5222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1111) took 1931 ms. Trace:
I20260709 12:24:44.454385  5222 rpcz_store.cc:276] 0709 12:24:42.515848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.516068 (+   220us) service_pool.cc:224] Handling call
0709 12:24:42.516533 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.921725 (+405192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.446462 (+1524737us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b5400514854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:44.446979 (+   517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917250944}
I20260709 12:24:44.456712  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:44.457490  5222 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:44.451094  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:44.442585  8815 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:44.458747  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 48760.6us
I20260709 12:24:44.460767  8815 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:44.442152  5217 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:44.458846  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:44.462653  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:44.462761  5222 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:44.470779  5146 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:44.365461  9046 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:44.474284  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 104317us
I20260709 12:24:44.475435  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1287) took 1661 ms. Trace:
I20260709 12:24:44.474340  8975 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:44.477792  5217 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:44.483044  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:44.484414  8982 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:44.484522  5900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.553s (SENT) [suppressed 27 similar messages]
I20260709 12:24:44.486434  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:44.488670  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:44.490427  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:44.492116  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:44.493001  5222 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:44.493098  5218 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:44.496501  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:44.498752  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:44.509495  5218 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:44.501808  5146 rpcz_store.cc:276] 0709 12:24:42.811445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.264275 (+1452830us) service_pool.cc:224] Handling call
0709 12:24:44.473103 (+208828us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:44.510998  5941 consensus_meta.cc:287] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.444s	user 0.019s	sys 0.025s
I20260709 12:24:44.516710  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:44.517690  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.519704  5222 raft_consensus.cc:2410] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:24:44.521502  5217 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
W20260709 12:24:44.522759  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 45727.5us
I20260709 12:24:44.521183  8982 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:44.520999  5941 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
W20260709 12:24:44.525099  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 823) took 1466 ms (client timeout 1796 ms). Trace:
W20260709 12:24:44.525650  5941 rpcz_store.cc:269] 0709 12:24:43.058300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.059344 (+  1044us) service_pool.cc:224] Handling call
0709 12:24:44.524882 (+1465538us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.534044  8940 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:44.549893  8917 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 1.495s	user 0.012s	sys 0.024s
I20260709 12:24:44.554512  8960 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:44.556253  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 68658.2us
I20260709 12:24:44.556430  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1287) took 3436 ms. Trace:
I20260709 12:24:44.558349  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:44.558578  9011 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:44.558846  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 961) took 1700 ms. Trace:
I20260709 12:24:44.561203  5948 rpcz_store.cc:276] 0709 12:24:42.857908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.858431 (+   523us) service_pool.cc:224] Handling call
0709 12:24:44.553450 (+1695019us) spinlock_profiling.cc:243] Waited 53.2 ms on lock 0x7b5400005854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:44.558595 (+  5145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122386176}
I20260709 12:24:44.561822  5188 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:44.558159  5073 rpcz_store.cc:276] 0709 12:24:41.120498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.120741 (+   243us) service_pool.cc:224] Handling call
0709 12:24:44.172615 (+3051874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.235832 (+ 63217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.556096 (+320264us) spinlock_profiling.cc:243] Waited 3.05 s on lock 0x7b5400327b50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:44.556318 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7017139968}
I20260709 12:24:44.566102  8708 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:44.566644  8708 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:44.567142  8708 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:44.566825  8940 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:44.567476  8959 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:44.568408  8959 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:44.577368  9011 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:44.580892  5190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.709s	user 0.014s	sys 0.023s
W20260709 12:24:44.581434  5190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.710s	user 0.014s	sys 0.023s
W20260709 12:24:44.582007  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 1.7151e+06us
I20260709 12:24:44.587986  8959 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:44.594660  8708 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:44.602415  8708 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:44.600255  5943 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.530s	user 0.013s	sys 0.029s
I20260709 12:24:44.604449  5943 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:44.605242  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 824) took 1544 ms. Trace:
I20260709 12:24:44.606338  5943 rpcz_store.cc:276] 0709 12:24:43.060459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.062182 (+  1723us) service_pool.cc:224] Handling call
0709 12:24:44.605144 (+1542962us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.620486  8400 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.622823  8400 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:44.630399  8400 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:44.635593  5898 connection.cc:256] RPC call timeout handler was delayed by -1.394888613999683s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19940000000000002s for a response
W20260709 12:24:44.638002  8735 consensus_meta.cc:287] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.378s	user 0.003s	sys 0.020s
I20260709 12:24:44.638720  8959 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.639310  8959 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:44.641139  8959 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:44.683698  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.684607  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.685015  8959 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:44.685786  8959 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:44.688866  8959 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:44.690585  8779 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.691125  8779 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:44.693733  8779 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:44.716333  5900 leader_election.cc:336] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.937s (SENT)
I20260709 12:24:44.717092  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:44.719097  9036 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:44.735304  5141 raft_consensus.cc:2410] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:24:44.743963  8098 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:44.746464  5141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 726) took 1928 ms (client timeout 1836 ms). Trace:
W20260709 12:24:44.747078  8819 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.743s	user 0.007s	sys 0.016s
W20260709 12:24:44.747659  8819 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.743s	user 0.007s	sys 0.016s
W20260709 12:24:44.747097  5141 rpcz_store.cc:269] 0709 12:24:42.818632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.472749 (+654117us) service_pool.cc:224] Handling call
0709 12:24:44.746185 (+1273436us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540036ea54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:44.746390 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2734765312}
W20260709 12:24:44.785379  8946 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.212s	user 0.032s	sys 0.058s
W20260709 12:24:44.789348  8946 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.216s	user 0.032s	sys 0.060s
I20260709 12:24:43.275621  8947 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 1.517s
W20260709 12:24:44.795264  5898 leader_election.cc:336] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.553s (SENT)
I20260709 12:24:44.809262  5150 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:44.873714  5145 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 12:24:44.874694  5142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1140) took 2341 ms (client timeout 1735 ms). Trace:
W20260709 12:24:44.875275  5142 rpcz_store.cc:269] 0709 12:24:42.531886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.539662 (+  7776us) service_pool.cc:224] Handling call
0709 12:24:44.872947 (+2333285us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54004fa954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:44.873312 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549778688}
W20260709 12:24:44.882521  5899 connection.cc:256] RPC call timeout handler was delayed by -1.6833809730001121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 12:24:44.883041  5899 connection.cc:256] RPC call timeout handler was delayed by -1.6775365210000928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
W20260709 12:24:44.883764  5899 connection.cc:256] RPC call timeout handler was delayed by -1.6648202699998365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
W20260709 12:24:44.884157  5899 connection.cc:256] RPC call timeout handler was delayed by -1.6207264889999351s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
W20260709 12:24:44.884467  5899 connection.cc:256] RPC call timeout handler was delayed by -1.612626488999922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19940000000000002s for a response
W20260709 12:24:44.886816  5899 leader_election.cc:336] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Timed out: RequestConsensusVote RPC to 127.1.123.68:43449 timed out after 1.814s (SENT)
I20260709 12:24:44.887388  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:44.888340  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:44.889365  5071 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:44.890283  8946 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.890991  8946 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:44.893396  8946 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:44.896623  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:44.897737  9022 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:44.898227  9022 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.898571  9022 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:44.901034  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.906849  5070 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:44.911936  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:44.915705  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:44.917970  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:44.919099  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:44.918831  9022 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:44.919948  5941 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:44.923380  5939 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:44.925173  5189 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:44.926553  9022 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:44.926899  5042 connection.cc:256] RPC call timeout handler was delayed by -3.1821217840001736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20910000000000004s for a response
I20260709 12:24:44.927282  9022 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:44.927800  9022 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.928153  9022 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:44.928411  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:44.929137  5947 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:44.933688  8708 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:44.934985  8708 raft_consensus.cc:3055] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:44.935364  8708 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:44.936486  8708 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:44.888557  8986 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:44.897322  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:44.939224  8708 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:44.939774  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:44.940608  5942 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:44.954825  5220 raft_consensus.cc:2410] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:24:44.935014  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:44.961033  5944 raft_consensus.cc:2393] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:44.961575  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 436201us
W20260709 12:24:44.963498  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.964107  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.964550  5042 connection.cc:256] RPC call timeout handler was delayed by -2.5769179570002052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 12:24:44.965131  5042 connection.cc:256] RPC call timeout handler was delayed by -2.4662179570000262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
W20260709 12:24:44.965616  5042 connection.cc:256] RPC call timeout handler was delayed by -2.4158179570003995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
W20260709 12:24:44.965662  5147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1267) took 3818 ms (client timeout 3275 ms). Trace:
W20260709 12:24:44.966040  5042 connection.cc:256] RPC call timeout handler was delayed by -2.3366179570002714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
W20260709 12:24:44.966311  5147 rpcz_store.cc:269] 0709 12:24:41.147477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.147824 (+   347us) service_pool.cc:224] Handling call
0709 12:24:44.965168 (+3817344us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b5400837554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:44.965360 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966555392}
I20260709 12:24:44.974067  5941 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:44.982421  5943 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:44.988785  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:44.990129  5941 raft_consensus.cc:2410] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 12:24:44.966531  5042 connection.cc:256] RPC call timeout handler was delayed by -2.0463323480003055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
W20260709 12:24:44.991426  5042 connection.cc:256] RPC call timeout handler was delayed by -1.8672323480000159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
W20260709 12:24:44.992264  5042 connection.cc:256] RPC call timeout handler was delayed by -1.761032348000299s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
W20260709 12:24:44.993024  5042 connection.cc:256] RPC call timeout handler was delayed by -1.6458323480001127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
I20260709 12:24:44.985317  5944 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:24:45.003564  5042 connection.cc:256] RPC call timeout handler was delayed by -1.6377323480000996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2243s for a response
W20260709 12:24:45.005082  5042 connection.cc:256] RPC call timeout handler was delayed by -1.3578323480001018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2554s for a response
I20260709 12:24:45.007066  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:45.006996  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.014731  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:45.013945  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.013128  9033 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:24:45.016202  5218 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 12:24:45.018707  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.018747  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:45.019848  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.020138  5218 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:45.020977  5222 raft_consensus.cc:2368] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:24:45.016309  5216 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:45.023729  5942 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:45.024849  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:45.025727  5943 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
W20260709 12:24:45.026016  5041 connection.cc:256] RPC call timeout handler was delayed by -2.9139701040003274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 12:24:45.026547  5041 connection.cc:256] RPC call timeout handler was delayed by -2.8032701040001484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
W20260709 12:24:45.026882  5041 connection.cc:256] RPC call timeout handler was delayed by -2.7915701040001295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260709 12:24:45.027225  5041 connection.cc:256] RPC call timeout handler was delayed by -2.6736701040003936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
W20260709 12:24:45.027556  5041 connection.cc:256] RPC call timeout handler was delayed by -2.636770104000334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
W20260709 12:24:45.027881  5041 connection.cc:256] RPC call timeout handler was delayed by -2.0157701040002394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2554s for a response
I20260709 12:24:45.028921  8779 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.030071  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:45.019701  9043 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.035055  9043 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:45.036407  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 730) took 2209 ms. Trace:
I20260709 12:24:45.036795  5145 rpcz_store.cc:276] 0709 12:24:42.826773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.305211 (+1478438us) service_pool.cc:224] Handling call
0709 12:24:44.873188 (+567977us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:44.874292 (+  1104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.036149 (+161857us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b54004fa950. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:45.036281 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1293627776}
W20260709 12:24:45.038000  5899 leader_election.cc:336] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Timed out: RequestConsensusVote RPC to 127.1.123.68:43449 timed out after 1.530s (SENT)
I20260709 12:24:45.038496  5899 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:45.039171  9022 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:45.042603  9022 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:45.042687  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.044432  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.033280  5070 raft_consensus.cc:2410] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:24:45.028992  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:45.034061  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.046456  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:45.050881  5042 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:45.052893  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.055111  5216 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:45.059212  5947 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:45.048116  8975 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:45.060420  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:45.060295  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:45.061785  9001 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:45.029487  8779 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:45.062147  8917 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:45.062703  8917 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:45.063119  8917 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.062031  5941 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:24:45.067353  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 45150.2us
I20260709 12:24:45.070072  8947 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:45.070839  8947 raft_consensus.cc:697] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:45.071182  5942 raft_consensus.cc:2410] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:24:45.072310  8947 consensus_queue.cc:237] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:45.075155  8948 consensus_queue.cc:1059] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:45.076046  5218 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:45.076592  9042 raft_consensus.cc:3055] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:45.077275  9042 raft_consensus.cc:740] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:45.078900  9042 consensus_queue.cc:260] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:45.051290  9022 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:45.037436  9043 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:45.083807  5216 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:24:45.083945  8779 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:45.093142  5222 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:24:45.081653  9042 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:45.096699  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 67812.6us
W20260709 12:24:45.098596  5075 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 1061) took 4637 ms (client timeout 2907 ms). Trace:
W20260709 12:24:45.099094  5075 rpcz_store.cc:269] 0709 12:24:40.456685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.479025 (+ 22340us) service_pool.cc:224] Handling call
0709 12:24:45.093129 (+4614104us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b54000307d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:45.093446 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1976060160}
I20260709 12:24:45.102270  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:45.102905  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:45.104139  9043 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:45.105381  9000 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:45.106377  9043 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:45.112138  9043 consensus_queue.cc:1059] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:45.112301  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:45.113356  9043 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:45.117061  5223 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:45.126435  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.130595  9000 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.131381  9000 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:45.134056  9000 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:45.140034  5188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.010s	sys 0.005s
W20260709 12:24:45.140710  5188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.010s	sys 0.005s
W20260709 12:24:45.141078  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 576721us
I20260709 12:24:45.144270  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:45.147256  9042 raft_consensus.cc:3055] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:45.148044  9042 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:45.154160  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 44870.5us
I20260709 12:24:45.156905  9033 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:24:45.157513  5171 debug-util.cc:398] Leaking SignalData structure 0x7b080039c4c0 after lost signal to thread 4933
I20260709 12:24:45.157840  9033 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:45.159233  5171 debug-util.cc:398] Leaking SignalData structure 0x7b080025c700 after lost signal to thread 5100
W20260709 12:24:45.159578  5171 debug-util.cc:398] Leaking SignalData structure 0x7b0800192820 after lost signal to thread 5101
I20260709 12:24:45.163849  9066 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:45.164481  9066 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:45.165753  9066 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:45.170706  9066 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:45.171332  9033 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:24:45.176986  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52954.5us
W20260709 12:24:45.177335  5190 connection.cc:256] RPC call timeout handler was delayed by -1.9613850589998947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3275s for a response
W20260709 12:24:45.177868  5190 connection.cc:256] RPC call timeout handler was delayed by -1.3135872339998969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26320000000000005s for a response
I20260709 12:24:45.182327  9068 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:24:45.189157  5143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 729) took 2362 ms (client timeout 1997 ms). Trace:
W20260709 12:24:45.189697  5143 rpcz_store.cc:269] 0709 12:24:42.826411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.543589 (+717178us) service_pool.cc:224] Handling call
0709 12:24:45.188832 (+1645243us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b54001b6554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:45.189046 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2058514688}
I20260709 12:24:45.188630  9042 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:45.172436  5188 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:45.204066  5188 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:45.220553  8777 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:45.221109  8777 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:45.221488  8777 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:45.250051  9067 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:45.254577  9067 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:45.201797  9042 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:45.261932  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:45.262574  5068 raft_consensus.cc:3055] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:45.263016  5071 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:45.264242  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:45.263899  5068 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:45.268117  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:45.268831  5941 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.269431  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:45.270339  5076 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:24:45.272848  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:45.275442  5071 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.281360  5068 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:45.281795  5076 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:45.282550  5068 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.282855  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.283098  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.284233  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.284463  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.285094  5222 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:45.285290  5223 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:45.286005  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.286211  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:45.286870  5218 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:45.287114  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.287812  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.288393  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.289198  5221 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:24:45.294226  9065 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:45.294554  5041 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:45.300244  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.296860  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.302297  5071 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:45.310498  5941 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:45.316767  5068 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:24:45.317437  5068 pending_rounds.cc:85] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 5
I20260709 12:24:45.317811  5068 pending_rounds.cc:107] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:24:45.314049  5073 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:45.318166  5068 raft_consensus.cc:2889] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:45.197448  9070 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:24:45.314479  5188 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:45.320189  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 120997us
I20260709 12:24:45.257959  9067 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:45.322269  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.332602  5071 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:45.337481  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:45.343430  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:45.344812  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:45.346436  5941 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
W20260709 12:24:45.347123  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.351367  5941 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:24:45.358466  5067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1293) took 1168 ms. Trace:
I20260709 12:24:45.398015  9076 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:45.398702  9076 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:45.394675  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 62852us
W20260709 12:24:45.414505  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 77360.7us
I20260709 12:24:45.418099  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.418931  5218 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:24:45.420025  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 76975.5us
I20260709 12:24:45.399111  9076 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:45.414757  8951 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.359005  5067 rpcz_store.cc:276] 0709 12:24:44.190811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.197307 (+  6496us) service_pool.cc:224] Handling call
0709 12:24:44.198310 (+  1003us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:44.198913 (+   603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:44.198986 (+    73us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:44.200025 (+  1039us) log.cc:844] Serialized 26 byte log entry
0709 12:24:44.271795 (+ 71770us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:44.272038 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.272120 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:44.778231 (+506111us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b540030f820. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a84b0dbe 00007fd8a84bb930 00007fd8a84bb8d8 00007fd8a84bb858 00007fd8a84bb813 00007fd8a84baa9c 00007fd8ae3a62f6 00007fd8ae3a0aa8 00007fd8a84f77cb 00007fd8a84fa2c0
0709 12:24:45.356502 (+578271us) raft_consensus.cc:1685] finished
0709 12:24:45.356536 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.358236 (+  1700us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4306816,"thread_start_us":65243,"threads_started":1,"wal-append.queue_time_us":66933}
W20260709 12:24:45.437822  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 98797us
I20260709 12:24:45.439620  9071 consensus_queue.cc:1059] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:45.440991  9071 raft_consensus.cc:3055] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:45.441480  9071 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:45.442935  9071 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:45.444728  9071 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:45.439647  5041 leader_election.cc:400] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:24:45.446106  5041 leader_election.cc:403] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:45.447500  9070 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:24:45.448684  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 1313) took 1158 ms. Trace:
I20260709 12:24:45.448989  5076 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:24:45.449328  5076 pending_rounds.cc:85] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 7
I20260709 12:24:45.449697  5076 pending_rounds.cc:107] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:24:45.449939  5076 raft_consensus.cc:2889] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:45.418978  9067 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:45.453125  9067 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:45.472719  5042 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:45.425051  9077 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:45.475672  8965 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:45.475312  9077 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.476485  9077 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:45.478819  9066 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:45.479337  9066 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:45.479735  9066 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:45.483341  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:45.462587  8913 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.465926  9081 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:45.083331  9022 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:45.449848  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:45.471347  5190 leader_election.cc:336] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 3.275s (SENT)
I20260709 12:24:45.475395  9080 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:45.485410  5940 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:45.470454  8777 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:45.489507  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:45.449184  5070 rpcz_store.cc:276] 0709 12:24:44.290088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.294317 (+  4229us) service_pool.cc:224] Handling call
0709 12:24:45.448368 (+1154051us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b5400347454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:45.448545 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1406368512}
I20260709 12:24:45.487211  9065 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:45.486564  9087 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:24:45.498241  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 48533.2us
W20260709 12:24:45.500990  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.488308  8913 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:45.491380  9070 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:45.503599  5042 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:45.497473  9000 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:45.508123  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:45.517303  8913 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:45.509766  9000 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.518878  8777 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:45.519163  9000 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:45.519870  8777 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:45.522043  9022 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:45.518872  8913 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:45.524894  8913 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:45.525360  8913 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:45.503194  9081 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:45.527177  8975 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:45.527266  8777 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:45.529502  9081 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:45.534287  9070 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.535344  9043 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:45.541752  8975 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:45.543720  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:45.544587  8779 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:45.545106  5042 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:45.546717  5042 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:45.549935  5041 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:45.554792  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:45.555658  5223 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:45.557900  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.558676  5220 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:45.558756  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:45.559520  5218 raft_consensus.cc:2376] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11. Re-sending same reply.
I20260709 12:24:45.559738  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.560742  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:45.549490  9065 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:45.551462  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.562856  5218 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:45.562031  9065 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:45.575116  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 2.28487e+06us
I20260709 12:24:45.575543  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:45.576756  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 105482us
I20260709 12:24:45.581398  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.581795  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:45.582175  5218 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:45.582315  5220 raft_consensus.cc:2376] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14. Re-sending same reply.
I20260709 12:24:45.583088  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.583159  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.583801  5218 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:45.592257  9066 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:45.599495  9070 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:45.592891  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 46279.7us
I20260709 12:24:45.600224  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 966) took 2556 ms. Trace:
I20260709 12:24:45.541049  9076 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:45.602159  9072 raft_consensus.cc:3055] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:24:45.601764  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 53466.8us
I20260709 12:24:45.589380  5223 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:45.601150  5042 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:45.609591  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:45.610466  5217 raft_consensus.cc:2393] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 11: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:45.589380  8951 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.567916  8965 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:45.602886  9065 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:45.600692  5946 rpcz_store.cc:276] 0709 12:24:43.043363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.043578 (+   215us) service_pool.cc:224] Handling call
0709 12:24:43.044039 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.044372 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:43.044395 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:44.953728 (+1909333us) log.cc:844] Serialized 26 byte log entry
0709 12:24:45.049678 (+ 95950us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:45.049738 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.061617 (+ 11879us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:45.598678 (+537061us) spinlock_profiling.cc:243] Waited 9.81 ms on lock 0x7b54000a30a0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a84b0dbe 00007fd8a84bb930 00007fd8a84bb8d8 00007fd8a84bb858 00007fd8a84bb813 00007fd8a84baa9c 00007fd8ae3a62f6 00007fd8ae3a0aa8 00007fd8a84f77cb 00007fd8a84fa2c0
0709 12:24:45.599704 (+  1026us) raft_consensus.cc:1685] finished
0709 12:24:45.599730 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.600000 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":98,"spinlock_wait_cycles":22570880,"thread_start_us":68472,"threads_started":1,"wal-append.queue_time_us":79884}
I20260709 12:24:45.567560  9071 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:45.607869  8947 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:45.626345  8965 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:45.628216  5215 consensus_meta.cc:287] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 2.547s	user 0.016s	sys 0.015s
I20260709 12:24:45.607134  9072 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:45.640861  5215 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:45.612708  9087 raft_consensus.cc:697] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:45.645591  5899 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:45.650403  5222 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:45.647500  5149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 734) took 2814 ms (client timeout 1784 ms). Trace:
I20260709 12:24:45.646421  8947 raft_consensus.cc:697] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:45.653913  5149 rpcz_store.cc:269] 0709 12:24:42.832610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.647144 (+2814534us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:45.647311 (+   167us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:24:45.635635  9071 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:45.616755  9067 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:24:45.646828  8948 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:24:45.660977  8947 consensus_queue.cc:237] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:45.665823  9076 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:45.666641  9000 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:45.667456  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:45.668290  5943 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:45.669864  9000 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:45.657917  8965 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:45.671371  9077 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:45.671970  8965 raft_consensus.cc:697] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:45.673703  9087 consensus_queue.cc:237] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:45.677918  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:45.679913  5071 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:45.678103  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:45.646623  9042 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:45.687376  9065 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:45.687742  9066 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:45.683861  5074 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:45.691987  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:45.691403  9077 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:45.691392  9076 raft_consensus.cc:3055] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:45.704314  9076 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:45.695305  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:45.696130  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 95175.4us
I20260709 12:24:45.707152  5042 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:45.708369  8779 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:45.709476  9042 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:45.710393  8708 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:45.711261  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:45.711937  8708 raft_consensus.cc:3055] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:45.712253  9081 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:45.712499  8708 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:45.712847  9081 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:45.713692  9076 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:45.698382  5943 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:45.699641  5899 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:45.702545  5070 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:45.710530  9042 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:45.727679  9000 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:45.658087  9071 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:45.716045  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.697997  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 1127) took 2609 ms. Trace:
I20260709 12:24:45.713348  9081 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:45.731158  5188 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:45.733847  5067 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:45.724190  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:45.735219  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.682263  8965 consensus_queue.cc:237] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:45.735227  5215 rpcz_store.cc:276] 0709 12:24:43.078746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.078927 (+   181us) service_pool.cc:224] Handling call
0709 12:24:45.682622 (+2603695us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:45.728366  9000 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.750710  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:45.755119  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:45.728220  8913 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:45.757720  9093 log.cc:927] Time spent T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:24:45.758693  8913 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:45.747434  8951 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:45.764463  8905 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:45.765110  8905 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:45.766172  8905 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:45.771659  9001 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:45.772377  9001 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:45.774858  9065 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:45.765264  5945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 950) took 3325 ms (client timeout 1629 ms). Trace:
I20260709 12:24:45.779035  8951 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:45.779510  9076 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:45.777230  5945 rpcz_store.cc:269] 0709 12:24:42.440186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.443128 (+  2942us) service_pool.cc:224] Handling call
0709 12:24:45.764873 (+3321745us) spinlock_profiling.cc:243] Waited 91.4 ms on lock 0x7b540023ffd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:45.765061 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210158464}
I20260709 12:24:45.785372  5943 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:45.787405  5041 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:45.788517  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:45.754087  9000 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:45.794427  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:45.820524  5946 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:45.822971  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:45.729403  8708 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:45.823607  5070 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:45.793907  9042 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:45.780189  8951 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:45.881258  5068 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
W20260709 12:24:45.882313  5219 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 2.818s	user 0.017s	sys 0.009s
I20260709 12:24:45.886072  5219 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:45.886700  9033 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.887647  9033 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:45.889358  5190 leader_election.cc:336] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 3.097s (SENT)
I20260709 12:24:45.890079  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:45.891808  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 1125) took 2830 ms. Trace:
I20260709 12:24:45.892366  5219 rpcz_store.cc:276] 0709 12:24:43.061714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.061937 (+   223us) service_pool.cc:224] Handling call
0709 12:24:45.891703 (+2829766us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:45.892956  8777 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:24:45.875481  8917 consensus_meta.cc:287] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.812s	user 0.014s	sys 0.001s
I20260709 12:24:45.881248  5899 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:45.894659  8917 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:45.895227  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 68545.1us
I20260709 12:24:45.895368  8965 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:45.895959  8965 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:45.896126  5068 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:45.897111  8917 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:45.896899  8965 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:45.897413  5068 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: []
W20260709 12:24:45.897655  9096 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.207s	user 0.000s	sys 0.002s
I20260709 12:24:45.897776  5075 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:45.870208  9042 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:45.921986  5067 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:45.923118  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:45.921775  5071 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:45.923781  5219 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:45.932843  9071 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:45.939182  5070 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:45.941785  5071 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:45.921710  8708 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:45.974893  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 732) took 3147 ms. Trace:
W20260709 12:24:45.979072  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 4.66951e+06us
I20260709 12:24:45.983523  5073 raft_consensus.cc:1275] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:45.944339  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:45.949736  9092 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:24:45.950766  9042 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:45.951687  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:45.960212  9000 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:45.961457  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:45.994405  5072 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:45.962338  9001 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:45.994825  5944 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:45.971220  9071 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:45.989923  5219 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:24:45.975637  5144 rpcz_store.cc:276] 0709 12:24:42.831801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.474447 (+1642646us) service_pool.cc:224] Handling call
0709 12:24:44.497561 (+ 23114us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.498085 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:44.787834 (+289749us) log.cc:844] Serialized 14 byte log entry
0709 12:24:45.974342 (+1186508us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:45.974433 (+    91us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:45.974512 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.974631 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.974824 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1165854,"prepare.queue_time_us":100357,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":77,"thread_start_us":309089,"threads_started":2,"wal-append.queue_time_us":19996,"wal-append.run_cpu_time_us":1743,"wal-append.run_wall_time_us":1008800}
I20260709 12:24:45.992470  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:45.995766  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:45.963138  5148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1149) took 3397 ms (client timeout 2234 ms). Trace:
I20260709 12:24:45.992588  5220 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:45.995985  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:45.997179  5071 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:45.993873  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:45.996487  5148 rpcz_store.cc:269] 0709 12:24:42.574700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.574841 (+   141us) service_pool.cc:224] Handling call
0709 12:24:45.962694 (+3387853us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54003ae3d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:45.962912 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091640192}
I20260709 12:24:46.003545  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:46.005162  5171 debug-util.cc:398] Leaking SignalData structure 0x7b080035a720 after lost signal to thread 5174
W20260709 12:24:46.005601  5171 debug-util.cc:398] Leaking SignalData structure 0x7b080038c3e0 after lost signal to thread 5175
W20260709 12:24:46.006422  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.011318  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:46.012290  5069 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:46.012749  5171 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd7e0 after lost signal to thread 5248
I20260709 12:24:46.014693  5948 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:46.015841  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 971) took 1103 ms. Trace:
W20260709 12:24:46.016045  5171 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0b20 after lost signal to thread 5973
W20260709 12:24:46.016680  5171 debug-util.cc:398] Leaking SignalData structure 0x7b08001bdc80 after lost signal to thread 5974
W20260709 12:24:46.017426  5171 debug-util.cc:398] Leaking SignalData structure 0x7b0800099840 after lost signal to thread 8827
W20260709 12:24:46.018416  5171 debug-util.cc:398] Leaking SignalData structure 0x7b0800276140 after lost signal to thread 8939
W20260709 12:24:46.018920  5171 debug-util.cc:398] Leaking SignalData structure 0x7b080033cce0 after lost signal to thread 8952
W20260709 12:24:46.019273  5171 debug-util.cc:398] Leaking SignalData structure 0x7b0800363f80 after lost signal to thread 8961
I20260709 12:24:46.019481  9000 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:46.019641  5171 debug-util.cc:398] Leaking SignalData structure 0x7b08002115a0 after lost signal to thread 8969
I20260709 12:24:46.021041  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.021849  5071 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:46.023149  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:46.024235  9001 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:46.024056  5221 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:46.016181  5948 rpcz_store.cc:276] 0709 12:24:44.912726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.917663 (+  4937us) service_pool.cc:224] Handling call
0709 12:24:46.015609 (+1097946us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400009e54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:46.015776 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2520170752}
W20260709 12:24:46.024798  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.033310  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:46.044296  8982 raft_consensus.cc:2764] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:24:46.044454  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:46.045895  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:46.047039  8400 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.047894  9056 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:46.047761  8400 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:46.048504  9056 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:46.055373  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:46.049732  9022 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:24:46.059156  5068 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:46.059681  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:46.061007  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:46.061684  5068 raft_consensus.cc:2376] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12. Re-sending same reply.
I20260709 12:24:46.066143  5945 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:46.055472  8917 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:46.069777  9072 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:46.071235  9072 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:46.073141  5942 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:46.080863  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:46.087910  5221 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:46.101616  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:46.104924  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:46.106694  8647 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:46.106968  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:46.107206  8647 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:46.107910  5946 raft_consensus.cc:2393] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:46.108192  8647 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:46.109099  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:46.109604  8647 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:46.111091  5069 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:46.112434  5188 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:46.113871  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:46.105057  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:46.117285  8777 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:46.049499  9056 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:46.123966  9033 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:46.125185  5939 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:24:46.128722  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1274) took 4951 ms. Trace:
W20260709 12:24:46.114678  5224 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 695) took 1843 ms (client timeout 1814 ms). Trace:
I20260709 12:24:46.129276  5150 rpcz_store.cc:276] 0709 12:24:41.177414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.177613 (+   199us) service_pool.cc:224] Handling call
0709 12:24:41.216356 (+ 38743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.811545 (+3595189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.128363 (+1316818us) spinlock_profiling.cc:243] Waited 3.59 s on lock 0x7b540045a954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:46.128618 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8261014784}
W20260709 12:24:46.134213  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.131716  5224 rpcz_store.cc:269] 0709 12:24:44.295978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.114289 (+1818311us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:46.114437 (+   148us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:24:46.136023  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.137005  5069 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:46.138234  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:46.140213  5068 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:46.141691  9011 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:46.055344  9081 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:46.135390  8777 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:46.145923  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.147119  5222 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:46.146621  8777 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:46.147523  9081 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:46.149199  8975 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:46.149510  5899 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:46.149892  8975 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:46.150902  8975 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:46.151532  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:46.152247  5075 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:46.153898  9000 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:46.154233  8400 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:46.154484  9000 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:46.156018  9095 log.cc:927] Time spent T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.290s	user 0.002s	sys 0.000s
I20260709 12:24:46.142536  8982 consensus_queue.cc:1059] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:24:46.160290  8982 raft_consensus.cc:3055] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:46.160630  8982 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:46.161569  8982 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:46.162842  8982 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:46.166229  5900 leader_election.cc:336] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.553s (SENT)
I20260709 12:24:46.155691  9000 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:46.148211  5942 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:24:46.168373  5220 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:46.168322  5900 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:46.168236  5075 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:46.172611  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:46.172930  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:46.167821  8735 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:24:46.199376  5245 debug-util.cc:398] Leaking SignalData structure 0x7b080002f880 after lost signal to thread 1517
I20260709 12:24:46.228109  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.225405  8819 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.227221  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.228945  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.248238  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.249001  5068 raft_consensus.cc:2410] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 12:24:46.204532  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.249863  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.222162  9022 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:46.227017  5071 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:46.237145  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 735) took 3404 ms. Trace:
I20260709 12:24:46.246029  8905 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:46.268713  5041 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:46.266125  5146 rpcz_store.cc:276] 0709 12:24:42.832933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.517592 (+1684659us) service_pool.cc:224] Handling call
0709 12:24:44.525856 (+  8264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.982976 (+1457120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.983008 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.983658 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.103178 (+119520us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:46.103211 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.103294 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.200691 (+ 97397us) raft_consensus.cc:1685] finished
0709 12:24:46.200710 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.237078 (+ 36368us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1445486,"raft.queue_time_us":1698588,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":149,"thread_start_us":119095,"threads_started":1,"wal-append.queue_time_us":160887}
I20260709 12:24:46.265399  5215 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:24:46.170266  9011 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:46.227415  8496 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:46.240895  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.242362  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:46.246208  5944 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:46.286370  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.246235  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.246837  5067 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:46.287317  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:46.251048  9022 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:46.251376  8986 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.258054  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.311021  5070 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 9: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:46.294976  5073 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:46.240159  8946 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:24:46.250977  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 56031.3us
W20260709 12:24:46.277902  5042 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.735s (SENT)
I20260709 12:24:41.780125  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:46.247231  5245 debug-util.cc:398] Leaking SignalData structure 0x7b080011b000 after lost signal to thread 1520
I20260709 12:24:46.288336  5071 raft_consensus.cc:2376] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11. Re-sending same reply.
I20260709 12:24:46.290664  8788 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:46.317237  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.308650  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:46.261286  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.314232  8708 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:46.319658  5042 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:46.321650  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.322371  5076 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:46.322393  5072 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:46.318478  5245 debug-util.cc:398] Leaking SignalData structure 0x7b08002afe40 after lost signal to thread 4933
W20260709 12:24:46.326889  5245 debug-util.cc:398] Leaking SignalData structure 0x7b0800240180 after lost signal to thread 5100
W20260709 12:24:46.327340  5245 debug-util.cc:398] Leaking SignalData structure 0x7b0800240380 after lost signal to thread 5101
W20260709 12:24:46.329088  5245 debug-util.cc:398] Leaking SignalData structure 0x7b08003a5240 after lost signal to thread 5174
W20260709 12:24:46.329488  5245 debug-util.cc:398] Leaking SignalData structure 0x7b08003a51a0 after lost signal to thread 5175
I20260709 12:24:46.318884  5074 raft_consensus.cc:2376] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14. Re-sending same reply.
W20260709 12:24:46.332252  5245 debug-util.cc:398] Leaking SignalData structure 0x7b0800158080 after lost signal to thread 5248
W20260709 12:24:46.335737  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45621.5us
I20260709 12:24:46.293013  9042 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:24:46.324359  5900 leader_election.cc:336] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.677s (SENT)
I20260709 12:24:46.318409  8986 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:46.347194  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 232524us
W20260709 12:24:46.348431  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.351276  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.351795  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:46.375782  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.348902  9001 consensus_queue.cc:1059] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:24:46.268980  8819 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:46.351302  8917 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:46.371601  5900 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:46.323908  8905 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:24:46.352306  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.379427  9122 consensus_queue.cc:1059] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:46.386344  5070 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:46.385375  5215 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:24:46.405086  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 54469.8us
I20260709 12:24:46.346081  5216 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:46.391321  9087 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:46.391378  5245 debug-util.cc:398] Leaking SignalData structure 0x7b08003aa5c0 after lost signal to thread 5973
W20260709 12:24:46.406226  5245 debug-util.cc:398] Leaking SignalData structure 0x7b08002c76c0 after lost signal to thread 5974
I20260709 12:24:46.406962  5074 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:46.407284  5071 raft_consensus.cc:2393] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:46.371722  8788 raft_consensus.cc:3055] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:46.418885  5222 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:46.420333  9081 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.420904  9081 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:46.423265  9081 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:46.423619  8819 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:46.409080  8788 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:46.425041  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.425128  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 147454us
I20260709 12:24:46.400672  8975 consensus_queue.cc:1059] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:46.400961  8946 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:46.434825  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 110502us
W20260709 12:24:46.437340  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 53957.3us
I20260709 12:24:46.389195  8951 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:24:46.445528  5042 leader_election.cc:336] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.234s (SENT)
I20260709 12:24:46.447777  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:46.448525  5070 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:46.449441  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:46.441248  8788 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:46.452387  8788 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:46.404357  9022 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:46.462757  9001 consensus_queue.cc:1048] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:46.458415  8946 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:46.408110  9011 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:46.466461  8986 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:46.467628  4898 catalog_manager.cc:5697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 9 to 10, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 10 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:24:46.475939  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:46.478896  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:46.441972  9124 raft_consensus.cc:3055] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:46.476795  9132 raft_consensus.cc:3055] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:46.486269  9132 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:46.486436  9092 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:46.487423  9132 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:46.434267  9025 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:24:46.490867  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.491425  5041 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:46.472946  9122 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:46.449440  9126 raft_consensus.cc:3055] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:46.497227  5215 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:24:46.475939  9075 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.498689  9075 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:46.501060  9075 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:46.501914  8905 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:24:46.502243  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 59878.8us
I20260709 12:24:46.494431  8948 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:46.505141  8948 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:46.495990  5900 leader_election.cc:336] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.784s (SENT)
W20260709 12:24:46.506217  5900 leader_election.cc:336] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.825s (SENT)
I20260709 12:24:46.506721  9001 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
W20260709 12:24:46.510241  5900 leader_election.cc:336] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.831s (SENT)
I20260709 12:24:46.510797  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:46.486105  9124 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:46.511315  5188 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:46.511986  5900 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.836s (SENT)
I20260709 12:24:46.512506  9124 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:46.513748  9056 consensus_queue.cc:1059] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:46.514607  8951 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:24:46.515771  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 69741.9us
I20260709 12:24:46.512529  5900 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:46.516920  5898 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:46.517966  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:46.517875  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:46.519089  5944 raft_consensus.cc:2410] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 13 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:24:46.519145  5222 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:46.488544  8965 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:46.525480  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:46.526532  8986 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:46.527801  9131 raft_consensus.cc:3055] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:46.494843  9076 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:46.529089  5898 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:46.528270  9131 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:46.530848  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.533180  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:46.533586  9131 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:46.506345  8948 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:46.535056  9131 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:46.541576  9132 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:46.542120  5073 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:46.495235  9126 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:46.549441  5942 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:46.515105  8951 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.514196  9124 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:46.522209  9001 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:46.557047  5945 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:46.530274  9122 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:46.566721  9002 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:46.558228  9126 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:46.568922  9126 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:46.569119  8752 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:46.569626  8752 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:46.570575  8752 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:46.572268  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.592546  5041 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:46.573770  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 997369us
W20260709 12:24:46.608202  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 79209us
I20260709 12:24:46.607409  9071 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.613610  8917 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:46.524266  9075 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:24:46.571834  8951 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:46.616665  5943 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:46.618647  5898 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:46.553879  8905 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:46.572105  8965 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:46.626478  5945 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:46.588918  9002 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:46.627822  5900 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.881s (SENT)
W20260709 12:24:46.632061  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.616577  8400 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:46.641464  8788 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.642134  8788 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:46.614282  8917 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:46.646081  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:46.615090  8960 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:46.655303  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.656627  5898 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:46.656839  5074 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:46.620843  9075 consensus_queue.cc:1243] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:46.651839  8819 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.620677  9071 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:46.655498  8946 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:46.688061  5115 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:46.664767  9122 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:24:46.696434  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 704784us
I20260709 12:24:46.707895  9014 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 12:24:46.645496  9002 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:46.686544  8905 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:46.715509  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 45630.8us
I20260709 12:24:46.701638  9122 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.674417  9033 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:24:46.665681  8960 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:46.674252  9118 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:24:46.673966  9056 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:24:46.686487  9011 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
W20260709 12:24:46.731240  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 74690.5us
W20260709 12:24:46.731617  5900 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.997s (SENT)
I20260709 12:24:46.740773  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:46.741649  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.743932  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:46.750670  5114 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:46.702029  8819 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:46.761420  5222 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:46.744269  9122 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:46.783258  5898 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:46.786067  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 66894.4us
I20260709 12:24:46.709793  8917 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:46.761646  9127 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:46.750088  8400 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:46.806672  5222 pending_rounds.cc:85] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 4
I20260709 12:24:46.807200  5222 pending_rounds.cc:107] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:24:46.807529  5222 raft_consensus.cc:2889] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:46.755406  8946 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:46.795006  9014 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:24:46.826663  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 43531.5us
I20260709 12:24:46.827983  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:46.805085  9127 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:46.798440  8496 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:46.803862  9076 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:46.833858  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.820277  8400 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:46.843750  5898 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:24:46.851675  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.846513  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.803797  8905 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:24:46.856196  5146 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:24:46.835803  9146 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:46.843961  8946 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:46.873709  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.877147  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 134948us
W20260709 12:24:46.858974  9136 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:24:46.847520  8788 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:46.972716  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 129066us
I20260709 12:24:46.850517  9014 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:46.866389  9071 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:46.988575  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:46.990409  5898 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:46.996557  5150 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:46.997771  8819 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:46.844974  8400 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:47.011312  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.013490  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:47.014010  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 303102us
I20260709 12:24:47.014389  5146 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
W20260709 12:24:47.011065  9139 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.299s	user 0.000s	sys 0.002s
I20260709 12:24:47.030361  5898 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:46.979434  8788 consensus_queue.cc:1059] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:24:46.994949  8946 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:47.050989  5073 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:47.052846  9033 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:47.053184  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:47.054246  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.971172  9127 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:47.064695  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.030596  8975 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.082008  5067 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:47.078142  5148 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:47.084442  4898 catalog_manager.cc:5697] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 8 to 11, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 11 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.067657  8905 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:47.079326  8960 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:47.092953  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.100937  5150 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:47.080310  9033 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:47.087550  9134 raft_consensus.cc:2764] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:24:47.115964  9147 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 12:24:47.092108  9002 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:47.129585  8951 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:47.130717  9124 consensus_meta.cc:287] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.558s	user 0.013s	sys 0.000s
I20260709 12:24:47.137312  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:47.142236  9138 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 12:24:47.081418  9014 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:47.138142  5150 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:47.099869  8905 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.151386  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:47.153806  5148 raft_consensus.cc:2376] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12. Re-sending same reply.
I20260709 12:24:47.153879  8838 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:47.166992  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:47.168015  5148 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:47.170387  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:47.170704  8951 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:47.172113  5114 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:47.173023  8838 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:47.173786  8838 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:47.173908  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:47.174240  8838 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.174918  8838 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:47.175716  9149 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:47.176420  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:47.177510  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.177918  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.178411  5149 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:47.178493  5145 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:47.177299  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.180060  5146 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:47.180296  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.179324  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.180989  5149 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:47.181318  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.181977  5147 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
W20260709 12:24:47.182449  9132 consensus_meta.cc:287] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.640s	user 0.013s	sys 0.000s
I20260709 12:24:47.181978  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:47.177414  5144 raft_consensus.cc:2410] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:24:47.184010  5222 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:47.193329  5941 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:47.153231  8905 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:47.195881  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.196023  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.195986  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.201025  5147 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:47.197782  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:47.201984  5939 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:47.198521  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:47.176299  9076 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:47.200382  5143 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:47.203861  5222 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:47.222601  8400 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:24:47.227555  9163 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:24:47.227723  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:47.232416  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.232797  5143 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:47.228543  5190 leader_election.cc:336] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.066s (SENT)
I20260709 12:24:47.233702  9000 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:24:47.234633  5114 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:47.236141  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:47.242224  8838 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:47.246762  9042 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.247365  9042 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:47.249714  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:47.250743  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.251449  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.252099  5900 leader_election.cc:336] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.183s (SENT)
I20260709 12:24:47.252499  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:47.253109  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:47.253290  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:47.253827  5073 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:47.254019  5075 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
W20260709 12:24:47.255515  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.259001  5143 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:24:47.266021  8708 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:47.266687  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:47.252959  9149 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:24:47.270785  5939 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:47.271656  5939 raft_consensus.cc:606] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:47.272027  5939 raft_consensus.cc:614] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:47.271329  9168 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.281754  9168 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:47.285686  9168 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:47.239749  9126 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.669s	user 0.014s	sys 0.000s
I20260709 12:24:47.239095  8960 raft_consensus.cc:2764] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:24:47.311604  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 59539us
I20260709 12:24:47.312515  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:47.317970  5143 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:47.253762  8400 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:47.223403  9132 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 12:24:47.307869  9000 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:47.327209  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:47.337488  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:47.338534  5148 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:47.339123  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 117059us
W20260709 12:24:47.340615  5190 leader_election.cc:336] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.102s (SENT)
I20260709 12:24:47.341037  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:47.346921  8946 raft_consensus.cc:3055] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:47.347460  8946 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:47.348477  8946 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:47.350206  9165 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:24:47.353689  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.362296  8838 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:47.361196  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:47.366091  5072 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:47.374686  5070 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:47.375003  5075 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:47.376204  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:47.382668  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:47.383596  5069 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:47.387820  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:47.391649  5143 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:47.394004  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:47.397173  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:47.401444  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:47.350217  9033 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:47.409010  5939 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:47.409938  5939 raft_consensus.cc:606] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:47.410442  5939 raft_consensus.cc:614] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:47.412614  5072 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:47.386932  8946 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:47.417788  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.388584  8917 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.422050  5143 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:47.426738  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.428171  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.394497  8960 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.402330  9076 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.436570  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:47.439113  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:47.446970  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:47.448431  5076 raft_consensus.cc:2393] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:47.448386  5143 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:47.422448  8838 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.428210  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:47.456413  5072 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:47.452016  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.453994  5075 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
W20260709 12:24:47.447788  9122 consensus_meta.cc:287] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.639s	user 0.014s	sys 0.000s
I20260709 12:24:47.457137  5147 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:47.436056  9153 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.460717  5148 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:47.441690  9011 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:47.448272  9076 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:47.466503  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:47.469776  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:47.470793  5900 leader_election.cc:336] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.198s (SENT)
I20260709 12:24:47.470976  5069 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:24:47.471318  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:47.469805  5072 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:24:47.473763  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:47.474112  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:47.433714  8917 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:47.475744  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:47.476063  5072 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:47.476082  5069 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:47.479390  5076 raft_consensus.cc:2393] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:24:47.482482  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 141882us
I20260709 12:24:47.485518  5075 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 12:24:47.495059  5190 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.187s (SENT)
I20260709 12:24:47.496271  5942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:47.497133  5942 raft_consensus.cc:606] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:24:47.497597  5942 raft_consensus.cc:614] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:47.466082  9014 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:47.504753  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.505491  5148 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:47.442958  8960 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:47.500334  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 61362.7us
I20260709 12:24:47.500878  5070 raft_consensus.cc:2410] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:24:47.503655  5072 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:47.457958  9122 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:47.516443  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.500945  8752 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:47.501206  9132 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:47.467834  8838 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:47.526165  5069 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:47.530072  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.531895  5148 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
W20260709 12:24:47.535743  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 80453.6us
I20260709 12:24:47.468909  9153 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:47.537007  5188 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:47.539633  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:47.541610  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.542325  5148 raft_consensus.cc:2376] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13. Re-sending same reply.
I20260709 12:24:47.543960  8917 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:47.544683  5216 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:47.543970  5215 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:47.547459  8838 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:47.555565  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:47.556783  5215 raft_consensus.cc:2410] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:24:47.557468  5216 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:47.559219  5143 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:47.561213  9132 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:47.561677  9164 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:24:47.566461  5072 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:47.567963  5144 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:47.568507  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:47.569600  5072 raft_consensus.cc:2368] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13.
I20260709 12:24:47.571688  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:47.572081  9076 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:47.572696  5224 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:24:47.576695  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:47.576953  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:47.577455  5219 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:47.578017  5218 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:47.578626  9122 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:47.580953  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 97000.7us
I20260709 12:24:47.576987  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:47.592367  5188 leader_election.cc:400] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13.
I20260709 12:24:47.592794  5188 leader_election.cc:403] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:47.602973  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 998) took 1079 ms. Trace:
I20260709 12:24:47.603626  5941 rpcz_store.cc:276] 0709 12:24:46.523813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.524118 (+   305us) service_pool.cc:224] Handling call
0709 12:24:47.602706 (+1078588us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b5400006254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:47.602889 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1492626304}
I20260709 12:24:47.607982  5148 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:47.607030  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:47.612190  5945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:47.613186  5945 raft_consensus.cc:606] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:47.613632  5945 raft_consensus.cc:614] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:47.619374  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 126210us
W20260709 12:24:47.619603  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 112303us
I20260709 12:24:47.580181  9033 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:47.616324  8965 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:47.600966  9164 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 12:24:47.637864  8975 consensus_meta.cc:287] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.553s	user 0.012s	sys 0.000s
I20260709 12:24:47.608958  9132 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:24:47.654590  8960 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:47.654991  9178 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:24:47.601662  9168 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:24:47.655512  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:47.657598  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:47.658479  9153 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:47.660948  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:47.665145  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:47.679062  5944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:47.679912  5944 raft_consensus.cc:606] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:47.601464  9144 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:24:47.672892  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:47.675630  5218 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:47.680661  5222 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:47.685822  5221 raft_consensus.cc:2393] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:47.694098  4898 catalog_manager.cc:5697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 9 to 11, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 11 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.702415  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:47.703415  5942 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:47.703639  8400 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:24:47.705844  4899 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43226 (request call id 130) took 32307 ms (client timeout 15000 ms). Trace:
I20260709 12:24:47.706637  8905 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:47.671492  8975 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:47.696439  4898 catalog_manager.cc:5697] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.723847  5150 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:47.732267  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:47.735693  5945 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:47.746603  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:47.747740  5943 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:47.750633  5942 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:47.728933  5041 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:47.760689  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:47.760370  4898 catalog_manager.cc:5697] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 7 to 9, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 9 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:47.770934  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 150155us
I20260709 12:24:47.770880  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:47.767933  9132 consensus_queue.cc:1059] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 12:24:47.791520  9140 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:24:47.792274  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:47.792702  8965 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:47.793932  8965 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:47.794322  8965 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:47.795341  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:47.794979  4898 catalog_manager.cc:5697] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 7 to 10, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 10 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.795976  5114 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:47.797585  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:47.807017  4898 catalog_manager.cc:5697] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:47.809839  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 171910us
I20260709 12:24:47.814363  5041 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:47.795014  9138 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:24:47.822443  5190 leader_election.cc:336] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.059s (SENT)
I20260709 12:24:47.819818  5945 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:47.813115  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:47.826355  5939 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:47.782629  9145 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:47.833647  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:47.830739  5941 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:47.737946  9042 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:47.839926  4898 catalog_manager.cc:5697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.817446  9033 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:47.850833  4898 catalog_manager.cc:5697] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 5 to 8, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 8 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
W20260709 12:24:47.706317  4899 rpcz_store.cc:269] 0709 12:24:15.398416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.401871 (+  3455us) service_pool.cc:224] Handling call
0709 12:24:47.705525 (+32303654us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:34.543878 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:34.618438 (+ 74560us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:34.618534 (+    96us) write_op.cc:435] Acquired schema lock
0709 12:24:34.618565 (+    31us) tablet.cc:662] Decoding operations
0709 12:24:34.620140 (+  1575us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:34.620326 (+   186us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:34.620372 (+    46us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:24:34.647981 (+ 27609us) tablet.cc:701] Row locks acquired
0709 12:24:34.648051 (+    70us) write_op.cc:260] PREPARE: finished
0709 12:24:34.657466 (+  9415us) write_op.cc:270] Start()
0709 12:24:34.667265 (+  9799us) write_op.cc:276] Timestamp: P: 1783599874653469 usec, L: 0
0709 12:24:34.667296 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:24:34.668097 (+   801us) log.cc:844] Serialized 3618 byte log entry
0709 12:24:35.054962 (+386865us) op_driver.cc:464] REPLICATION: finished
0709 12:24:36.738606 (+1683644us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b1800136c60. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:36.738792 (+   186us) write_op.cc:301] APPLY: starting
0709 12:24:36.738913 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:36.739137 (+   224us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:36.739174 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:36.740455 (+  1281us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:36.740773 (+   318us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:24:36.740823 (+    50us) write_op.cc:312] APPLY: finished
0709 12:24:36.741183 (+   360us) log.cc:844] Serialized 70 byte log entry
0709 12:24:37.120863 (+379680us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:37.130258 (+  9395us) write_op.cc:454] Released schema lock
0709 12:24:37.150141 (+ 19883us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:37.160557 (+ 10416us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:37.160595 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2225158,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":1426662,"apply.run_cpu_time_us":40753,"apply.run_wall_time_us":421698,"num_ops":9,"prepare.queue_time_us":185420,"prepare.run_cpu_time_us":24593,"prepare.run_wall_time_us":222963,"raft.queue_time_us":119260,"raft.run_cpu_time_us":31450,"raft.run_wall_time_us":1523872,"replication_time_us":339357,"spinlock_wait_cycles":29461888,"thread_start_us":1902817,"threads_started":5,"wal-append.queue_time_us":761678,"wal-append.run_cpu_time_us":15656,"wal-append.run_wall_time_us":2138012}]]}
I20260709 12:24:47.832261  8960 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:47.825299  9153 raft_consensus.cc:2764] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:24:47.879153  4898 catalog_manager.cc:5697] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:24:47.833094  9145 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.888057  5073 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 12. Ops: []
I20260709 12:24:47.890010  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:47.895390  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 123452us
I20260709 12:24:47.815819  9164 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:24:47.844540  8752 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.836522  8838 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.902347  5900 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:47.907896  5114 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:47.905175  5224 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:24:47.831298  9022 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:24:47.795205  8965 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:47.922482  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:47.922658  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 104540us
I20260709 12:24:47.924330  4898 catalog_manager.cc:5697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 9 to 13, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 13 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.898437  9145 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:47.910784  8496 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:47.906616  9000 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.953428  5041 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:47.962491  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.963320  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.964177  4898 catalog_manager.cc:5697] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 6, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 6 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.990303  4898 catalog_manager.cc:5697] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 6 to 10, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 10 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.992159  4898 catalog_manager.cc:5697] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.915961  9056 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13.
I20260709 12:24:47.884163  8400 consensus_queue.cc:1059] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 12:24:47.915706  8960 raft_consensus.cc:697] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:47.904155  8975 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:24:47.994786  5147 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:47.995569  5041 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:47.998481  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 96224.9us
I20260709 12:24:48.002249  5900 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:47.951223  9014 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:47.948964  8838 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:47.952828  8752 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:48.041960  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:47.986001  9097 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.129s
I20260709 12:24:48.039225  9033 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:48.048992  9189 log.cc:927] Time spent T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:24:48.051093  8838 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:48.051857  4898 catalog_manager.cc:5697] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:48.052937  9153 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:48.053362  9153 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.053804  9153 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:48.053390  4898 catalog_manager.cc:5697] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:48.055644  4898 catalog_manager.cc:5697] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 6 to 7, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 7 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:48.056427  5222 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:48.042730  9193 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:48.057559  4898 catalog_manager.cc:5697] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 5 to 11, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 11 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:48.058318  9042 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:48.026337  8965 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:48.049083  9097 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.073984  9025 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.076323  9194 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:48.022035  8905 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:48.080334  4898 catalog_manager.cc:5697] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 10 to 12, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 12 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
W20260709 12:24:48.082159  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 185594us
W20260709 12:24:48.095902  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 173505us
I20260709 12:24:48.058331  9193 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.000952  9000 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:48.112067  4898 catalog_manager.cc:5697] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 5 to 7, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 7 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:48.115204  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 113015us
I20260709 12:24:48.116721  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.119742  4898 catalog_manager.cc:5697] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 5 to 8, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 8 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
W20260709 12:24:48.095458  8946 consensus_meta.cc:287] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.664s	user 0.011s	sys 0.003s
I20260709 12:24:48.121945  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.125753  5148 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:48.080021  8838 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:48.122710  5900 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:48.127072  9207 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:48.127916  9207 raft_consensus.cc:697] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:48.128883  9157 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.979s	user 0.007s	sys 0.004s
W20260709 12:24:48.130146  9156 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:24:48.130792  5114 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:48.131808  9155 log.cc:927] Time spent T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:24:48.132498  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:48.133920  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.134630  5149 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:48.098166  9097 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:48.138428  9097 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:48.139015  8975 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:48.139559  8975 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:48.140013  8975 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:48.097692  9092 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:24:48.142938  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.144014  5144 raft_consensus.cc:2410] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:24:48.095363  9194 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.106846  9193 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:48.153937  5114 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:48.160316  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:48.026883  9014 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:48.258739  9097 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:48.234328  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:48.230005  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1005) took 1593 ms. Trace:
I20260709 12:24:48.161341  9132 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:48.225589  9149 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:48.243300  9210 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:48.094867  9025 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:48.324678  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:48.259202  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 192069us
I20260709 12:24:48.487576  5146 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:48.489933  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.490687  5148 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:48.491590  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.492268  5148 raft_consensus.cc:2393] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:48.492959  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:48.372000  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.494042  5147 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:48.149570  8960 consensus_queue.cc:237] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:48.386292  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 929) took 1113 ms. Trace:
I20260709 12:24:48.498380  5948 rpcz_store.cc:276] 0709 12:24:47.273047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.273240 (+   193us) service_pool.cc:224] Handling call
0709 12:24:47.314352 (+ 41112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.417320 (+102968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.417380 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:47.417441 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:47.417487 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.385890 (+968403us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400004954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:48.385967 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.386196 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236181248}
I20260709 12:24:48.242136  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:48.500531  5944 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:48.197196  8946 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:48.501505  8946 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:48.502108  8946 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:48.502383  9171 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 1.036s	user 0.003s	sys 0.005s
I20260709 12:24:48.130864  9122 raft_consensus.cc:3055] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:48.503840  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.164947  8400 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:48.376566  9217 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:48.381177  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.188621  5150 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:48.326256  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:48.519627  5144 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:48.165005  9194 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:48.215713  5224 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:48.129172  9207 consensus_queue.cc:237] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:48.196154  8838 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:24:48.390013  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.437592  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.452178  9216 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.452891  9204 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.453116  5141 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:24:48.453725  9218 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.458683  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 1085) took 1092 ms. Trace:
I20260709 12:24:48.598156  5070 rpcz_store.cc:276] 0709 12:24:47.366262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.375603 (+  9341us) service_pool.cc:224] Handling call
0709 12:24:48.458384 (+1082781us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54003570d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:48.458595 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":243378048}
W20260709 12:24:48.457024  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 60482.9us
I20260709 12:24:48.363775  8752 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:48.325904  9192 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 12:24:48.606670  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:48.322364  9205 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:24:48.288003  8496 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:48.611671  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:48.371865  9211 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:48.321451  9190 log.cc:927] Time spent T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
I20260709 12:24:48.189095  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:48.181551  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 59662.9us
I20260709 12:24:48.436000  9153 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:48.462872  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:48.483881  9215 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.471261  5946 rpcz_store.cc:276] 0709 12:24:46.614054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.620757 (+  6703us) service_pool.cc:224] Handling call
0709 12:24:46.648135 (+ 27378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.665136 (+ 17001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.665165 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.676144 (+ 10979us) log.cc:844] Serialized 26 byte log entry
0709 12:24:47.038404 (+362260us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:47.038471 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.038587 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.331953 (+293366us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b4800408258. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ad464aa1 00007fd8ad45bb37 00007fd8ad45b9ca 00007fd8ad55326f 00007fd8ad552f37 00007fd8ad56755c 00007fd8ad567508 00007fd8ad567488 00007fd8ad567443 00007fd8ad5666ec 00007fd8ac490a96
0709 12:24:48.128134 (+796181us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:48.207031 (+ 78897us) raft_consensus.cc:1685] finished
0709 12:24:48.207065 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.207315 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7668,"raft.queue_time_us":9078,"raft.run_cpu_time_us":5550,"raft.run_wall_time_us":669526,"spinlock_wait_cycles":20728960,"thread_start_us":310488,"threads_started":1,"wal-append.queue_time_us":409697}
I20260709 12:24:48.481810  9097 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:48.485913  9149 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:48.507846  5146 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:48.596727  9000 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:48.429719  9145 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:48.600291  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 840) took 1552 ms. Trace:
W20260709 12:24:48.604302  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 88249.7us
I20260709 12:24:48.513794  9122 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:48.515951  8400 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:48.622723  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.627840  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:48.628727  5939 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:48.629027  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 172385us
I20260709 12:24:48.516588  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.651518  5148 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:48.641512  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.604092  9132 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:48.656972  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.657351  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:48.647575  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.607334  8838 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 11 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:48.612510  9219 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:24:48.642688  5068 rpcz_store.cc:276] 0709 12:24:47.047776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.049305 (+  1529us) service_pool.cc:224] Handling call
0709 12:24:47.049792 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.177433 (+127641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.177473 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.178091 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:24:47.314342 (+136251us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:47.314383 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.314448 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.464690 (+1150242us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:48.518136 (+ 53446us) raft_consensus.cc:1685] finished
0709 12:24:48.518163 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.600192 (+ 82029us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127128,"raft.queue_time_us":128552,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"spinlock_wait_cycles":15927552,"thread_start_us":135829,"threads_started":1,"wal-append.queue_time_us":254315}
I20260709 12:24:48.662910  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.665657  5147 raft_consensus.cc:2393] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:48.667762  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 513885us
I20260709 12:24:48.670100  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:48.670640  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:48.609432  9218 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:48.579960  9217 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:48.676604  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:48.676631  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:48.609844  9216 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:48.660712  9022 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.656239  9131 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.612061  9204 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:48.680079  5141 raft_consensus.cc:2376] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14. Re-sending same reply.
I20260709 12:24:48.746798  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1075) took 1431 ms. Trace:
W20260709 12:24:48.746798  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.743525  8400 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:48.655335  9210 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:48.674005  8913 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:24:48.640022  9211 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:48.783371  9014 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:48.784436  8975 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.644s	user 0.005s	sys 0.009s
I20260709 12:24:48.784869  8975 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:48.786190  5942 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:48.742225  9132 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:48.758932  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 80472.4us
I20260709 12:24:48.761174  5073 rpcz_store.cc:276] 0709 12:24:47.315142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.315346 (+   204us) service_pool.cc:224] Handling call
0709 12:24:47.315698 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.891997 (+576299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.727031 (+835034us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b540024efd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:48.746724 (+ 19693us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1315770240}
I20260709 12:24:48.748430  5147 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:24:48.767197  5142 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:24:48.680800  8960 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:48.769457  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1374) took 1190 ms. Trace:
I20260709 12:24:48.772003  9213 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:48.772562  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 102584us
I20260709 12:24:48.746080  9076 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:48.776084  8965 consensus_meta.cc:287] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.683s	user 0.008s	sys 0.003s
I20260709 12:24:48.641520  9215 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:48.790652  5144 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:48.797238  5072 rpcz_store.cc:276] 0709 12:24:47.579230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.579497 (+   267us) service_pool.cc:224] Handling call
0709 12:24:47.579936 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:47.606041 (+ 26105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.606074 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:47.629427 (+ 23353us) log.cc:844] Serialized 52 byte log entry
0709 12:24:47.670169 (+ 40742us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:47.670228 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.670319 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.769156 (+1098837us) raft_consensus.cc:1685] finished
0709 12:24:48.769189 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.769377 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2340,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":9199,"thread_start_us":10602,"threads_started":1,"wal-append.queue_time_us":271656}
I20260709 12:24:48.800772  5141 raft_consensus.cc:3055] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:48.751693  8496 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:48.764784  9122 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:48.814522  9217 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:48.814829  9122 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:48.815726  9097 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:48.766242  9022 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:48.816579  9181 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:48.817164  9181 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:48.742614  8838 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:48.810552  5141 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:48.803574  9076 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:48.847059  9231 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:48.849224  9022 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:48.852516  5217 raft_consensus.cc:2410] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 12:24:48.852592  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.782150  9210 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:48.768013  9131 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:48.827075  9193 consensus_meta.cc:287] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.002s
W20260709 12:24:48.863704  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 235916us
I20260709 12:24:48.867352  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:48.870911  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 58775.8us
I20260709 12:24:48.872188  5114 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:48.854023  9217 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:48.878964  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.802485  8946 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:48.869432  8400 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:48.862902  8948 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:48.873312  9219 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.866621  8917 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:48.882256  9231 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:48.858727  9236 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:24:48.890151  8948 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:48.891884  9014 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.899165  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:48.903817  5146 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:48.882961  8960 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:24:48.901119  9226 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:24:48.931990  5146 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:48.937093  9218 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:48.937899  8948 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:48.938349  9227 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:24:48.938514  8948 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:48.939034  8948 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:48.945875  9234 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:48.947697  9230 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:24:48.948657  9215 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:48.870752  9193 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:48.948849  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 764) took 1135 ms. Trace:
I20260709 12:24:48.922183  9145 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:48.950001  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:48.950773  9193 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:48.951627  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 763) took 1139 ms. Trace:
W20260709 12:24:48.933431  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.952102  5215 rpcz_store.cc:276] 0709 12:24:47.812568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.812728 (+   160us) service_pool.cc:224] Handling call
0709 12:24:47.838711 (+ 25983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.234387 (+395676us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:48.283695 (+ 49308us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.664963 (+381268us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.665146 (+   183us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.676787 (+ 11641us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.676986 (+   199us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:48.677040 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.677126 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.951325 (+274199us) raft_consensus.cc:1685] finished
0709 12:24:48.951355 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.951552 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170747,"prepare.queue_time_us":35844,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":97,"raft.queue_time_us":281742,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":8300,"spinlock_wait_cycles":12397056,"thread_start_us":623546,"threads_started":2,"wal-append.queue_time_us":583959}
I20260709 12:24:48.899765  9153 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:48.897190  9231 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:48.949613  5220 rpcz_store.cc:276] 0709 12:24:47.812893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.813045 (+   152us) service_pool.cc:224] Handling call
0709 12:24:47.837340 (+ 24295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.298640 (+461300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.298693 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.306021 (+  7328us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.628509 (+322488us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:48.628572 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.628707 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.948598 (+319891us) raft_consensus.cc:1685] finished
0709 12:24:48.948628 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.948776 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29762,"raft.queue_time_us":178782,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":9810,"thread_start_us":656859,"threads_started":2,"wal-append.queue_time_us":325498}
I20260709 12:24:48.960089  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:48.924686  9221 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 12:24:48.884902  9217 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:48.906860  9149 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:48.966775  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 47728.3us
W20260709 12:24:48.952028  9233 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:24:48.941416  8948 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:48.934526  8960 raft_consensus.cc:697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:48.896941  9219 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:49.007965  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.008953  5071 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:49.009109  8960 consensus_queue.cc:237] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:49.014886  9127 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:49.015154  9217 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:48.901502  8917 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:48.946619  9234 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:48.984380  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.026196  5072 raft_consensus.cc:2410] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:24:48.988138  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:48.970497  8975 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:48.968436  9231 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:48.976969  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:48.952070  9216 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:49.024585  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:49.020814  9238 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.023313  9226 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:24:49.027453  5146 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:48.922675  9014 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:49.043771  5145 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:49.075711  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.077201  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.080674  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.081594  5042 connection.cc:256] RPC call timeout handler was delayed by -1.2834771380003076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
W20260709 12:24:49.082177  5042 connection.cc:256] RPC call timeout handler was delayed by -1.2690771380002843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
W20260709 12:24:49.082587  5042 connection.cc:256] RPC call timeout handler was delayed by -1.2681771380002829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
I20260709 12:24:49.031064  9072 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:49.082656  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.089681  5042 connection.cc:256] RPC call timeout handler was delayed by -1.1898771380001563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
W20260709 12:24:49.090523  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 48657.1us
W20260709 12:24:49.091507  5042 connection.cc:256] RPC call timeout handler was delayed by -1.0314771380003549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
I20260709 12:24:49.096684  5146 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:49.080142  9216 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:49.102334  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.080657  8975 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.078946  9204 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:49.103530  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.106492  5072 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:49.107314  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.084607  8948 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:49.086416  9076 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:49.082841  9218 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.112588  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:49.118965  5068 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 9: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:49.119371  5075 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:49.112160  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.120190  5069 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:49.120079  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:49.113444  5042 leader_election.cc:336] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.504s (SENT)
I20260709 12:24:49.097591  9149 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.121191  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:49.123047  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.126868  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.068187  9238 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:49.089834  9072 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:49.110107  9131 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.183794  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:49.192633  5070 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:49.193884  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:49.194407  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.198864  9076 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.199458  9076 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:49.127959  8946 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:49.124233  9215 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:49.126360  9149 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:49.202420  9240 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
W20260709 12:24:49.204707  9243 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:24:49.120944  8975 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:49.210458  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:49.206754  9209 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:24:49.214771  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1292) took 1097 ms. Trace:
I20260709 12:24:49.241266  5143 rpcz_store.cc:276] 0709 12:24:48.117493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.123530 (+  6037us) service_pool.cc:224] Handling call
0709 12:24:48.123896 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.517024 (+393128us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:48.750331 (+233307us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.041834 (+291503us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:49.041908 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:49.041987 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.214466 (+172479us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b540036e554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:49.214522 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.214700 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":229500,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":19250,"raft.queue_time_us":731,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":113,"spinlock_wait_cycles":903347968,"thread_start_us":365633,"threads_started":2,"wal-append.queue_time_us":325160}
I20260709 12:24:49.229110  5067 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:49.202877  9246 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:49.121155  8948 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:49.280427  9181 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:49.209414  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 95978.9us
I20260709 12:24:49.118407  9216 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:49.237852  5072 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:49.238742  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:49.251343  5070 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:49.252305  9197 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.253674  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.254077  9238 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:49.257292  9149 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:49.259326  9014 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:49.261224  9025 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:49.254854  9076 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:49.228667  9215 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:49.265744  9131 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.268146  5190 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.632s (SENT)
I20260709 12:24:49.268922  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.337462  9215 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.342276  9212 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:49.342979  9212 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:49.344383  9122 consensus_meta.cc:287] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.529s	user 0.012s	sys 0.001s
I20260709 12:24:49.126055  9218 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:49.368129  5948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "552d065b39ba44699c339e80bcc02744"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:49.368927  5948 raft_consensus.cc:606] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:49.369340  5948 raft_consensus.cc:614] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:49.370402  9218 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.370654  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:49.339263  8975 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:49.372077  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.376377  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:49.381630  9234 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.390561  5940 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:49.301677  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1294) took 1152 ms. Trace:
I20260709 12:24:49.306082  5143 raft_consensus.cc:2376] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13. Re-sending same reply.
W20260709 12:24:49.392307  8400 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.000s
I20260709 12:24:49.393260  5150 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:49.393411  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.393952  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.394474  5143 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:49.395416  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.395684  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:49.396090  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.396766  5143 raft_consensus.cc:2393] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:49.309603  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.302109  9194 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.697s	user 0.008s	sys 0.004s
I20260709 12:24:49.293406  8946 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:49.294201  9232 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:49.321776  9216 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:49.297434  8965 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:49.321985  9220 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:49.322746  9169 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:49.323979  9241 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 12:24:49.329913  9242 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:24:49.301077  9132 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:49.269497  9219 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:49.340162  9188 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.338066  9131 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:49.272123  5141 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:49.344965  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.296519  8905 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:49.301463  8917 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:49.345506  9212 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:49.313270  9197 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:49.339313  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 71195.4us
I20260709 12:24:49.356041  5146 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:49.458333  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:49.459159  5146 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:49.364321  5941 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:49.365605  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.367560  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:49.290851  8948 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:49.373342  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.397413  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.466104  5150 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:49.397979  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.397509  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:49.402674  5145 raft_consensus.cc:2410] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:24:49.467373  5220 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:49.398289  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:49.403849  9195 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 1.142s	user 0.012s	sys 0.008s
I20260709 12:24:49.398658  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.409155  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:24:49.409971  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 118535us
I20260709 12:24:49.470103  5940 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:49.377390  5148 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:49.396651  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.471238  5224 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
W20260709 12:24:49.471453  5042 leader_election.cc:336] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.703s (SENT)
I20260709 12:24:49.391281  5149 rpcz_store.cc:276] 0709 12:24:48.149645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.149951 (+   306us) service_pool.cc:224] Handling call
0709 12:24:48.187987 (+ 38036us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.229948 (+ 41961us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:48.368755 (+138807us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.392756 (+ 24001us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b1800351820. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:48.446355 (+ 53599us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.446777 (+   422us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.447437 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.447608 (+   171us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:48.447806 (+   198us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.466680 (+ 18874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.481040 (+ 14360us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.301386 (+820346us) raft_consensus.cc:1685] finished
0709 12:24:49.301410 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.301609 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21515,"prepare.queue_time_us":1193274,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":8499,"raft.queue_time_us":56494,"raft.run_cpu_time_us":402,"raft.run_wall_time_us":11602,"spinlock_wait_cycles":27706752,"thread_start_us":168300,"threads_started":2,"wal-append.queue_time_us":248588}
I20260709 12:24:49.420594  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 1136) took 1206 ms. Trace:
W20260709 12:24:49.422240  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.471572  5222 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:24:49.472472  5042 leader_election.cc:336] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.757s (SENT)
W20260709 12:24:49.475032  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.425201  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.430610  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.431699  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.478103  5146 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:49.424108  9042 consensus_queue.cc:1059] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:24:49.434934  9201 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:49.434653  8965 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:49.435937  9202 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:49.435719  9169 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:49.436793  9216 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:49.437482  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:49.505719  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:49.439116  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.506108  5223 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.506433  5145 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:49.507565  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.506135  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.509560  5067 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
W20260709 12:24:49.428861  9247 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.117s	user 0.000s	sys 0.003s
I20260709 12:24:49.416478  9211 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.437390  9072 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:49.427752  9232 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:49.481739  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.427528  8946 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:49.465422  5142 raft_consensus.cc:2376] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13. Re-sending same reply.
I20260709 12:24:49.485026  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.485121  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.522073  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:49.486099  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.490325  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:49.523973  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.524099  5146 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:49.491660  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:49.519243  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.525642  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.525902  5941 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 7.
I20260709 12:24:49.493819  5076 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:49.528069  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.450703  9132 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:49.438462  9220 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:49.544494  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.548103  5145 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:49.502700  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.549953  5150 raft_consensus.cc:2376] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13. Re-sending same reply.
I20260709 12:24:49.502745  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.551201  5216 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
W20260709 12:24:49.502736  5190 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 3.323s (SENT) [suppressed 65 similar messages]
W20260709 12:24:49.552362  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 49663.2us
I20260709 12:24:49.552500  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.426208  9194 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:49.501350  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:49.555471  9194 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:49.439503  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:49.509630  5042 leader_election.cc:336] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.803s (SENT)
I20260709 12:24:49.556556  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:49.510995  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.513217  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:49.557361  5143 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.558331  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.560707  5224 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:49.464702  9219 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.561421  9219 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:49.561817  9234 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:49.513813  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.563256  5215 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:49.563581  9219 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:49.447626  9122 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:24:49.522346  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.472647  5217 rpcz_store.cc:276] 0709 12:24:48.207712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.207888 (+   176us) service_pool.cc:224] Handling call
0709 12:24:49.414145 (+1206257us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54001635d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:49.414282 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876190592}
I20260709 12:24:49.553431  5216 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:49.565855  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.528821  5219 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
I20260709 12:24:49.523155  5149 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:49.566461  5148 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:49.495044  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:49.499934  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.567176  5942 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:24:49.568184  5145 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 13. Ops: []
I20260709 12:24:49.568707  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.549453  5220 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:49.542349  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:49.569567  5145 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:49.569978  5147 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:49.570850  5145 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:24:49.571755  5147 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:49.456866  8905 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:49.572166  5143 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:49.573287  5143 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:49.567010  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 65762.2us
I20260709 12:24:49.575681  5948 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
W20260709 12:24:49.556694  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 117303us
I20260709 12:24:49.579255  5115 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:49.566644  5142 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:49.534240  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.571966  5145 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:49.585258  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.586555  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.454295  9188 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:49.585594  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:49.528422  9211 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.599354  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:49.600051  5216 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 12:24:49.612895  5221 raft_consensus.cc:2410] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 12:24:49.616202  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:49.617906  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 185189us
I20260709 12:24:49.619462  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.620134  5947 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:49.551131  5223 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:49.616734  9211 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:49.628947  5149 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 11. Ops: [9.7-9.7]
I20260709 12:24:49.518596  9216 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:49.621299  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.631254  5947 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 7.
W20260709 12:24:49.633461  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 54306.5us
I20260709 12:24:49.583644  9072 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:49.584507  8917 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:49.594367  9234 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:49.639885  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1017) took 1867 ms. Trace:
I20260709 12:24:49.640808  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.641902  5942 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
W20260709 12:24:49.643141  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 133542us
I20260709 12:24:49.640548  5943 rpcz_store.cc:276] 0709 12:24:47.772372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.772511 (+   139us) service_pool.cc:224] Handling call
0709 12:24:47.772869 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.773242 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.773272 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.798594 (+ 25322us) log.cc:844] Serialized 26 byte log entry
0709 12:24:47.920108 (+121514us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:47.920178 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.920294 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.402995 (+1482701us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:49.629644 (+226649us) raft_consensus.cc:1685] finished
0709 12:24:49.629677 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.639822 (+ 10145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":561,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":22787,"spinlock_wait_cycles":47670656,"thread_start_us":71272,"threads_started":1,"wal-append.queue_time_us":363682}
I20260709 12:24:49.644765  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.610976  9014 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.665526  5143 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:24:49.618678  9242 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:24:49.667444  5188 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:49.673339  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:49.666733  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:49.684043  5898 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:49.685101  5898 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:49.685856  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:49.686244  5042 leader_election.cc:336] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.443s (SENT)
I20260709 12:24:49.686726  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:49.686902  5941 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:49.597923  9042 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:24:49.728369  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.688794  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.690747  5041 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:49.729208  5943 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:24:49.691686  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:49.678696  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:49.694751  5141 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:49.696144  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:49.738130  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 46538.3us
I20260709 12:24:49.646435  8917 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:49.763497  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1323) took 1151 ms. Trace:
W20260709 12:24:49.740451  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 49822.1us
I20260709 12:24:49.695333  9071 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:49.691556  9202 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.770433  5041 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:49.645949  8975 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 12:24:49.740530  5941 raft_consensus.cc:1275] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
W20260709 12:24:49.771150  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 84931.4us
I20260709 12:24:49.764217  5144 rpcz_store.cc:276] 0709 12:24:48.611713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.812015 (+200302us) service_pool.cc:224] Handling call
0709 12:24:48.812550 (+   535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.848505 (+ 35955us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.848544 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.849065 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.976876 (+127811us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:49.028322 (+ 51446us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.028626 (+   304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.028693 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.289433 (+260740us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b1800425e60. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:49.310230 (+ 20797us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b1800425e60. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e522 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:49.763179 (+452949us) raft_consensus.cc:1685] finished
0709 12:24:49.763216 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.763411 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24073,"prepare.queue_time_us":51137,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":36523,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":216,"spinlock_wait_cycles":35001344,"thread_start_us":126893,"threads_started":1,"wal-append.queue_time_us":228585}
I20260709 12:24:49.729477  9201 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:49.737469  8400 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:49.778718  5941 pending_rounds.cc:85] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 3
I20260709 12:24:49.781427  5941 pending_rounds.cc:107] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:24:49.781875  5941 raft_consensus.cc:2889] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:49.777064  5145 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:49.738394  9204 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:24:49.779682  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.763788  8917 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:49.687808  9014 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:49.798278  9014 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.799665  8948 raft_consensus.cc:3055] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:49.800395  8948 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:49.741111  9076 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:49.807770  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 160285us
W20260709 12:24:49.779431  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 112064us
I20260709 12:24:49.786216  8400 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:49.809027  5190 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:49.786607  9131 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:49.788496  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.788369  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.737108  9217 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:24:49.764487  9072 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:49.805369  9220 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:49.806769  9211 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:49.819980  5947 raft_consensus.cc:2410] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:24:49.786640  8838 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:24:49.785302  9231 consensus_meta.cc:287] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.713s	user 0.014s	sys 0.009s
I20260709 12:24:49.784001  9164 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:49.786216  9153 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:49.783596  9201 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.852351  9201 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:49.786283  9197 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:49.854378  9197 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:49.854954  9197 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:49.805572  9188 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:49.857306  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:49.857884  9132 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.813108  8400 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:49.858692  8400 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:49.782142  9191 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:49.859916  9191 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:49.810202  9076 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.861838  9076 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.862433  9191 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:49.822014  9219 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:49.864048  9219 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.864415  9219 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:49.864846  8946 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.837682  9072 raft_consensus.cc:697] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:49.866447  9072 consensus_queue.cc:237] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:49.846210  9231 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:49.869922  9225 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:49.870397  8905 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.871871  9231 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:49.809759  9202 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:49.873780  5942 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:49.850284  9169 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.848724  9204 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:49.857240  9197 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:49.860276  9216 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.874238  9202 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:49.901129  5141 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:49.902081  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1319) took 1327 ms. Trace:
I20260709 12:24:49.838126  8838 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:49.903256  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 70302.2us
I20260709 12:24:49.902391  5940 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:49.904160  5141 rpcz_store.cc:276] 0709 12:24:48.574472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.770510 (+196038us) service_pool.cc:224] Handling call
0709 12:24:48.800639 (+ 30129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.271664 (+471025us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b4c00381180. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6807a20 00007fd8a6818a29 00007fd8a67de644 00007fd8a6805f2d 00007fd8a6805b93 00007fd8a853618b 00007fd8a8536cb5 00007fd8a8531f30 00007fd8a85337de 00007fd8a853db49 00007fd8a853e416
0709 12:24:49.694173 (+422509us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b58006656d0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a84970aa 00007fd8a8519447 00007fd8a8531fe7 00007fd8a85337de 00007fd8a853db49 00007fd8a853e416 00007fd8a853ad8f 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4
0709 12:24:49.901698 (+207525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.902009 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610101504}
I20260709 12:24:49.840780  9056 raft_consensus.cc:2764] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:24:49.908892  5188 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:49.898020  9025 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:49.917979  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.922230  5075 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:49.920801  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:49.911226  8838 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:49.904877  8752 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
W20260709 12:24:49.927325  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.904256  9197 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.937913  9197 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:49.940346  9197 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:49.914196  8960 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.831014  8948 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:49.903200  9131 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:24:49.952929  9232 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:49.955837  8965 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:49.955732  5899 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:49.925285  9181 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:49.972474  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:49.922768  9191 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:24:49.990649  5189 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:49.929956  9194 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:24:49.975103  9181 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.976313  9202 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:49.999476  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.999831  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:50.000568  9187 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:50.001478  9169 raft_consensus.cc:993] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: : Instructing follower 90e5220a62ed42fe83c82d0302d18fac to start an election
I20260709 12:24:50.001935  5946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:49.984745  8965 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:50.005658  9231 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:50.006230  8975 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.005756  8982 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:50.030208  9217 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.002621  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:49.958752  9219 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:50.034588  5946 raft_consensus.cc:606] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.043195  5946 raft_consensus.cc:614] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:50.037812  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:50.043828  5947 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:49.987942  9197 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.030776  9181 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.034003  9169 raft_consensus.cc:1081] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Signalling peer 90e5220a62ed42fe83c82d0302d18fac to start an election
I20260709 12:24:49.972710  8960 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:49.993929  8948 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:50.045476  9201 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:50.049959  9187 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:50.032374  9202 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.051072  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:50.036290  8965 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.942305  8917 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:50.055881  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.070286  5067 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:50.035770  9076 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:50.042438  9231 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:49.990878  9204 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:50.052984  9234 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:50.089870  5071 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:50.044827  8975 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:50.093693  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 397642us
I20260709 12:24:50.065088  5947 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:50.069525  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:50.070847  5075 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:50.064167  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.070343  5068 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:50.131459  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:50.082203  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:50.083683  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.095549  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:50.060539  9187 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:50.096679  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.097316  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.065227  9202 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:50.100360  9076 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:50.052793  9197 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:50.105926  9219 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:50.114995  9201 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:50.116771  9231 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:50.118618  8917 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:50.044695  9217 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:50.064231  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:50.122846  8400 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:50.122956  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1166) took 1165 ms. Trace:
I20260709 12:24:50.194960  5148 rpcz_store.cc:276] 0709 12:24:48.957179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.576271 (+619092us) service_pool.cc:224] Handling call
0709 12:24:49.576631 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.617770 (+ 41139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.617807 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.633674 (+ 15867us) log.cc:844] Serialized 26 byte log entry
0709 12:24:50.052879 (+419205us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:50.052928 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.053002 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.122666 (+ 69664us) raft_consensus.cc:1685] finished
0709 12:24:50.122695 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.122885 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17454,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":12140,"thread_start_us":315595,"threads_started":1,"wal-append.queue_time_us":446045}
I20260709 12:24:50.069937  8965 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:50.127521  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.128345  5073 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:50.132920  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.134837  5146 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:50.136152  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.136632  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.096212  9153 raft_consensus.cc:697] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:50.144297  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48811.1us
I20260709 12:24:50.163291  5947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:50.208005  5947 raft_consensus.cc:606] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.151875  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.182830  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:50.199199  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 187536us
I20260709 12:24:50.209781  5074 raft_consensus.cc:1275] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:50.206305  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:50.210587  5115 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:50.210677  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:50.212980  5067 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:50.206840  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.208648  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:50.203704  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 140062us
I20260709 12:24:50.216723  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:50.208885  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:50.224128  5224 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:50.193828  9076 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:50.209844  5071 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:50.238543  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.244676  5141 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:50.244311  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.245363  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.240162  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:50.246023  5143 raft_consensus.cc:2376] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15. Re-sending same reply.
I20260709 12:24:50.242551  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:50.242551  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:50.206429  9118 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.252537  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:50.252698  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.253090  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.253495  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.259637  5145 raft_consensus.cc:2376] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13. Re-sending same reply.
I20260709 12:24:50.259701  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:50.151377  9271 raft_consensus.cc:2764] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:24:50.254581  5189 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:50.268390  5141 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:50.269169  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.277352  5147 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:50.270148  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.271771  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:50.275116  5217 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:50.283388  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.284142  5147 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:50.281169  9118 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:50.288388  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.289707  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.290648  9118 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:50.290952  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.290485  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.282886  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:50.292253  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.293110  5149 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:50.298653  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.235103  9234 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:50.299556  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.299520  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.307978  5041 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:50.308434  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.311028  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.312674  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.314636  5115 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.316321  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:50.317382  9225 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:50.318419  9225 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:50.323146  5115 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.326045  9225 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:50.332231  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:50.335485  9225 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:50.335484  9042 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:50.336963  9149 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.337450  9149 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:50.339126  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.339679  9149 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:50.343050  9225 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:50.344516  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:50.345254  5945 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:50.344849  5115 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.348024  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:50.348762  5948 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.349359  9225 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:50.350659  9270 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:24:50.352667  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1161) took 1445 ms. Trace:
I20260709 12:24:50.353921  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:50.353039  5150 rpcz_store.cc:276] 0709 12:24:48.907686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.575252 (+667566us) service_pool.cc:224] Handling call
0709 12:24:49.612403 (+ 37151us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:49.638222 (+ 25819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.638248 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:49.665849 (+ 27601us) log.cc:844] Serialized 52 byte log entry
0709 12:24:49.903872 (+238023us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:50.272442 (+368570us) log.cc:844] Serialized 14 byte log entry
0709 12:24:50.272740 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.272840 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.352301 (+ 79461us) raft_consensus.cc:1685] finished
0709 12:24:50.352320 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.352588 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57934,"prepare.queue_time_us":429681,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":19070,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":42845,"thread_start_us":492469,"threads_started":2,"wal-append.queue_time_us":354086}
I20260709 12:24:50.355286  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.356130  5150 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:50.357062  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.355508  9275 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:50.357645  9042 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:50.358059  9275 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:50.358536  9275 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:50.358889  9042 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:50.364076  5041 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:50.365155  9071 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:50.356143  8400 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:50.365710  9071 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:50.366226  9071 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:50.366317  5150 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:50.370937  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.372048  5144 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:50.350970  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.375705  8975 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:50.377723  8400 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:50.375022  9275 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:50.384583  9071 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:50.386161  9275 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:50.298458  9072 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:50.378695  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.387863  8400 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:50.373484  5150 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:50.390755  9197 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:50.299232  8982 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:50.392295  8400 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:50.392362  5948 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:50.394212  9217 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:50.282913  8917 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:50.395190  8917 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:50.395713  8917 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:50.395874  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:50.252231  9056 consensus_queue.cc:1059] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.146s
I20260709 12:24:50.390790  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.398726  9076 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:50.262410  9269 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.157s
I20260709 12:24:50.399391  9076 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:50.400086  9076 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.400236  9071 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:50.402266  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.273239  9187 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:50.412629  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.412720  8838 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:50.414615  8400 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:50.415597  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.417165  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:50.418897  5948 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:50.373364  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.430557  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:50.439148  5067 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:50.448230  5148 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:50.454890  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:50.462299  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:50.465590  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.472975  5141 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:50.484457  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:50.494114  5068 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
W20260709 12:24:50.484952  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 275746us
I20260709 12:24:50.485437  8400 consensus_queue.cc:1243] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:50.387117  9072 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:50.502108  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:50.503554  5150 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 15: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:50.509933  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.499636  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.517197  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:50.517437  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.306264  5217 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:57002
I20260709 12:24:50.497788  5071 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:50.497089  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.521056  5217 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:50.516404  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.501345  5945 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:24:50.521512  5217 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:50.492806  5941 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 6.
I20260709 12:24:50.528941  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:50.531807  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.533795  5070 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 15: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:50.538610  5944 raft_consensus.cc:2410] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:24:50.591909  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.456892  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:50.617859  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.651973  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 142133us
I20260709 12:24:50.654117  5898 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:50.655344  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 158333us
I20260709 12:24:50.457753  9071 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:50.521823  5945 pending_rounds.cc:85] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 5
I20260709 12:24:50.685772  5945 pending_rounds.cc:107] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:24:50.686015  8400 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:50.683773  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 178568us
I20260709 12:24:50.686892  8400 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:50.686254  5945 raft_consensus.cc:2889] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:50.688972  5143 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:24:50.690481  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.691061  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:50.692406  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.695611  5143 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:50.696048  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.698550  5142 raft_consensus.cc:2410] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:24:50.701336  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.703574  5143 raft_consensus.cc:2376] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9. Re-sending same reply.
I20260709 12:24:50.461048  9197 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:50.464085  9278 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:50.706156  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.444499  9076 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:50.708503  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.711615  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.714162  5900 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:50.515146  9269 raft_consensus.cc:3055] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:50.520157  9072 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:50.758610  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44551.2us
I20260709 12:24:50.788365  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1184) took 1403 ms. Trace:
I20260709 12:24:50.760459  5143 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:50.789135  5146 rpcz_store.cc:276] 0709 12:24:49.377915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.527680 (+149765us) service_pool.cc:224] Handling call
0709 12:24:50.781231 (+1253551us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b54000ebd54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:50.781436 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1168548736}
I20260709 12:24:50.693773  9265 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:50.727885  5144 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 11: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:50.461577  8948 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:50.804535  8948 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:50.807123  8948 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:50.808959  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.809863  5144 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:50.810920  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.811720  5144 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:50.812754  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.812784  9169 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.814201  9033 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:50.814819  9033 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.815229  9033 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:50.818470  9169 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:50.821501  9187 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:50.824394  5946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:50.824877  9169 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:50.724830  5149 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:50.826848  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.827463  5149 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:50.791939  9122 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:50.828296  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.829291  5149 raft_consensus.cc:2368] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:24:50.830168  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.831282  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.831929  5150 raft_consensus.cc:2393] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:50.833372  9014 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:50.834592  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.835357  5148 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:50.839715  9225 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.840314  9225 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:50.848627  9285 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.118s	user 0.003s	sys 0.000s
I20260709 12:24:50.736197  9231 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:50.744812  5141 raft_consensus.cc:2376] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13. Re-sending same reply.
I20260709 12:24:50.867960  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.868829  5141 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:50.794579  9265 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:50.870952  9265 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:50.825227  5946 raft_consensus.cc:606] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:24:50.878041  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:50.894476  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.896606  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.902036  5141 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:50.673380  9000 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:50.792498  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.947536  5142 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:50.792693  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.707737  9188 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:50.952811  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.953986  5143 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:50.955155  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.956535  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:50.957880  9188 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:50.959353  9188 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:50.962898  5944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:42690
I20260709 12:24:50.963964  5944 raft_consensus.cc:606] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:50.964572  5944 raft_consensus.cc:614] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:50.957468  5143 raft_consensus.cc:2368] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11.
I20260709 12:24:50.967916  5143 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:50.763167  8400 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:50.766462  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 309679us
I20260709 12:24:50.798725  8838 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:50.798815  5900 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:50.972208  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 173475us
I20260709 12:24:50.973140  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.973802  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:50.974870  9225 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:50.802615  8960 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:50.975986  9000 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:50.976514  9000 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:50.976961  9000 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:50.980687  9231 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:50.803123  9076 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:50.850031  9169 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:50.854882  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.986924  5149 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:50.705686  8752 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.151s
I20260709 12:24:50.734561  9056 raft_consensus.cc:3055] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:50.516228  9153 consensus_queue.cc:237] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:50.728539  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:50.996809  5947 raft_consensus.cc:2410] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:24:50.758108  9181 consensus_meta.cc:287] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.701s	user 0.004s	sys 0.007s
I20260709 12:24:50.741765  9197 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:50.864466  5217 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:50.756575  9269 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:50.906724  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.575408  9271 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.920652  5071 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
W20260709 12:24:50.921275  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 267233us
I20260709 12:24:51.020684  5217 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:50.926524  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.927623  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:51.024924  5146 raft_consensus.cc:2376] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12. Re-sending same reply.
I20260709 12:24:50.929489  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:51.029090  5144 raft_consensus.cc:2376] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12. Re-sending same reply.
I20260709 12:24:51.031183  5144 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:50.944504  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:50.730687  9071 raft_consensus.cc:697] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:50.975124  5900 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:50.990968  5142 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 15: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:24:51.038064  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 63032.3us
I20260709 12:24:51.039297  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:50.981287  9231 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:50.971398  8917 consensus_meta.cc:287] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.575s	user 0.013s	sys 0.002s
W20260709 12:24:50.995265  9279 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.055s	sys 0.115s
W20260709 12:24:51.052008  9279 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.055s	sys 0.116s
I20260709 12:24:51.002313  5188 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:51.003018  5076 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.075s	sys 0.112s
W20260709 12:24:51.053090  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50894.6us
W20260709 12:24:51.053519  5076 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.076s	sys 0.112s
I20260709 12:24:51.054749  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1397) took 1129 ms. Trace:
I20260709 12:24:51.055207  9204 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:51.056427  9122 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:51.057237  9122 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:51.058511  9122 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:51.035857  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:51.022531  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:51.062004  9231 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:51.062870  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:51.063213  9269 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:51.063829  5041 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:51.064533  9269 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:51.064967  8975 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:51.064863  9076 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:51.016148  9181 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:51.051323  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.065377  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:51.051975  8917 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:51.063895  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:50.995930  9056 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:51.055131  5076 rpcz_store.cc:276] 0709 12:24:49.925235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.925423 (+   188us) service_pool.cc:224] Handling call
0709 12:24:49.925836 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.926119 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:50.196126 (+270007us) log.cc:844] Serialized 14 byte log entry
0709 12:24:51.054221 (+858095us) log.cc:844] Serialized 14 byte log entry
0709 12:24:51.054400 (+   179us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:51.054453 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:51.054500 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.054565 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.054689 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":193287,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"raft.queue_time_us":1399,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":188,"thread_start_us":1080273,"threads_started":2}
I20260709 12:24:51.046252  9197 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:51.067250  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.088444  9181 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:51.078085  8975 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:51.089061  9202 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.944s	user 0.010s	sys 0.000s
I20260709 12:24:51.097482  5075 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:51.078511  9204 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:51.114085  5069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.039s	sys 0.129s
W20260709 12:24:51.114573  5069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.039s	sys 0.129s
I20260709 12:24:51.112761  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:51.115520  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 850) took 1047 ms. Trace:
I20260709 12:24:51.115749  5074 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:51.115938  5069 rpcz_store.cc:276] 0709 12:24:50.068026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.073613 (+  5587us) service_pool.cc:224] Handling call
0709 12:24:50.074023 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.074315 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:50.457865 (+383550us) log.cc:844] Serialized 14 byte log entry
0709 12:24:51.115076 (+657211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:51.115139 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:51.115205 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.115289 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.115449 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32419,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"thread_start_us":1014984,"threads_started":2}
W20260709 12:24:51.117623  5145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.098s	sys 0.143s
W20260709 12:24:51.117990  5145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.098s	sys 0.143s
W20260709 12:24:51.120764  5190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.004s	sys 0.037s
W20260709 12:24:51.121177  5190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.004s	sys 0.037s
W20260709 12:24:51.123705  5189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.035s	sys 0.170s
W20260709 12:24:51.124209  5189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.035s	sys 0.170s
W20260709 12:24:51.124540  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 841665us
I20260709 12:24:51.126017  5148 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:51.129596  5188 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:51.130843  5042 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.027s	sys 0.016s
W20260709 12:24:51.131297  5042 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.027s	sys 0.016s
W20260709 12:24:51.131630  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 622842us
W20260709 12:24:51.082820  5939 consensus_meta.cc:287] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.636s	user 0.013s	sys 0.001s
W20260709 12:24:51.082938  5147 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.786s	user 0.009s	sys 0.001s
I20260709 12:24:51.132470  5147 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:24:51.065840  9169 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:51.085752  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 45009.1us
I20260709 12:24:51.086735  8917 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:51.065728  9127 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:51.066133  9076 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.042295  9271 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:51.089876  9000 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:51.089383  8975 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:51.091255  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:51.068503  5071 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:51.094375  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.094077  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:51.134326  5939 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
W20260709 12:24:51.145118  5224 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.049s	sys 0.153s
I20260709 12:24:51.147984  5144 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: [14.6-14.6]
W20260709 12:24:51.167655  5224 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.049s	sys 0.153s
I20260709 12:24:51.169669  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:51.170024  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:51.173425  5221 raft_consensus.cc:2393] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 8: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:51.110769  9204 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:51.121019  9202 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:51.183705  9071 consensus_queue.cc:237] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:51.164145  9191 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:51.168151  8917 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.195099  9000 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:51.196820  5076 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:51.155822  9169 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:51.110613  9056 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:51.163873  9076 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.201990  9181 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:51.213886  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:51.254401  5220 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:51.246131  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:51.256764  5948 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:51.260756  5215 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:51.263899  9202 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:51.264787  9202 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:51.268033  9204 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:51.249125  8965 consensus_meta.cc:287] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.038s	user 0.007s	sys 0.007s
W20260709 12:24:51.275473  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.279332  5071 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:51.244683  9191 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.277510  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:51.314525  5076 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:51.269035  9204 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:51.320147  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:51.255481  8917 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:51.313966  4888 maintenance_manager.cc:419] P ce6fc7638c8549faa2535fb5774c32c1: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033398
I20260709 12:24:51.333616  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:51.316936  9191 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:51.345714  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:51.331602  8708 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:51.383304  9271 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:51.384339  9231 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:51.385771  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:51.337291  9056 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:51.312827  8965 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:51.269341  8975 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:51.377664  9286 log.cc:927] Time spent T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:24:51.280485  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:51.476842  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 196453us
I20260709 12:24:51.332264  9265 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:51.451689  5187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.009s	sys 0.033s
W20260709 12:24:51.442822  9288 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
W20260709 12:24:51.482663  5187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.010s	sys 0.033s
I20260709 12:24:51.354171  9025 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 12:24:51.489884  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:51.493242  5940 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:51.493569  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:51.378252  9187 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
W20260709 12:24:51.506628  5943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.015s	sys 0.021s
I20260709 12:24:51.507236  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:51.485596  9149 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:51.516062  5115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.021s	sys 0.039s
I20260709 12:24:51.432495  8708 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:51.519812  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:51.520555  5115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.022s	sys 0.039s
W20260709 12:24:51.526772  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:51.524901  5116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.027s	sys 0.030s
W20260709 12:24:51.533524  5116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.027s	sys 0.030s
W20260709 12:24:51.507515  5943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.015s	sys 0.021s
I20260709 12:24:51.538233  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:51.539075  5144 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:51.541749  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1104) took 1475 ms. Trace:
I20260709 12:24:51.542418  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:51.543300  5220 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:51.542930  5942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.006s	sys 0.006s
W20260709 12:24:51.545563  5942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.006s	sys 0.006s
I20260709 12:24:51.542317  5072 rpcz_store.cc:276] 0709 12:24:50.065993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.067111 (+  1118us) service_pool.cc:224] Handling call
0709 12:24:50.092135 (+ 25024us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:50.105942 (+ 13807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.105980 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:50.106570 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 12:24:50.450481 (+343911us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:50.450551 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.450656 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:51.541484 (+1090828us) raft_consensus.cc:1685] finished
0709 12:24:51.541509 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.541675 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":226,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":5197,"thread_start_us":337852,"threads_started":1,"wal-append.queue_time_us":1049194}
I20260709 12:24:51.551673  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:51.552505  5147 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:51.553218  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:51.564069  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:51.564738  5144 raft_consensus.cc:2376] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11. Re-sending same reply.
I20260709 12:24:51.507498  9076 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:51.570881  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:51.574386  5144 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:24:51.575037  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.575937  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:51.576546  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.577028  5147 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:51.573177  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:51.580793  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:51.584793  5150 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.006s	sys 0.012s
W20260709 12:24:51.585330  5150 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.007s	sys 0.013s
W20260709 12:24:51.563014  9002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.007s	sys 0.007s
I20260709 12:24:51.595319  5216 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:51.594280  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:51.599810  5114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.004s	sys 0.013s
W20260709 12:24:51.603127  5114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.004s	sys 0.013s
W20260709 12:24:51.603557  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 916144us
I20260709 12:24:51.604789  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:51.607584  5899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.003s	sys 0.018s
W20260709 12:24:51.678596  5899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.004s	sys 0.026s
W20260709 12:24:51.600476  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 453253us
I20260709 12:24:51.531008  9296 consensus_queue.cc:1059] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
W20260709 12:24:51.615053  5898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.012s	sys 0.008s
I20260709 12:24:51.557950  5220 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:24:51.586350  9169 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:51.667263  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:51.563512  9271 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 12:24:51.671455  5149 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.015s	sys 0.015s
W20260709 12:24:51.708905  5146 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.014s	sys 0.022s
W20260709 12:24:51.709348  5149 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.015s	sys 0.015s
W20260709 12:24:51.672260  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.580008  9202 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:51.559854  9149 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:24:51.600291  5948 raft_consensus.cc:2410] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 12:24:51.596736  9002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.007s	sys 0.008s
W20260709 12:24:51.679654  5898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.012s	sys 0.008s
I20260709 12:24:51.722638  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:51.723552  5948 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:51.724680  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:51.725338  5947 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
W20260709 12:24:51.679615  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 614259us
I20260709 12:24:51.681767  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:51.683075  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:51.679013  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 74307.6us
W20260709 12:24:51.712741  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 45537.8us
I20260709 12:24:51.935957  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:51.709506  5146 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.014s	sys 0.022s
I20260709 12:24:51.938858  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:51.709052  8752 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:51.716059  9289 log.cc:927] Time spent T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
W20260709 12:24:51.716846  9295 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:24:51.959697  5948 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
W20260709 12:24:51.678874  9033 consensus_meta.cc:287] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.863s	user 0.010s	sys 0.004s
I20260709 12:24:51.763257  5899 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:51.763384  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 845) took 1303 ms. Trace:
I20260709 12:24:51.786885  9231 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:51.967723  5943 rpcz_store.cc:276] 0709 12:24:50.459835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.459978 (+   143us) service_pool.cc:224] Handling call
0709 12:24:50.460359 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:50.460786 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.460820 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:50.499836 (+ 39016us) log.cc:844] Serialized 26 byte log entry
0709 12:24:51.534803 (+1034967us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:51.534841 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.534909 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:51.763140 (+228231us) raft_consensus.cc:1685] finished
0709 12:24:51.763162 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.763315 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":39614,"thread_start_us":862145,"threads_started":1,"wal-append.queue_time_us":1001563}
I20260709 12:24:51.721975  9202 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:51.973143  9202 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:51.766649  5945 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
W20260709 12:24:51.754623  9293 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
W20260709 12:24:51.717698  9292 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 12:24:51.765300  9072 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.905s	user 0.012s	sys 0.000s
W20260709 12:24:51.784515  5217 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.023s	sys 0.031s
W20260709 12:24:51.975935  5217 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.031s	sys 0.047s
I20260709 12:24:51.975850  9072 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:51.976765  9122 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:24:51.767025  5223 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.307s	user 0.007s	sys 0.019s
W20260709 12:24:51.977897  5223 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.525s	user 0.020s	sys 0.021s
W20260709 12:24:51.941488  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 259782us
I20260709 12:24:51.948004  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:51.716924  9271 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:51.982400  5218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.017s	sys 0.022s
W20260709 12:24:51.982962  5218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.017s	sys 0.023s
I20260709 12:24:51.979096  5944 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:51.964679  5939 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:24:51.970489  9033 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:51.976145  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 212932us
W20260709 12:24:51.679832  9042 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.005s	sys 0.025s
I20260709 12:24:51.979492  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:51.980484  5900 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:51.963688  9301 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.377s	user 0.006s	sys 0.000s
I20260709 12:24:51.978389  9072 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:51.990551  5899 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:51.979975  9153 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.929s	user 0.028s	sys 0.043s
I20260709 12:24:51.950806  9169 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:52.008924  5948 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:52.009236  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:52.018767  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:51.990343  5145 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.023s	sys 0.078s
W20260709 12:24:52.041553  5145 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.024s	sys 0.078s
W20260709 12:24:51.997298  9127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.028s	sys 0.073s
I20260709 12:24:51.951284  8975 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:52.049367  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:52.035993  9042 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.072s	user 0.007s	sys 0.025s
W20260709 12:24:52.054646  5900 connection.cc:256] RPC call timeout handler was delayed by -1.5432391079998524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:24:52.056941  5215 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:51.990332  9265 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:52.058279  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 67841.7us
I20260709 12:24:52.059043  5899 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:52.044312  9289 log.cc:927] Time spent T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 12:24:52.044044  9153 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.028s	sys 0.043s
I20260709 12:24:52.042236  8708 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:51.986187  8752 consensus_queue.cc:1059] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:24:51.961633  9303 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 12:24:51.990491  9302 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
W20260709 12:24:52.065892  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.081753  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1403) took 1875 ms. Trace:
I20260709 12:24:52.082275  5070 rpcz_store.cc:276] 0709 12:24:50.205879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.682258 (+476379us) service_pool.cc:224] Handling call
0709 12:24:50.682709 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:50.683181 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.683221 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:50.707066 (+ 23845us) log.cc:844] Serialized 26 byte log entry
0709 12:24:51.147565 (+440499us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:51.147617 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.147704 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.081455 (+933751us) raft_consensus.cc:1685] finished
0709 12:24:52.081485 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.081679 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":584,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":19479,"thread_start_us":411840,"threads_started":1,"wal-append.queue_time_us":782895}
I20260709 12:24:52.070622  9072 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:24:52.043923  9127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.028s	sys 0.073s
I20260709 12:24:52.044576  9169 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.008599  9231 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:24:52.041167  9122 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:52.076375  8975 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:52.095739  5188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.018s	sys 0.022s
W20260709 12:24:52.117812  5188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.018s	sys 0.022s
W20260709 12:24:52.098974  5190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.010s	sys 0.028s
I20260709 12:24:52.073433  9202 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:52.052361  9271 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:24:52.063577  9315 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:52.092573  9127 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:52.086133  9000 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:52.080976  9265 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:52.120016  5190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.010s	sys 0.029s
I20260709 12:24:52.096349  8752 consensus_queue.cc:1243] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:52.097394  9076 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:52.122449  5042 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.005s	sys 0.005s
W20260709 12:24:52.158989  5042 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.005s	sys 0.005s
W20260709 12:24:52.159379  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 668959us
I20260709 12:24:52.094871  5948 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
W20260709 12:24:52.125229  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 869500us
W20260709 12:24:52.160981  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.101204  9169 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:52.186184  5150 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.020s	sys 0.012s
W20260709 12:24:52.187747  5150 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.020s	sys 0.013s
I20260709 12:24:52.189246  5070 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:52.161294  5143 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.006s	sys 0.007s
W20260709 12:24:52.196395  5143 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.007s	sys 0.007s
W20260709 12:24:52.185012  5219 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.011s	sys 0.016s
I20260709 12:24:52.126511  9315 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:52.198830  5219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.012s	sys 0.016s
I20260709 12:24:52.199775  5222 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:24:52.128636  9000 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:52.209638  5072 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: []
W20260709 12:24:52.217195  9056 consensus_meta.cc:287] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.771s	user 0.007s	sys 0.003s
I20260709 12:24:52.122036  8975 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:52.163940  8752 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:52.225028  5149 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.000s	sys 0.020s
W20260709 12:24:52.229895  5149 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.000s	sys 0.021s
I20260709 12:24:52.192049  9092 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.155884  9312 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 12:24:52.185844  9072 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:24:52.237996  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:52.203150  9315 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:52.188305  9153 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:24:52.232080  9303 raft_consensus.cc:3055] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:52.245090  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:52.250809  5072 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:52.208312  9042 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:24:52.204795  9076 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 12:24:52.228780  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.277429  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 776) took 2030 ms. Trace:
W20260709 12:24:52.278070  5190 leader_election.cc:336] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.694s (SENT)
W20260709 12:24:52.279052  5190 leader_election.cc:336] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.967s (SENT)
I20260709 12:24:52.279516  5190 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:52.254145  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 318264us
W20260709 12:24:52.258659  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 314435us
I20260709 12:24:52.286206  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:52.260046  5142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.005s	sys 0.009s
W20260709 12:24:52.287125  5142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.006s	sys 0.009s
W20260709 12:24:52.262045  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 66550.6us
I20260709 12:24:52.277979  5145 rpcz_store.cc:276] 0709 12:24:50.267124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.267314 (+   190us) service_pool.cc:224] Handling call
0709 12:24:50.267834 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:50.268276 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.268309 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:50.268930 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 12:24:51.118364 (+849434us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:52.041924 (+923560us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.277029 (+235105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.277128 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.277174 (+    46us) raft_consensus.cc:1685] finished
0709 12:24:52.277186 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.277351 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1073328,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":12927,"raft.queue_time_us":184595,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":11465,"thread_start_us":1979359,"threads_started":3,"wal-append.queue_time_us":1239364}
I20260709 12:24:52.289707  5042 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:52.250129  9033 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:52.294601  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 46294.4us
I20260709 12:24:52.281626  5041 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:52.287549  9306 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:24:52.243902  9305 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:52.233156  8752 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:52.276844  9303 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:52.286496  9217 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:52.236601  9092 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:52.351831  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 303185us
I20260709 12:24:52.358489  5898 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:52.240658  9127 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:52.395291  5072 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:52.396049  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:52.391279  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 112253us
W20260709 12:24:52.355595  9308 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 12:24:52.343658  9122 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:52.306712  9056 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:24:52.407763  5898 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:52.420595  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:52.404675  9181 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.139s	user 0.017s	sys 0.023s
W20260709 12:24:52.390765  9318 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.192s	user 0.001s	sys 0.001s
W20260709 12:24:52.410269  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50028.2us
I20260709 12:24:52.357954  9322 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:52.424885  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 123363us
I20260709 12:24:52.295372  9000 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:52.391000  9313 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:52.446215  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:52.427006  9181 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.017s	sys 0.023s
I20260709 12:24:52.475437  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 848) took 1778 ms. Trace:
I20260709 12:24:52.349390  9305 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:52.427661  5940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 1038) took 2429 ms (client timeout 2862 ms). Trace:
I20260709 12:24:52.412355  9202 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:52.424131  9153 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:52.414307  9215 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
W20260709 12:24:52.434514  9316 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 12:24:52.427073  9042 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:52.476111  5942 rpcz_store.cc:276] 0709 12:24:50.697013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.792180 (+ 95167us) service_pool.cc:224] Handling call
0709 12:24:50.799611 (+  7431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:50.843048 (+ 43437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.843097 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:50.930577 (+ 87480us) log.cc:844] Serialized 26 byte log entry
0709 12:24:51.546476 (+615899us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:51.546521 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.717411 (+170890us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b5400008054. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:51.717461 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.435093 (+717632us) raft_consensus.cc:1685] finished
0709 12:24:52.435125 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.475346 (+ 40221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6748,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":109,"spinlock_wait_cycles":98725120,"thread_start_us":596216,"threads_started":1,"wal-append.queue_time_us":1127887}
W20260709 12:24:52.485960  5940 rpcz_store.cc:269] 0709 12:24:49.998191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.998313 (+   122us) service_pool.cc:224] Handling call
0709 12:24:52.427414 (+2429101us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400004e54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:52.427590 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3338291456}
W20260709 12:24:52.429785  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 143676us
I20260709 12:24:52.491727  8917 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:52.492607  9307 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.326s	user 0.000s	sys 0.001s
W20260709 12:24:52.490350  9339 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:24:52.482548  9344 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:52.510990  9014 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:52.530334  9014 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:52.531698  9332 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.216s	user 0.000s	sys 0.007s
I20260709 12:24:52.532730  9014 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:52.533521  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:52.534149  9153 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:52.495411  9305 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:52.496045  9000 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:52.496285  8752 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:52.497882  9092 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:52.500514  9303 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:52.482018  9301 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:24:52.428614  9337 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.509497  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:52.613714  5220 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
W20260709 12:24:52.535609  9269 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.471s	user 0.010s	sys 0.008s
I20260709 12:24:52.525708  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 787) took 2247 ms. Trace:
W20260709 12:24:52.484664  9308 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:24:52.524536  9191 consensus_meta.cc:287] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.136s	user 0.010s	sys 0.004s
I20260709 12:24:52.552985  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:52.665302  5148 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:52.614970  5223 rpcz_store.cc:276] 0709 12:24:50.259862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.259992 (+   130us) service_pool.cc:224] Handling call
0709 12:24:50.302148 (+ 42156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.302469 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:51.978365 (+1675896us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.488176 (+509811us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.506757 (+ 18581us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:52.506905 (+   148us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:52.507037 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.507234 (+   197us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.507460 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109202,"prepare.queue_time_us":163287,"prepare.run_cpu_time_us":161,"prepare.run_wall_time_us":16818,"raft.queue_time_us":174935,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":35195,"spinlock_wait_cycles":10159360,"thread_start_us":1949345,"threads_started":3,"wal-append.queue_time_us":372743}
I20260709 12:24:52.559666  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:52.576078  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 791) took 2263 ms. Trace:
I20260709 12:24:52.667745  5147 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:24:52.595094  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.511181  9042 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:52.516561  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 440875us
I20260709 12:24:52.511185  9202 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:52.613162  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:52.681380  5940 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:52.525671  9131 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:52.511315  8708 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:24:52.670351  5218 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.014s	sys 0.016s
I20260709 12:24:52.535497  9335 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.677830  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:52.667565  5149 rpcz_store.cc:276] 0709 12:24:50.312179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.987774 (+675595us) service_pool.cc:224] Handling call
0709 12:24:50.988241 (+   467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:50.988734 (+   493us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:51.710160 (+721426us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.230361 (+520201us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:52.236717 (+  6356us) log.cc:844] Serialized 52 byte log entry
0709 12:24:52.236965 (+   248us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:52.247428 (+ 10463us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.290298 (+ 42870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.290403 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.575726 (+285323us) raft_consensus.cc:1685] finished
0709 12:24:52.575755 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.575980 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68213,"prepare.queue_time_us":1175808,"prepare.run_cpu_time_us":279,"prepare.run_wall_time_us":966,"raft.queue_time_us":289669,"raft.run_cpu_time_us":615,"raft.run_wall_time_us":51692,"thread_start_us":1233257,"threads_started":2,"wal-append.queue_time_us":586027}
W20260709 12:24:52.683743  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 394075us
I20260709 12:24:52.698249  5147 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:52.699070  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:52.699555  5042 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:52.700517  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:52.700982  5042 leader_election.cc:400] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:24:52.701363  5042 leader_election.cc:403] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:24:52.701733  9327 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:52.702085  9153 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:24:52.702438  5042 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:52.704413  9153 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:52.704198  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:52.706153  9153 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:52.706451  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:52.708236  9071 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:52.708562  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:52.709569  9071 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:52.709923  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:52.709969  9071 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:52.711979  9071 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:52.707721  9153 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:52.714342  8917 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:52.714483  9153 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:52.702086  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:52.720103  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:52.686180  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:52.688323  5218 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.014s	sys 0.017s
I20260709 12:24:52.720897  5147 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:52.691823  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 271308us
I20260709 12:24:52.693980  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 797) took 2380 ms. Trace:
I20260709 12:24:52.439996  9322 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:52.613833  9324 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.093s	user 0.003s	sys 0.001s
I20260709 12:24:52.720307  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1344) took 1558 ms. Trace:
I20260709 12:24:52.730576  5217 rpcz_store.cc:276] 0709 12:24:51.162072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.162228 (+   156us) service_pool.cc:224] Handling call
0709 12:24:51.162745 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.976524 (+813779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.976559 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.977055 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 12:24:52.258002 (+280947us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:52.258055 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.258153 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.720058 (+461905us) raft_consensus.cc:1685] finished
0709 12:24:52.720078 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.720245 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18681,"raft.queue_time_us":37706,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":139,"thread_start_us":1069552,"threads_started":2,"wal-append.queue_time_us":326532}
I20260709 12:24:52.544231  9344 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:52.721588  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1343) took 1560 ms. Trace:
I20260709 12:24:52.572956  9305 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:52.723032  5146 rpcz_store.cc:276] 0709 12:24:50.313954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.028350 (+714396us) service_pool.cc:224] Handling call
0709 12:24:51.092338 (+ 63988us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:51.111709 (+ 19371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.111765 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:51.119687 (+  7922us) log.cc:844] Serialized 52 byte log entry
0709 12:24:51.937875 (+818188us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:52.276872 (+338997us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.300343 (+ 23471us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.412184 (+111841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.412290 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.693734 (+281444us) raft_consensus.cc:1685] finished
0709 12:24:52.693758 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.693912 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57552,"prepare.queue_time_us":363759,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":90,"raft.queue_time_us":357486,"raft.run_cpu_time_us":487,"raft.run_wall_time_us":33485,"thread_start_us":1187583,"threads_started":3,"wal-append.queue_time_us":987302}
I20260709 12:24:52.535404  9334 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.207s
I20260709 12:24:52.664934  9181 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:52.560595  9122 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:52.779771  9345 consensus_queue.cc:1059] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:52.779989  9181 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:52.679243  9042 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:52.695603  9313 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:52.687776  9269 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.747838  5218 rpcz_store.cc:276] 0709 12:24:51.161220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.161639 (+   419us) service_pool.cc:224] Handling call
0709 12:24:51.178787 (+ 17148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.179024 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:51.209784 (+ 30760us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.520784 (+1311000us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b18001e5320. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:52.721129 (+200345us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:52.721202 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:52.721257 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.721344 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.721485 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37174,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":72,"raft.queue_time_us":173689,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":23001,"spinlock_wait_cycles":9515904,"thread_start_us":1510888,"threads_started":2,"wal-append.queue_time_us":977764}
I20260709 12:24:52.697559  8975 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:52.776495  9305 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:52.777719  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:52.777346  9344 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:52.782181  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 100736us
I20260709 12:24:52.677465  9191 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:52.698206  9319 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:52.747704  9308 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:52.818418  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:52.819976  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:52.820384  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:52.821067  5073 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:52.822109  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:52.822373  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:52.823633  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:52.825001  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:52.825440  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 139301us
I20260709 12:24:52.825901  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:52.826876  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:52.828085  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:52.829677  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:52.830538  5223 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:52.830909  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:52.831488  5220 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:52.718009  8708 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:52.856351  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:52.856933  5941 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:52.857200  5941 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:52.785944  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:52.792474  9345 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:52.677826  9337 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:52.792692  9343 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:24:52.806149  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 360014us
I20260709 12:24:52.868968  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:52.785377  9169 consensus_meta.cc:287] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.000s
I20260709 12:24:52.809374  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:52.830561  5071 raft_consensus.cc:2410] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:24:52.837042  5939 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:52.877869  5220 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:52.843605  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:52.844409  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:52.880424  5940 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:52.801694  9350 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:52.850220  9149 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:52.851464  9305 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:52.802100  8917 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:24:52.855084  5075 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:52.855631  9191 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:52.855885  5946 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:52.856192  5224 raft_consensus.cc:2393] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:24:52.858585  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 72718.1us
I20260709 12:24:52.875944  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:52.913519  5146 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:52.868104  9202 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:52.832541  9311 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.481s	user 0.000s	sys 0.001s
I20260709 12:24:52.685636  5900 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:52.878026  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:52.846097  9220 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:52.859388  9318 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:24:52.882531  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:52.869771  5941 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:52.924217  9341 raft_consensus.cc:3055] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:52.924728  9341 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:52.925155  5941 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:52.885454  9169 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:52.885026  9363 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:52.864230  9315 consensus_meta.cc:287] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.001s
W20260709 12:24:52.885919  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 76639us
I20260709 12:24:52.885392  5076 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:52.887609  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:52.903324  5147 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:52.931989  5147 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:52.725996  9335 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:52.927119  5944 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:52.930220  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:52.931236  5939 raft_consensus.cc:2393] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 8: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:52.904731  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:52.801924  9269 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:52.864346  9358 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:52.880769  8752 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.924023  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:52.941501  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:52.907406  9215 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:24:52.922711  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 237151us
I20260709 12:24:52.834106  9319 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:52.930701  9326 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:24:52.928027  9169 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:52.967186  5146 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
W20260709 12:24:52.930099  9342 log.cc:927] Time spent T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:24:52.972256  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:52.930445  9343 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:52.970067  5900 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:52.874747  9345 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:52.928547  9363 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:52.909376  9149 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:52.930316  9153 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:52.932946  8975 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:24:53.000591  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:53.002028  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:52.983505  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:52.938076  5939 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:52.988644  5946 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:53.006038  5041 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:52.969734  9181 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:52.929790  9315 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:52.982299  5190 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:52.984457  9202 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:52.908133  9350 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:52.968876  9333 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:52.939157  9220 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:52.995052  9345 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:52.995747  9353 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.968612  9215 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:52.997880  5073 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:52.971751  9365 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.019953  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:53.047595  5147 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:52.990700  5941 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:52.968976  8752 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:52.982131  9343 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.982543  9379 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.022264  5900 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:53.026062  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44161.8us
I20260709 12:24:52.984208  9191 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:53.024612  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:53.035061  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:53.037017  5224 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35624 (request call id 1351) took 1791 ms (client timeout 2060 ms). Trace:
I20260709 12:24:53.038067  5221 raft_consensus.cc:1240] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:53.033710  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:52.996665  9149 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:52.991597  9363 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:53.021823  9169 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:52.925650  9341 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:52.961440  9358 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:53.055644  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:53.176049  5899 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:53.180603  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:53.042651  9326 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:24:53.102404  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 373644us
I20260709 12:24:53.033825  9181 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:53.406109  5217 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
W20260709 12:24:53.415028  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 239053us
I20260709 12:24:53.126302  9322 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:53.052030  5042 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:53.434458  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:53.033998  9333 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.131477  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:53.154702  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.438503  5223 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 15: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:53.389264  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 790) took 3077 ms. Trace:
W20260709 12:24:53.403797  5145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.005s	sys 0.007s
I20260709 12:24:53.431675  5899 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:53.185796  9372 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:24:53.190569  5222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1152) took 1605 ms. Trace:
I20260709 12:24:53.244489  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1341) took 2370 ms. Trace:
W20260709 12:24:53.051091  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 45115.8us
I20260709 12:24:53.459347  5222 rpcz_store.cc:276] 0709 12:24:51.585603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.670579 (+ 84976us) service_pool.cc:224] Handling call
0709 12:24:51.680937 (+ 10358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:52.200995 (+520058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.190265 (+989270us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b540067fcd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:53.190492 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1192413440}
I20260709 12:24:53.461091  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:53.121238  9342 log.cc:927] Time spent T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:24:53.407356  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 385140us
W20260709 12:24:53.437073  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 385110us
I20260709 12:24:53.411878  9343 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:53.351715  9310 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 1.247s	user 0.005s	sys 0.010s
I20260709 12:24:53.048372  9345 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.408452  5948 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:53.439610  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:53.244405  9042 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:53.162967  9269 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:53.437762  8917 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:53.116122  8965 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:53.158021  9362 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:53.048189  9215 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:53.422379  8975 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:53.347903  9385 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.179131  9014 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.405407  5224 rpcz_store.cc:269] 0709 12:24:51.245430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.245733 (+   303us) service_pool.cc:224] Handling call
0709 12:24:53.036756 (+1791023us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54004835d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:53.036940 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3680507648}
I20260709 12:24:53.473980  9000 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:24:53.178491  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:53.461935  5899 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:53.171526  9344 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:53.523329  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 61449.9us
I20260709 12:24:53.182466  9380 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.463155  5145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.005s	sys 0.009s
I20260709 12:24:53.458719  9298 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:53.244127  8752 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:53.460681  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:53.177428  9315 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:53.113447  9327 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:53.406396  9191 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:53.469031  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:53.469224  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:53.471333  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:53.537948  5076 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:24:53.179934  9220 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:53.408784  9169 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:53.431895  9181 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:53.527318  9122 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:53.430899  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:53.455165  5143 rpcz_store.cc:276] 0709 12:24:50.311868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.968951 (+657083us) service_pool.cc:224] Handling call
0709 12:24:50.969375 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.013349 (+ 43974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.013603 (+   254us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.018700 (+  5097us) log.cc:844] Serialized 26 byte log entry
0709 12:24:51.565874 (+547174us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:53.162747 (+1596873us) spinlock_profiling.cc:243] Waited 63.1 ms on lock 0x7b1800255dc0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a80e34bc 00007fd8a543148b 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8abf333c7 00007fd8a85459a7 00007fd8a85550bf
0709 12:24:53.162896 (+   149us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.354370 (+191474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.354475 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:53.354531 (+    56us) raft_consensus.cc:1685] finished
0709 12:24:53.388883 (+ 34352us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.389166 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3499,"prepare.queue_time_us":659415,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":235046,"raft.run_cpu_time_us":394,"raft.run_wall_time_us":14638,"spinlock_wait_cycles":145090944,"thread_start_us":1236891,"threads_started":3,"wal-append.queue_time_us":1105007}
I20260709 12:24:53.162870  9319 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:53.380650  9337 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:53.166743  9335 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:53.354609  9377 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:53.459403  5142 rpcz_store.cc:276] 0709 12:24:50.801084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.042123 (+241039us) service_pool.cc:224] Handling call
0709 12:24:51.042621 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.042930 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:51.567931 (+525001us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.170986 (+1603055us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b1800351820. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a80e34bc 00007fd8a543148b 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8a8533060 00007fd8a8545716 00007fd8a8556287
0709 12:24:53.171098 (+   112us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:53.171147 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:53.171192 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.171267 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.171384 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1599,"prepare.queue_time_us":1115812,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":154723,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":5988,"spinlock_wait_cycles":16056192,"thread_start_us":1328966,"threads_started":3,"wal-append.queue_time_us":854816,"wal-append.run_cpu_time_us":1468,"wal-append.run_wall_time_us":1003690}
I20260709 12:24:53.129647  9383 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:53.443058  9333 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:53.061267  9365 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:53.185086  9153 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:52.805166  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:53.494306  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1144) took 2007 ms. Trace:
I20260709 12:24:53.495932  9297 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:53.496879  9273 raft_consensus.cc:3055] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:53.498823  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1345) took 1563 ms. Trace:
W20260709 12:24:53.499019  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 59525us
I20260709 12:24:53.602219  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:53.535387  9344 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:53.168553  9071 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:53.050540  9353 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:53.507925  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1147) took 2018 ms. Trace:
I20260709 12:24:53.606884  5219 rpcz_store.cc:276] 0709 12:24:51.489253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.489360 (+   107us) service_pool.cc:224] Handling call
0709 12:24:51.489687 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.490061 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:51.517735 (+ 27674us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.201657 (+683922us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:52.523430 (+321773us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b2c000af528. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:24:52.523949 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 12:24:52.524117 (+   168us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:52.524161 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.524249 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:53.505835 (+981586us) raft_consensus.cc:1685] finished
0709 12:24:53.507663 (+  1828us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.507868 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7936,"prepare.queue_time_us":21387,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":4805,"raft.queue_time_us":135305,"raft.run_cpu_time_us":8022,"raft.run_wall_time_us":64080,"spinlock_wait_cycles":32561536,"thread_start_us":637981,"threads_started":1,"wal-append.queue_time_us":1153729}
I20260709 12:24:53.514221  5074 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:53.517551  9357 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:53.517726  9385 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:53.518947  8975 raft_consensus.cc:697] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:53.508212  9269 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:53.526110  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.507799  9362 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:53.612498  9362 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:53.613920  9362 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:53.615635  9357 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:53.616380  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1145) took 2128 ms. Trace:
I20260709 12:24:53.527097  9360 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:53.527292  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 793) took 3214 ms. Trace:
I20260709 12:24:53.618120  5150 rpcz_store.cc:276] 0709 12:24:50.312771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.023708 (+710937us) service_pool.cc:224] Handling call
0709 12:24:51.024109 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.024522 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.024563 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.025128 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 12:24:51.585757 (+560629us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:52.191122 (+605365us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.295879 (+104757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.295968 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:53.351041 (+1055073us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:53.526835 (+175794us) raft_consensus.cc:1685] finished
0709 12:24:53.526960 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.527141 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19110,"prepare.queue_time_us":651596,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"raft.queue_time_us":112720,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":45907,"spinlock_wait_cycles":19444992,"thread_start_us":1244145,"threads_started":3,"wal-append.queue_time_us":1050430}
W20260709 12:24:53.527621  9340 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.999s	user 0.003s	sys 0.000s
I20260709 12:24:53.528221  9392 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.529771  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.622655  9303 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:53.625005  9385 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:53.625653  9269 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:53.628360  8975 consensus_queue.cc:237] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:53.535398  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 74824.2us
I20260709 12:24:53.626840  9365 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:53.515900  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:53.541697  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:53.543270  9396 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.559265  9397 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:53.561576  9191 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:53.405431  9379 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:53.538159  5217 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:53.526765  9400 consensus_queue.cc:1059] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:24:53.529038  9042 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:53.568491  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:53.622560  9392 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:53.522153  9370 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:53.165747  9350 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:53.576462  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 76421.6us
I20260709 12:24:53.587980  9377 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:53.647991  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:53.578701  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:53.579452  5072 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:53.544620  8917 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:53.584728  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:53.574227  9169 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:53.587126  9383 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:53.588846  9393 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.589352  9391 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:24:53.532310  9014 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:53.577849  9181 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:53.590342  5215 rpcz_store.cc:276] 0709 12:24:51.487367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.487509 (+   142us) service_pool.cc:224] Handling call
0709 12:24:51.509707 (+ 22198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.057392 (+547685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.493911 (+1436519us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b5400163d54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:53.494207 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1253925120}
I20260709 12:24:53.540107  9380 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:53.601234  5141 rpcz_store.cc:276] 0709 12:24:51.935528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.935656 (+   128us) service_pool.cc:224] Handling call
0709 12:24:51.962423 (+ 26767us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.987713 (+ 25290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.987755 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.988397 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 12:24:52.258404 (+270007us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:52.293342 (+ 34938us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.718155 (+424813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.718235 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:53.498561 (+780326us) raft_consensus.cc:1685] finished
0709 12:24:53.498588 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.498755 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30445,"prepare.queue_time_us":40741,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":76,"raft.queue_time_us":411907,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":10047,"spinlock_wait_cycles":7634176,"thread_start_us":618756,"threads_started":2,"wal-append.queue_time_us":569366}
I20260709 12:24:53.584015  9122 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:53.298177  9313 consensus_meta.cc:287] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.005s
W20260709 12:24:53.598140  9386 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 12:24:53.604657  5190 leader_election.cc:400] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11.
I20260709 12:24:53.605588  9219 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:53.504813  9322 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:24:53.631281  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 115480us
I20260709 12:24:53.632264  5146 raft_consensus.cc:2376] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13. Re-sending same reply.
I20260709 12:24:53.601140  9273 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:53.680259  5115 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:53.633569  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 714405us
I20260709 12:24:53.599634  9297 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:53.684896  9398 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:24:53.686589  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:53.687297  5067 raft_consensus.cc:2376] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12. Re-sending same reply.
I20260709 12:24:53.687613  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:53.687928  5071 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:53.634650  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:53.689050  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:53.616954  5221 rpcz_store.cc:276] 0709 12:24:51.487721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.488142 (+   421us) service_pool.cc:224] Handling call
0709 12:24:53.037871 (+1549729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.409289 (+371418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.616058 (+206769us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54004835d0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:53.616287 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3563117312}
W20260709 12:24:53.605406  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 75165.7us
I20260709 12:24:53.690394  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:53.690476  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:53.643877  5145 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:53.503208  9345 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:53.695863  9345 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:53.649426  8917 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:53.696676  8917 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:53.650462  9400 raft_consensus.cc:3055] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:53.697757  9400 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:53.652472  8708 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:53.654613  5114 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:53.639374  9385 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:53.645768  9358 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:53.670368  5190 leader_election.cc:403] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:24:53.713428  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:53.714385  5221 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:24:53.647351  9396 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:53.655995  9350 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:53.688293  5073 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:53.700284  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45763.1us
I20260709 12:24:53.669996  9269 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:53.689414  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:53.741004  5076 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:53.676591  9357 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:53.708349  9392 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:53.708619  9370 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:53.710613  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 76087.7us
I20260709 12:24:53.678025  9322 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:53.673774  9313 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:53.738750  5114 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:53.670081  9319 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:24:53.700336  5147 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:53.761583  5221 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:53.744943  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:53.662472  9393 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:53.708400  8965 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:53.712257  9385 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:53.719692  9302 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:53.737008  9365 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:53.728775  9373 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:53.751788  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:53.672487  9220 consensus_queue.cc:1059] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:24:53.734731  9153 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:24:53.756577  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 152580us
I20260709 12:24:53.753713  9322 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:53.767266  5941 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:53.772030  5114 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:53.745939  9219 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11.
I20260709 12:24:53.761082  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:53.775409  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:53.774874  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:53.803723  5947 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:53.775979  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:53.776131  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:53.757157  9370 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:53.770581  9298 raft_consensus.cc:3055] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:53.742300  9297 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:53.785264  5147 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:53.786651  5899 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:53.672910  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:53.768298  9042 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:53.790920  5145 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:53.791563  8917 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:53.792755  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:53.793912  9379 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:53.759800  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:53.774294  9149 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:53.797631  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:53.798413  5142 raft_consensus.cc:2376] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12. Re-sending same reply.
I20260709 12:24:53.735236  9401 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:24:53.805994  5898 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:53.773098  9385 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:53.709373  9383 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:53.808451  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:53.775063  9373 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.855666  5898 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:53.857275  9313 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:53.857894  9313 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:53.858340  9313 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:53.859257  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:53.831919  9406 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:53.765522  9327 consensus_queue.cc:1059] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
W20260709 12:24:53.878998  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 70680.5us
I20260709 12:24:53.838825  9297 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:53.765024  9350 consensus_queue.cc:1059] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:24:53.831110  9325 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:53.833508  5219 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:53.805119  9269 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:53.809451  8965 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:53.845654  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:53.809417  9298 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:53.814863  9302 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:53.794934  9365 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:53.800814  9319 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:24:53.815178  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:53.876127  9406 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.832816  9071 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:53.868988  9401 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:53.872499  9393 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:53.874029  9404 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:53.832562  9380 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:53.880966  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:53.829414  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 156578us
I20260709 12:24:53.874706  9153 raft_consensus.cc:697] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:53.835282  9149 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:53.771834  9122 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:53.772102  9400 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:53.861742  9373 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:53.888321  5145 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:53.930918  9400 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:53.931591  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:53.886233  9271 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:53.935138  9360 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.935813  9360 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:53.937732  9315 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:53.938090  9302 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:53.938687  9302 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.938422  9315 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:53.939107  9302 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:53.889808  9350 raft_consensus.cc:3055] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:53.887815  8965 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:53.940548  9411 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:24:53.940953  8965 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:53.886706  9402 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:24:53.941604  9220 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 12:24:53.942296  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.908636  9325 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:24:53.912783  9149 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:53.903568  9406 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:53.907397  9404 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:53.833134  9042 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:53.861596  9312 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:53.865406  8917 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:53.895396  5073 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:53.855340  9353 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:53.874779  9000 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:24:53.908572  9056 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:53.941385  9153 consensus_queue.cc:237] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:53.912041  5189 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:24:53.891040  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 93589.4us
I20260709 12:24:53.883733  9363 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:53.954043  9416 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:53.957242  9344 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:53.978593  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:53.974864  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:53.987468  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 75482.1us
I20260709 12:24:53.940754  9350 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:24:53.989473  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.907074  9418 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:53.992751  5189 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:53.996840  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:54.000432  9352 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:53.971230  9056 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:53.973778  9071 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:54.009915  9187 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:53.973613  9312 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:54.015558  5189 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:53.980405  9413 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:54.001135  9352 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:53.905861  9191 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:53.986586  9343 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:54.026043  9127 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:54.026582  9127 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:54.026993  9127 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:54.027822  9056 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.029184  9056 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:54.033146  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:54.034013  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.036325  5219 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:54.037715  9413 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:54.038874  5219 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:53.965489  9298 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:54.039907  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:54.001678  8917 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:54.041385  5947 raft_consensus.cc:2368] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11.
W20260709 12:24:54.044034  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 208861us
I20260709 12:24:54.051060  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:54.056573  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:54.020560  9071 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:54.073896  5224 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:54.073489  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 78092.9us
I20260709 12:24:54.075873  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:54.086795  5041 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:54.053876  8917 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:54.094944  9215 consensus_meta.cc:287] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.000s
I20260709 12:24:54.088330  9361 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.038758  9413 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.017269  9418 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.098556  9033 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:54.119372  9418 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.120708  9423 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:54.106775  9419 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:24:54.060473  9333 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:54.004369  9416 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.125180  9127 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:54.137168  5944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 973) took 2091 ms (client timeout 2366 ms). Trace:
I20260709 12:24:54.098706  9169 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:54.097774  9072 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 12:24:54.140251  5944 rpcz_store.cc:269] 0709 12:24:52.045623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.049016 (+  3393us) service_pool.cc:224] Handling call
0709 12:24:54.136906 (+2087890us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b54002d78d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:54.137097 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2015670016}
I20260709 12:24:54.110177  9033 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:54.143772  5218 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:54.104950  9215 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:54.149717  5220 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:54.103753  9352 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:54.152979  9215 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:54.110345  9149 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.152837  9033 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.108264  9361 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.156627  9033 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:54.157923  9361 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.160651  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 1106) took 1342 ms. Trace:
I20260709 12:24:54.161124  5072 rpcz_store.cc:276] 0709 12:24:52.818218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.825651 (+  7433us) service_pool.cc:224] Handling call
0709 12:24:54.160425 (+1334774us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b540058aa54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:54.160552 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1705025280}
I20260709 12:24:54.166409  9396 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.166617  8917 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:54.168123  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1213) took 1167 ms. Trace:
I20260709 12:24:54.168582  5142 rpcz_store.cc:276] 0709 12:24:53.000452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.584420 (+583968us) service_pool.cc:224] Handling call
0709 12:24:54.167924 (+583504us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54000ebd54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:54.168045 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343553792}
I20260709 12:24:54.169965  9191 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:54.171926  9333 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.141494  9181 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:54.175796  9181 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.142402  9042 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:54.175886  9303 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.552s	user 0.014s	sys 0.000s
I20260709 12:24:54.102626  9312 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.135313  8965 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.179800  9149 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.149822  9363 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.133327  9071 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:54.186159  9421 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:54.196560  5149 raft_consensus.cc:2410] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 13 because replica has last-logged OpId of term: 12 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:24:54.195312  9396 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:54.202641  9392 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.198999  9337 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:54.212707  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 125989us
I20260709 12:24:54.213671  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.213655  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.216002  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:54.217962  9422 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.114s	user 0.000s	sys 0.002s
I20260709 12:24:54.213395  9392 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:54.222059  9416 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.214459  9337 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:54.224993  9337 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:54.227874  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.229276  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.233736  9153 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:54.211417  9313 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.236295  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.236300  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1220) took 1226 ms. Trace:
I20260709 12:24:54.240252  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.237471  5041 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:54.238178  5142 rpcz_store.cc:276] 0709 12:24:53.010088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.194864 (+1184776us) service_pool.cc:224] Handling call
0709 12:24:54.195337 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.195661 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:54.196037 (+   376us) log.cc:844] Serialized 14 byte log entry
0709 12:24:54.235760 (+ 39723us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:54.235823 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:54.235882 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.235957 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.236095 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1810,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":81,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":2548,"thread_start_us":39273,"threads_started":1,"wal-append.queue_time_us":41959}
I20260709 12:24:54.242772  5142 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:24:54.243374  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1221) took 1233 ms. Trace:
I20260709 12:24:54.229133  9416 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:54.251367  5068 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:54.212749  9396 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:54.243681  5142 rpcz_store.cc:276] 0709 12:24:53.010402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.242166 (+1231764us) service_pool.cc:224] Handling call
0709 12:24:54.242521 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.243141 (+   620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.243314 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.243919  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.245069  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:54.248780  5115 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.250064  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:54.263806  5145 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:54.252174  5042 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:54.252502  5076 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:54.252864  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:54.254153  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.264609  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:54.268466  5073 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:24:54.276764  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:54.277508  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.277727  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:54.278384  5142 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 8.
I20260709 12:24:54.291144  5075 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:54.292172  5070 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:54.282480  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.284487  5145 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:54.288930  5073 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:54.296680  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:54.296736  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.297056  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.281872  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.298460  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.298861  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.299626  5069 raft_consensus.cc:2393] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:54.282876  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.293733  5944 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:54.279469  5067 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:54.293465  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.293968  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:54.255934  9149 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:54.301553  5068 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:54.304790  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.343067  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.343896  5070 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
I20260709 12:24:54.340076  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.303929  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.344560  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:54.340534  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.345204  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.341591  9127 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:54.348039  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.348874  5067 raft_consensus.cc:2410] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:24:54.351011  9273 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.352349  9273 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.354949  9313 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:54.358960  5071 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:54.359454  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:54.386217  5944 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:54.386235  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:54.292654  9415 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:54.387885  5898 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:54.394302  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.387674  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:54.359884  5073 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:54.392171  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.392359  5145 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:24:54.278980  9337 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:54.279584  8917 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:54.394412  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.298072  9361 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.294361  9432 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.285804  9271 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:54.396235  5898 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:54.399367  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.278817  9431 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:54.403862  9361 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.409621  9431 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:54.411542  9435 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:24:54.398578  9365 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:54.396561  9149 consensus_queue.cc:1059] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:24:54.388222  9334 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.398371  9415 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:54.433092  9334 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.393636  9302 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:54.413527  9432 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:54.295856  9153 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:54.408349  9337 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 11 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:54.443775  9396 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.444458  9337 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:54.449597  9373 consensus_meta.cc:287] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.520s	user 0.004s	sys 0.004s
I20260709 12:24:54.450492  9343 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:54.423547  9404 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:54.429363  9436 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:54.430039  9365 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:54.437526  9394 raft_consensus.cc:3055] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:54.450171  9373 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:54.400816  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:54.388818  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:54.512120  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 124256us
I20260709 12:24:54.514181  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:54.513911  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:54.514858  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.515333  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1217) took 1513 ms. Trace:
W20260709 12:24:54.517010  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 120674us
I20260709 12:24:54.405383  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.517524  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:54.522568  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.587371  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.606392  5145 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:54.387952  9072 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:54.386518  9127 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:54.609027  9127 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:54.610987  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.403043  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.512120  5067 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:54.584653  9358 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:54.417423  9297 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:54.459770  9404 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:54.596266  5147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 812) took 1566 ms (client timeout 1942 ms). Trace:
W20260709 12:24:54.427804  9322 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.001s
W20260709 12:24:54.613255  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 1.14708e+06us
I20260709 12:24:54.457969  9394 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:54.463820  9406 consensus_meta.cc:287] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.000s
I20260709 12:24:54.463958  9432 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:54.460868  9436 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:54.562863  9365 raft_consensus.cc:697] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:54.567771  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1218) took 1558 ms. Trace:
I20260709 12:24:54.608927  9072 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:54.462046  9415 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:54.593670  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.607872  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:54.615036  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1061) took 1562 ms. Trace:
W20260709 12:24:54.623409  5149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 803) took 2069 ms (client timeout 2272 ms). Trace:
I20260709 12:24:54.617244  9322 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:54.633894  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.635456  9072 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:54.619632  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.636746  9219 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:54.613484  9127 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:54.638379  9424 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:54.640445  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:54.575428  9334 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.614897  9297 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.588486  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.573179  9361 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:54.615965  5147 rpcz_store.cc:269] 0709 12:24:53.029205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.050694 (+ 21489us) service_pool.cc:224] Handling call
0709 12:24:54.595621 (+1544927us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400710054. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:54.595937 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674180480}
I20260709 12:24:54.614248  9358 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:54.650647  9219 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:54.655205  9322 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.656118  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:54.648267  5149 rpcz_store.cc:269] 0709 12:24:52.555073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.701808 (+146735us) service_pool.cc:224] Handling call
0709 12:24:54.622907 (+1921099us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54003cda54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:54.623183 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3398082432}
I20260709 12:24:54.648146  5941 rpcz_store.cc:276] 0709 12:24:53.052124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.054491 (+  2367us) service_pool.cc:224] Handling call
0709 12:24:53.054945 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.797169 (+742224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.614278 (+817109us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b5400005854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:54.614771 (+   493us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1637485312}
I20260709 12:24:54.636665  9436 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:54.659680  9297 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:54.620599  9406 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.647845  9153 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:54.662349  9358 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.682206  5946 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:54.666009  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:54.678174  9269 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:54.678524  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:54.619639  9394 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:54.698653  5948 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:54.681408  9219 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.701440  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:54.706576  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:54.706863  5945 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:54.684139  9365 consensus_queue.cc:237] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:54.710824  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:54.717579  5075 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:54.720036  9406 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:54.723716  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.724910  5147 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:54.729120  9394 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:54.729691  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:54.729947  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 141904us
I20260709 12:24:54.733292  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.714321  9424 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:54.741581  9373 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:54.742461  9415 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.743350  9415 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:54.747040  9432 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.747272  9370 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:54.747749  9432 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:54.748540  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 117930us
W20260709 12:24:54.749342  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.751137  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:54.722628  9302 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:54.752497  9373 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:54.611878  5143 rpcz_store.cc:276] 0709 12:24:53.009062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.649111 (+640049us) service_pool.cc:224] Handling call
0709 12:24:53.649641 (+   530us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:53.650079 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.650110 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:53.650778 (+   668us) log.cc:844] Serialized 52 byte log entry
0709 12:24:53.736565 (+ 85787us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:53.753676 (+ 17111us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.796002 (+ 42326us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.827332 (+ 31330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.827425 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:53.935275 (+107850us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b1800371560. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:54.428133 (+492858us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b1800371560. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:54.514273 (+ 86140us) raft_consensus.cc:1685] finished
0709 12:24:54.514368 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.514867 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96266,"prepare.queue_time_us":40455,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":115,"raft.queue_time_us":54298,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":2810,"spinlock_wait_cycles":24759168,"thread_start_us":85184,"threads_started":1,"wal-append.queue_time_us":182878}
I20260709 12:24:54.753007  9373 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:54.753648  9373 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.755682  9377 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.746876  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:54.761755  9385 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.896s	user 0.003s	sys 0.016s
I20260709 12:24:54.722805  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:54.769315  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.769688  5940 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
W20260709 12:24:54.771080  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 107972us
I20260709 12:24:54.771318  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:54.774049  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:54.774931  5147 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:54.776073  9361 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:54.626003  5141 rpcz_store.cc:276] 0709 12:24:53.009458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.661243 (+651785us) service_pool.cc:224] Handling call
0709 12:24:53.672868 (+ 11625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.697913 (+ 25045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.697945 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.698502 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 12:24:53.821996 (+123494us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:53.861649 (+ 39653us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.884366 (+ 22717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.566601 (+682235us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b54006e2654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:54.566722 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.566838 (+   116us) raft_consensus.cc:1685] finished
0709 12:24:54.567044 (+   206us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.567513 (+   469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22429,"prepare.queue_time_us":113423,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":44386,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":2724,"spinlock_wait_cycles":15185792,"thread_start_us":162225,"threads_started":2,"wal-append.queue_time_us":171761}
I20260709 12:24:54.741250  9424 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:54.786190  9359 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:54.786972  9359 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:54.773166  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:54.783632  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.789662  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:54.799237  5141 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:54.800278  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1222) took 1789 ms. Trace:
I20260709 12:24:54.800653  5141 rpcz_store.cc:276] 0709 12:24:53.010684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.792421 (+1781737us) service_pool.cc:224] Handling call
0709 12:24:54.798647 (+  6226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.799928 (+  1281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.800189 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:54.801141  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.808323  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:54.806125  9377 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:54.814167  5900 connection.cc:256] RPC call timeout handler was delayed by -1.21147757000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 12:24:54.814283  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:54.817145  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 84109.6us
I20260709 12:24:54.817698  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:54.818125  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:54.818338  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.819267  5219 raft_consensus.cc:2393] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:54.820372  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:54.821008  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:54.788632  9359 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:54.822513  9415 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:54.825812  5940 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:24:54.827126  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.827976  5940 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:24:54.828553  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:54.829087  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.785178  9153 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:54.829792  9377 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:54.789860  9385 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:54.830394  9302 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:24:54.832396  9446 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.071s	user 0.002s	sys 0.000s
I20260709 12:24:54.844857  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:54.833258  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.846316  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.801977  9406 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:54.845857  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:54.843446  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1214) took 1842 ms. Trace:
W20260709 12:24:54.878921  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 74156.8us
I20260709 12:24:54.879379  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:54.839051  9313 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:54.883224  9383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.935s	user 0.014s	sys 0.007s
W20260709 12:24:54.883730  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 246303us
I20260709 12:24:54.843381  9334 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:54.883692  5900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.505s (SENT) [suppressed 15 similar messages]
I20260709 12:24:54.838105  9440 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:54.838617  9362 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:54.849432  8965 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:54.851243  9014 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:54.893831  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:54.880539  5150 rpcz_store.cc:276] 0709 12:24:53.000755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.620314 (+619559us) service_pool.cc:224] Handling call
0709 12:24:53.620682 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.666110 (+ 45428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.666153 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:53.666211 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:53.666250 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.842814 (+1176564us) spinlock_profiling.cc:243] Waited 44.9 ms on lock 0x7b540036ef54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:54.842913 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.843217 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":495,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":8362,"spinlock_wait_cycles":103298304}
W20260709 12:24:54.889153  9383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.014s	sys 0.008s
I20260709 12:24:54.833395  9373 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:54.905272  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.908655  5940 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:24:54.909569  5223 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:24:54.907506  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:54.904892  5075 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:54.909022  5149 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:54.912165  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:54.923089  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:54.918062  5941 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:54.900108  9269 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:54.925848  5141 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:54.929164  5147 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:54.930826  5948 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:54.883558  9313 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.933266  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:54.921844  9424 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:54.931527  9440 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:54.942338  9424 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:54.943809  9424 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:54.945088  9440 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:54.945704  5068 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:54.945720  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:54.966856  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:54.949642  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 811) took 1903 ms. Trace:
I20260709 12:24:54.949759  9362 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:54.950780  9169 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:54.956185  9322 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:54.976435  5898 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:54.977608  9360 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:54.979514  9271 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.019s
I20260709 12:24:54.980266  9271 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:54.981382  5142 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:54.982026  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:54.983306  9429 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.980268  9169 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:54.984829  9169 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.986019  9431 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:54.961956  9385 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:54.962402  9298 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:54.967785  5141 rpcz_store.cc:276] 0709 12:24:53.028174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.806908 (+1778734us) service_pool.cc:224] Handling call
0709 12:24:54.897447 (+ 90539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.928306 (+ 30859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.931759 (+  3453us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.895305  9406 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:54.967340  5219 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
W20260709 12:24:54.986721  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 47654.3us
I20260709 12:24:54.983789  9313 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:54.989045  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:54.994575  5041 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:54.985473  9169 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:54.911598  9334 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.009358  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.012866  5219 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
W20260709 12:24:55.016325  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 62995.4us
I20260709 12:24:54.911803  9358 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.030843  5145 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:24:55.026684  5222 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
W20260709 12:24:55.029840  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 198339us
I20260709 12:24:55.034473  5150 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 12:24:55.015597  9406 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:55.036479  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1224) took 1521 ms. Trace:
I20260709 12:24:55.039218  5141 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:55.040781  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.042029  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:55.043908  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:55.044622  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.046638  5068 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:55.048509  5150 rpcz_store.cc:276] 0709 12:24:53.515512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.918148 (+1402636us) service_pool.cc:224] Handling call
0709 12:24:55.000267 (+ 82119us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:55.036017 (+ 35750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.036414 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:55.050984  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 56567.3us
I20260709 12:24:55.053083  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:55.072854  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:55.074597  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:55.076943  9350 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.079047  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.080148  5073 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 15: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:55.081154  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.084030  5221 raft_consensus.cc:1240] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 12:24:55.085465  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 201801us
I20260709 12:24:55.088600  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:55.103642  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.103823  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:55.105034  5070 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 9: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:55.126719  5222 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:55.101496  9271 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:55.136842  5150 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:24:55.139436  9416 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:55.140120  9416 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.140195  9197 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:55.140571  9416 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:55.140900  9197 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:55.141470  9197 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:55.144883  5944 consensus_meta.cc:287] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.010s
I20260709 12:24:55.145514  5944 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
W20260709 12:24:55.146462  9409 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 1.257s	user 0.005s	sys 0.000s
I20260709 12:24:55.149978  5946 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:55.127146  9394 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:55.131749  9373 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:55.157152  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 108771us
I20260709 12:24:55.157732  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:55.158948  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 978) took 1180 ms. Trace:
I20260709 12:24:55.159685  9362 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:55.138603  9453 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:24:55.163993  5947 rpcz_store.cc:276] 0709 12:24:53.978044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.978221 (+   177us) service_pool.cc:224] Handling call
0709 12:24:55.158471 (+1180250us) spinlock_profiling.cc:243] Waited 51 ms on lock 0x7b54002d7dd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:55.158721 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117374720}
I20260709 12:24:55.154728  9394 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:55.177909  9404 consensus_meta.cc:287] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.536s	user 0.013s	sys 0.003s
I20260709 12:24:55.179191  5143 raft_consensus.cc:2410] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:55.174098  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.190951  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.190912  9215 consensus_queue.cc:1059] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:55.192643  9215 raft_consensus.cc:3055] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:55.193081  9215 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:55.194190  5221 raft_consensus.cc:3055] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:55.194707  5221 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:55.195859  5221 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.196974  5221 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:55.171859  9394 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:55.198342  5218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 1158) took 2020 ms (client timeout 2632 ms). Trace:
W20260709 12:24:55.198854  5218 rpcz_store.cc:269] 0709 12:24:53.177351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.178078 (+   727us) service_pool.cc:224] Handling call
0709 12:24:55.197801 (+2019723us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b540031d8d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:55.198089 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1442685696}
W20260709 12:24:55.202736  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.204869  5188 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:55.206470  5216 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:24:55.183962  9358 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:55.193516  9392 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:55.212646  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.212870  5947 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:55.214087  9153 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:55.214845  9153 raft_consensus.cc:697] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:24:55.215479  9448 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:24:55.216226  9153 consensus_queue.cc:237] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:55.220100  9297 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.543s	user 0.015s	sys 0.001s
I20260709 12:24:55.225917  5115 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:55.228363  5115 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:55.229454  9392 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:55.231653  5115 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:55.234236  9215 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:55.236711  9215 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.239825  9000 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.240542  9000 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:55.242167  5147 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:24:55.242609  5147 pending_rounds.cc:85] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 6
I20260709 12:24:55.242877  5147 pending_rounds.cc:107] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:24:55.243188  5147 raft_consensus.cc:2889] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.243456  9373 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:55.243917  9373 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.244300  9373 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:55.245107  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.243175  9000 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:55.250432  9169 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:55.255475  9188 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:55.231878  9392 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:55.256093  9188 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:55.260015  9358 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:55.263376  5947 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:55.263716  5944 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:55.266819  9341 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:55.271421  9273 consensus_meta.cc:287] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.918s	user 0.008s	sys 0.020s
I20260709 12:24:55.282051  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 807) took 1570 ms. Trace:
I20260709 12:24:55.282840  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 977) took 1306 ms. Trace:
I20260709 12:24:55.283231  5945 rpcz_store.cc:276] 0709 12:24:53.976324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.976525 (+   201us) service_pool.cc:224] Handling call
0709 12:24:53.996376 (+ 19851us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.724133 (+727757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.282526 (+558393us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b54002422d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:55.282770 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1626063104}
I20260709 12:24:55.282560  5216 rpcz_store.cc:276] 0709 12:24:53.712065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.712602 (+   537us) service_pool.cc:224] Handling call
0709 12:24:55.205742 (+1493140us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.207107 (+  1365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.281793 (+ 74686us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540031d8d0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:55.281948 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3374625024}
I20260709 12:24:55.279363  9169 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:55.287704  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1350) took 1002 ms. Trace:
I20260709 12:24:55.288126  5143 rpcz_store.cc:276] 0709 12:24:54.285457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.019768 (+734311us) service_pool.cc:224] Handling call
0709 12:24:55.287464 (+267696us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540036f454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:55.287620 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240974720}
I20260709 12:24:55.288126  9197 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:55.291980  9432 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:55.292464  9197 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:55.294169  9430 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.894s	user 0.002s	sys 0.002s
I20260709 12:24:55.220818  9297 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:55.174465  9355 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.985s	user 0.011s	sys 0.015s
I20260709 12:24:55.297662  9042 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:55.257406  9188 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:55.303016  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 69873.5us
W20260709 12:24:55.181340  9410 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 1.310s	user 0.004s	sys 0.000s
I20260709 12:24:55.306423  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1108) took 2487 ms. Trace:
I20260709 12:24:55.306945  9334 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:55.307302  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1230) took 1059 ms. Trace:
I20260709 12:24:55.296464  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:55.317095  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 109411us
W20260709 12:24:55.331911  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.325237  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:55.337636  5142 rpcz_store.cc:276] 0709 12:24:54.248081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.293675 (+ 45594us) service_pool.cc:224] Handling call
0709 12:24:55.307079 (+1013404us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b540077fbd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:55.307242 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1561425152}
I20260709 12:24:55.333374  5074 rpcz_store.cc:276] 0709 12:24:52.818745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.818881 (+   136us) service_pool.cc:224] Handling call
0709 12:24:52.845910 (+ 27029us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.610831 (+764921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.305846 (+1695015us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b540058a054. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:55.306209 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1536482688}
W20260709 12:24:55.320644  9355 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.207s	user 0.012s	sys 0.016s
W20260709 12:24:55.342010  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 53764.3us
I20260709 12:24:55.352355  5224 raft_consensus.cc:2393] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:24:55.352631  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.356500  5219 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:55.354421  5071 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:55.356336  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:55.336894  9365 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.000s	sys 0.011s
I20260709 12:24:55.354216  9352 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:55.369721  9415 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:55.370584  5067 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 7: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:55.356757  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:55.356261  5946 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:55.376677  5945 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:24:55.380985  8752 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:55.381515  9271 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:55.382318  9392 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.382570  9271 raft_consensus.cc:3055] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:55.383090  9271 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:55.354815  9325 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.383827  9325 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:55.384136  9271 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.355262  9317 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:55.385867  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:55.385725  9317 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:55.385625  9271 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:55.386766  5944 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:55.386811  5216 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:55.352767  9440 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:24:55.388041  9317 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:55.381512  8752 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:55.389108  8752 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:55.390406  9297 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:55.390918  9188 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:55.393016  9377 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:55.393779  5071 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:24:55.394250  9377 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:55.394682  9377 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:55.395880  5946 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:55.395748  9377 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:55.397100  9377 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:55.399663  9355 raft_consensus.cc:3055] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:55.400277  9355 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:55.401523  9355 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.402848  9355 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:55.403435  5188 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:55.404858  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:55.382911  9392 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:55.406757  9197 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:24:55.419298  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 123009us
I20260709 12:24:55.426949  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:55.431663  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:55.435899  8752 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:55.436344  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:55.438155  5946 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:55.438493  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1216) took 2437 ms. Trace:
I20260709 12:24:55.442893  9416 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:55.445880  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:55.371646  9365 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.000s	sys 0.011s
I20260709 12:24:55.476415  5945 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
W20260709 12:24:55.481755  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 45509.4us
I20260709 12:24:55.440388  9377 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:55.386180  9325 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:55.491721  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.499631  5041 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:55.475346  9197 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:55.476737  5144 rpcz_store.cc:276] 0709 12:24:53.001343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.638086 (+636743us) service_pool.cc:224] Handling call
0709 12:24:53.640791 (+  2705us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.641324 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.641371 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.647465 (+  6094us) log.cc:844] Serialized 26 byte log entry
0709 12:24:53.801861 (+154396us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:53.801921 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.802019 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.180394 (+1378375us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:55.437961 (+257567us) raft_consensus.cc:1685] finished
0709 12:24:55.437995 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.438293 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":802,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":3654,"spinlock_wait_cycles":6995328,"thread_start_us":126207,"threads_started":1,"wal-append.queue_time_us":191791}
I20260709 12:24:55.509690  9297 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:55.510486  8752 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:55.509764  9322 consensus_queue.cc:1059] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
W20260709 12:24:55.511883  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 108557us
I20260709 12:24:55.512450  9322 raft_consensus.cc:3055] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:55.512907  9322 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:55.513151  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:55.509896  9271 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.516261  9271 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:55.518854  9322 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:55.520265  9322 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:55.486075  9298 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:55.523337  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:55.525231  5221 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:55.531543  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:24:55.538349  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 92592.1us
I20260709 12:24:55.520844  9269 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:55.544687  9424 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:55.533669  9356 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.489975  9373 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:55.525787  9298 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:55.536101  9361 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:55.506753  9325 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:24:55.534612  9297 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:55.560336  9271 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:55.559898  9197 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.563373  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:55.547816  9356 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.550436  9298 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:55.562105  9197 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:55.571703  5074 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:55.572027  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.538375  9319 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:24:55.571542  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.578547  9394 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:55.569288  9219 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:24:55.577346  9350 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:55.587790  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 96469.3us
I20260709 12:24:55.568176  9356 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.595679  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:55.601872  5075 raft_consensus.cc:2393] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:24:55.602417  9465 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:24:55.602725  5067 raft_consensus.cc:2368] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16.
I20260709 12:24:55.602743  5072 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:55.604789  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.607779  5072 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:55.618755  9315 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:55.604916  9215 raft_consensus.cc:3055] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:55.626714  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.627336  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.628207  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:55.623023  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.628471  5068 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 16: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:55.621876  9416 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:55.636663  9466 log.cc:927] Time spent T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:24:55.637780  9424 raft_consensus.cc:3055] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:55.638345  9424 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:55.638548  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.639546  9424 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:55.640865  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:55.641124  9424 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.642825  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:55.657191  5217 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:24:55.644758  9464 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:24:55.645767  9269 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:55.635715  5073 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:55.634065  5041 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:55.649977  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.656258  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1228) took 1408 ms. Trace:
I20260709 12:24:55.659529  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:55.660874  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:55.649487  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:55.659529  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:55.666148  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.666966  5219 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:24:55.667362  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:55.668076  5947 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:55.667688  9269 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.623641  9215 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:55.668145  5945 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:24:55.675498  5221 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:55.664127  5149 rpcz_store.cc:276] 0709 12:24:54.247455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.964113 (+716658us) service_pool.cc:224] Handling call
0709 12:24:55.024785 (+ 60672us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:55.070943 (+ 46158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.070997 (+    54us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:55.132513 (+ 61516us) log.cc:844] Serialized 78 byte log entry
0709 12:24:55.335900 (+203387us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:55.457905 (+122005us) log.cc:844] Serialized 14 byte log entry
0709 12:24:55.477732 (+ 19827us) log.cc:844] Serialized 14 byte log entry
0709 12:24:55.479503 (+  1771us) log.cc:844] Serialized 14 byte log entry
0709 12:24:55.479839 (+   336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.479955 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.655953 (+175998us) raft_consensus.cc:1685] finished
0709 12:24:55.655975 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.656194 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28051,"prepare.queue_time_us":576666,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":2728,"raft.queue_time_us":60386,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":16190,"spinlock_wait_cycles":11151232,"thread_start_us":286395,"threads_started":2,"wal-append.queue_time_us":209942}
I20260709 12:24:55.684782  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.689182  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.688138  5217 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:55.690222  5222 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:55.696789  9394 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:55.698138  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:55.699697  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.700969  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.698307  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:55.683818  9416 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:55.701929  5222 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:55.702235  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:55.681753  9188 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:55.712469  9416 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:24:55.718890  5215 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:55.720616  5215 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:55.723228  9361 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:55.723526  9436 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:24:55.724498  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 87277.1us
W20260709 12:24:55.725116  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 74442.2us
I20260709 12:24:55.728735  9169 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:55.703315  9373 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:55.742194  5221 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
W20260709 12:24:55.732280  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 109072us
I20260709 12:24:55.733206  9432 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.735622  9269 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:55.764205  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:55.764573  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:55.724629  9436 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:55.753377  5190 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:55.758653  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:55.775161  5945 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:55.781890  5042 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:55.778512  9416 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:55.783118  9394 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:55.778182  9325 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:55.784232  9325 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:55.784674  9325 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:55.783562  9394 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.785553  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:55.788810  9169 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:55.789691  9394 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:55.790393  9269 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:55.791250  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.791855  9271 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:55.792429  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:55.789640  9373 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:55.793646  9377 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:55.793893  9373 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:55.795915  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.796491  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.791051  9269 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:55.780839  9341 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.513s	user 0.011s	sys 0.004s
I20260709 12:24:55.768478  9436 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:55.809988  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 51434.4us
I20260709 12:24:55.798245  9373 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:55.811805  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.813974  5222 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:24:55.812114  5215 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:55.812772  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:55.813197  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.815871  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:55.815987  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.817484  5899 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:55.761402  9215 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:55.819101  9440 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:55.819284  9215 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.820092  9440 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:55.820509  9440 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.820880  9440 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:55.821822  5224 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:24:55.821733  9284 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:24:55.816453  5148 raft_consensus.cc:2410] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:24:55.823817  9396 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.824357  9396 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:55.825752  9361 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:55.818534  5899 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:55.812548  9297 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:24:55.830940  9269 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:55.831534  9297 consensus_queue.cc:1243] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:24:55.807047  9188 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:24:55.792484  9271 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:55.817375  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.835126  9271 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:55.836437  9436 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:55.837882  9436 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:55.767757  9432 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:55.839594  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 129404us
I20260709 12:24:55.840119  9432 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:55.840476  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:55.841883  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:55.843137  9322 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:55.843218  5190 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:55.843924  9322 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:55.844360  9322 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:55.844759  9322 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:55.809780  9352 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260709 12:24:55.846146  5148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 809) took 3289 ms (client timeout 1505 ms). Trace:
W20260709 12:24:55.846662  5148 rpcz_store.cc:269] 0709 12:24:52.556966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.666650 (+109684us) service_pool.cc:224] Handling call
0709 12:24:53.180410 (+513760us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b580059f498. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:24:55.845864 (+2665454us) spinlock_profiling.cc:243] Waited 2.41 s on lock 0x7b5400582854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:55.846052 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5559308928}
I20260709 12:24:55.850980  9406 raft_consensus.cc:2764] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:24:55.856472  9392 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:55.851470  9356 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:55.857903  9352 raft_consensus.cc:3055] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:55.858363  9352 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:55.859301  9352 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.860737  9352 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.861755  9219 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.862375  9219 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:55.863153  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:55.863976  5221 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 10: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:55.864581  9219 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:55.865899  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 73623.4us
I20260709 12:24:55.865245  9273 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:55.867089  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:55.868415  9392 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.868849  9392 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:55.869518  9396 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:55.869240  9392 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.833310  9188 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:24:55.874027  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.874336  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 55882us
I20260709 12:24:55.884013  5899 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:55.838551  9298 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:55.869215  5115 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:55.878616  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:55.887120  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 43954.9us
I20260709 12:24:55.890458  5188 leader_election.cc:400] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16.
W20260709 12:24:55.892093  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 76544.2us
I20260709 12:24:55.895114  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.900068  9370 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:55.900202  9153 consensus_queue.cc:1059] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:55.901131  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:55.901384  9153 raft_consensus.cc:3055] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:55.870239  4895 catalog_manager.cc:5697] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:55.904182  4895 catalog_manager.cc:5697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:55.904752  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.895372  5074 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 7.
I20260709 12:24:55.901862  9153 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:55.916450  5114 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:55.918620  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:55.920343  5899 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:55.874917  9356 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:55.921603  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.867164  9273 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:55.922880  5899 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:55.928418  9439 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:55.929024  9439 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.929147  9271 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.930182  9271 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.932627  9416 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:55.928404  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:55.934932  9396 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:55.929482  9439 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:55.929018  9404 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.938423  9404 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.940760  9404 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:55.940973  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.942291  9273 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:55.881706  9188 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:55.946645  9188 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.947687  9188 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:55.947997  9271 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:55.950351  9373 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:55.950870  9373 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.951220  9373 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:55.953109  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.951037  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.954843  5069 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:55.964921  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.971383  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:55.973124  5114 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:55.975481  5070 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:55.972381  5071 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:24:55.975510  4895 catalog_manager.cc:5697] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 2 to 8, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 8 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:55.962967  9284 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:55.979496  9436 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.989421  5218 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:55.950356  9325 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:55.987418  9404 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:55.975471  9169 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:24:55.970623  9356 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:24:56.039616  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:56.015278  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 146106us
I20260709 12:24:56.023132  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:55.991946  9406 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:24:56.043996  9404 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.060454  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:56.010556  9355 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:24:56.013105  9284 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:24:56.068224  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.069154  5899 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:56.058080  9317 raft_consensus.cc:3055] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:56.058415  9365 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:56.052153  9474 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:56.073125  5899 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:56.063741  9404 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:56.076313  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 66980.6us
I20260709 12:24:56.085606  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:56.074311  9317 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:56.078068  9474 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.077298  9436 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.076069  9365 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:56.073686  9315 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:56.068605  9432 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
W20260709 12:24:56.126435  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 53359.9us
I20260709 12:24:56.073500  9273 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:24:56.130394  5041 leader_election.cc:400] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11.
I20260709 12:24:56.132545  5041 leader_election.cc:403] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:56.081348  9394 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:56.114964  9313 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:56.138429  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:56.138952  9477 log.cc:927] Time spent T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:24:56.118208  8752 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:56.113181  9298 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:56.163928  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:56.175837  9271 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:56.176236  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:56.161970  9317 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:56.177376  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:56.178175  8752 raft_consensus.cc:697] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:56.177810  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:56.179391  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:56.177598  9317 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:56.124285  9474 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:56.187479  5216 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:24:56.111255  9392 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:56.127552  9365 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:56.127233  9315 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:56.194686  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:56.198743  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:56.118412  9413 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.159s
I20260709 12:24:56.213577  5068 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:56.143078  9297 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.226617  5069 raft_consensus.cc:2368] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16.
W20260709 12:24:56.228705  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.231756  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 54023.6us
W20260709 12:24:56.191879  9467 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:24:56.142793  9356 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:56.239041  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.234488  9315 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:56.285966  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.141583  9122 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.209s
I20260709 12:24:56.253934  9396 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.272687  9269 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.303766  9269 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:56.306329  9480 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:24:56.307008  9392 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:56.307664  9269 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:56.249812  9341 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:56.311138  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 195650us
I20260709 12:24:56.311770  9341 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.312300  9341 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:56.312739  9419 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:24:56.310202  9479 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:24:56.272332  9483 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11.
I20260709 12:24:56.314448  9416 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:56.314482  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:56.316144  9484 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:24:56.309369  9481 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:56.262171  8752 consensus_queue.cc:237] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:56.317817  9475 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
W20260709 12:24:56.348767  9215 consensus_meta.cc:287] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.529s	user 0.012s	sys 0.000s
I20260709 12:24:56.333051  5947 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:24:56.173673  9284 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:56.255322  9297 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:56.352093  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.438405  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.292438  9325 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:56.270856  9356 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:56.321790  9341 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:56.348310  9467 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:56.340240  9313 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:56.318156  9439 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:56.432619  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 159314us
I20260709 12:24:56.460433  5115 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:56.322969  9396 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:56.363370  9122 raft_consensus.cc:3055] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:56.331894  9413 consensus_queue.cc:1059] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 12:24:56.433136  9269 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:56.433087  9197 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:56.431999  9394 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:56.291021  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 140111us
I20260709 12:24:56.495944  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:56.492142  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 178753us
I20260709 12:24:56.493901  5218 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
W20260709 12:24:56.529518  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 52582.2us
I20260709 12:24:56.532866  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:56.533385  5224 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:56.533705  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:56.534070  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:56.534332  5215 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:56.534997  5217 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:56.509789  9440 consensus_meta.cc:287] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.688s	user 0.013s	sys 0.000s
I20260709 12:24:56.492391  9493 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:56.490430  9122 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:24:56.540918  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 338645us
I20260709 12:24:56.499202  9197 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:56.472981  9416 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:56.506470  9392 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:56.521271  9341 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:56.552610  5188 leader_election.cc:400] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16.
I20260709 12:24:56.555869  5188 leader_election.cc:403] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:24:56.555922  5222 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:24:56.537629  9370 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:56.544224  9489 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:56.569741  9476 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:24:56.572714  5215 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:24:56.554562  9404 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:56.579797  9404 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:56.580536  9404 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:56.580929  9404 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.581266  9404 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:56.583953  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 88095.8us
I20260709 12:24:56.585340  5115 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:56.560022  9361 raft_consensus.cc:3055] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:24:56.601609  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.602365  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:56.548102  9440 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:56.560284  9496 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:56.556643  9416 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:56.727608  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 48687.4us
I20260709 12:24:56.609467  8752 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:56.557363  9392 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:56.682840  5900 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.505s (SENT)
I20260709 12:24:56.733830  5900 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:56.591698  9489 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:56.746527  9489 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:56.751230  9317 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:24:56.746623  9352 consensus_meta.cc:287] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.885s	user 0.013s	sys 0.000s
I20260709 12:24:56.603973  9361 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:56.752036  9317 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:56.778465  9492 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:56.780975  9317 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:56.786056  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 183818us
I20260709 12:24:56.788432  5148 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:56.789198  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:56.789639  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:56.789872  5149 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:56.790416  5142 raft_consensus.cc:2410] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:56.790979  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:56.791865  5144 raft_consensus.cc:2368] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13.
I20260709 12:24:56.791579  9315 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:56.795034  9315 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:56.551647  9493 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:56.570691  9370 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:56.730587  9322 consensus_meta.cc:287] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.885s	user 0.013s	sys 0.001s
I20260709 12:24:56.810338  9419 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:56.822608  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.823707  5898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.010s	sys 0.086s
W20260709 12:24:56.824421  5898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.010s	sys 0.086s
W20260709 12:24:56.825053  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 564498us
I20260709 12:24:56.829798  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:56.832044  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:56.833094  5948 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:56.775813  9188 consensus_meta.cc:287] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.820s	user 0.012s	sys 0.000s
I20260709 12:24:56.755290  9341 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:56.834852  5217 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
W20260709 12:24:56.864226  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 181428us
I20260709 12:24:56.859113  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:56.865231  5941 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:56.837805  9322 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:56.868319  9361 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:56.869591  9361 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:56.869693  9322 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:56.870666  9312 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:56.871127  9312 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:56.872141  9312 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:56.593612  9313 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:56.874106  5940 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:56.772785  9271 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:56.882691  5948 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
W20260709 12:24:56.862704  9373 consensus_meta.cc:287] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.911s	user 0.018s	sys 0.000s
I20260709 12:24:56.886607  5144 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:56.890642  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:56.890969  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:56.891665  5143 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:24:56.728850  9197 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:56.895826  5940 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:24:56.727474  9122 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:56.763607  9496 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:56.917135  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 364310us
I20260709 12:24:56.918133  9496 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:56.920146  9440 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:56.887997  9436 consensus_meta.cc:287] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.760s	user 0.010s	sys 0.000s
I20260709 12:24:56.899767  9370 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:56.900050  9396 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:56.900938  9491 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:24:56.901346  9439 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:56.970928  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.971553  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.908830  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:56.909785  9197 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:56.910604  9153 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:56.850075  9419 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:56.923254  5216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 810) took 1264 ms (client timeout 1531 ms). Trace:
I20260709 12:24:56.974339  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:56.961629  5939 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:56.962954  5941 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
W20260709 12:24:56.976331  5216 rpcz_store.cc:269] 0709 12:24:55.658918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.659068 (+   150us) service_pool.cc:224] Handling call
0709 12:24:56.922953 (+1263885us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54001437d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:56.923159 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785081088}
W20260709 12:24:56.970350  5220 consensus_meta.cc:287] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
I20260709 12:24:56.902814  9392 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:56.887693  9356 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:56.972213  5144 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:24:56.977794  5220 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
W20260709 12:24:56.904975  9365 consensus_meta.cc:287] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.654s	user 0.006s	sys 0.002s
W20260709 12:24:56.922631  9474 consensus_meta.cc:287] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.686s	user 0.011s	sys 0.000s
I20260709 12:24:56.980023  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:56.962798  9501 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.995352  5144 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:56.892239  9271 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:56.970379  9436 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:56.997821  5142 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:57.003326  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:56.894646  9373 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:56.973978  9499 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.012166  5144 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:56.995980  9361 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:57.009032  9501 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:57.030944  9269 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 12:24:57.031711  9269 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:56.994946  9474 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:57.035629  9404 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:57.037619  9322 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:57.038414  9322 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:57.040771  9322 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:57.026196  9153 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:57.012719  9499 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:56.995596  9365 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:57.044487  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:57.052130  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 499714us
I20260709 12:24:57.057300  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.064779  5147 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 14: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:57.043424  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.041501  5149 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:24:57.049961  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.072546  5146 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:57.048923  9499 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:57.073665  9499 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.073760  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.074199  9499 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:57.075307  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.051087  9413 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.076577  9413 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:57.078111  5148 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 12:24:57.078686  9413 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:57.053802  9504 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.314s	user 0.002s	sys 0.000s
I20260709 12:24:57.074610  5141 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:57.079228  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.080304  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.080513  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.080873  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.080991  5141 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:57.081871  9499 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:57.082140  5143 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:24:57.083334  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.083842  5143 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:57.042304  9416 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:57.086303  5941 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:57.053335  9365 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:57.047741  9495 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.318s	user 0.002s	sys 0.000s
I20260709 12:24:57.090054  5145 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:57.090756  5149 raft_consensus.cc:2368] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14.
I20260709 12:24:57.090919  9404 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:57.050763  9419 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:57.091732  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.092433  5149 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:57.050871  9312 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:57.085419  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.094260  5147 raft_consensus.cc:2393] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:57.052078  9373 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:57.096328  5147 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:24:57.053279  9271 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:57.096669  5147 pending_rounds.cc:85] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 5
I20260709 12:24:57.043490  9493 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:57.098178  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.081187  5148 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
W20260709 12:24:57.100443  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.104598  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:57.105302  5069 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:57.071907  9361 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:57.106362  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:57.107554  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:57.107719  5900 leader_election.cc:336] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.942s (SENT)
I20260709 12:24:57.092661  5145 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:57.122617  5219 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:57.096961  5147 pending_rounds.cc:107] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:24:57.124900  5147 raft_consensus.cc:2889] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:57.128355  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.054255  9298 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16.
I20260709 12:24:57.132594  5143 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:57.132905  5141 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:57.134883  5145 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:57.135188  5147 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:57.138203  9436 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:57.138811  9501 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:57.139519  9436 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:57.139940  9436 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:57.140848  9436 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:57.143131  9361 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:57.143695  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 816) took 1450 ms. Trace:
I20260709 12:24:57.146940  5149 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:24:57.155467  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.159369  5149 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:24:57.160707  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.160893  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.162695  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.163224  9493 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:57.164119  5141 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:57.166016  9365 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:57.154361  5188 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:57.153723  5220 rpcz_store.cc:276] 0709 12:24:55.693604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.697657 (+  4053us) service_pool.cc:224] Handling call
0709 12:24:57.143409 (+1445752us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54002b39d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:57.143570 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684504448}
I20260709 12:24:57.166760  9365 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:57.178221  5149 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:57.170975  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.179553  5146 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:57.182040  9436 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:57.186776  5145 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: [13.9-14.10]
I20260709 12:24:57.195083  5150 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:24:57.196252  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.197041  5147 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:57.168972  5946 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
W20260709 12:24:57.199128  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 44877.2us
I20260709 12:24:57.200311  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:57.202085  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.202064  9298 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.203756  9298 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:57.204612  9503 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.298s	user 0.002s	sys 0.000s
I20260709 12:24:57.206333  9298 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:57.207445  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1089) took 1039 ms. Trace:
I20260709 12:24:57.208012  5944 rpcz_store.cc:276] 0709 12:24:56.168410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.168556 (+   146us) service_pool.cc:224] Handling call
0709 12:24:56.210818 (+ 42262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:56.438210 (+227392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.438241 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:56.486525 (+ 48284us) log.cc:844] Serialized 26 byte log entry
0709 12:24:56.602986 (+116461us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:56.603065 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.603174 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:57.203943 (+600769us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:24:57.207051 (+  3108us) raft_consensus.cc:1685] finished
0709 12:24:57.207084 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.207367 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100389,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":40349440,"thread_start_us":251403,"threads_started":2,"wal-append.queue_time_us":355153}
I20260709 12:24:57.224103  5149 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
W20260709 12:24:57.228364  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.242604  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.247159  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.248442  5149 raft_consensus.cc:2368] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8.
W20260709 12:24:57.265764  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.222918  9501 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.222604  9188 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:57.234632  9313 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:57.235126  9439 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:57.298846  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.303789  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.304163  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.305351  5149 raft_consensus.cc:2368] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13.
I20260709 12:24:57.309900  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.310644  5223 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:24:57.297314  9370 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.316530  9370 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:57.324725  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.333339  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.333930  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.334084  5947 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:24:57.334523  5941 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:24:57.343389  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.344410  5220 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:24:57.345230  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:57.343391  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.350489  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.351254  5223 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:57.351346  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.351963  5220 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:24:57.279318  9501 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:57.350502  5941 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:57.350557  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:57.329774  9370 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:57.298697  9439 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:57.284755  9313 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:57.364609  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.366307  5141 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:57.342032  9341 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:57.330307  9153 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:57.366887  5149 raft_consensus.cc:2368] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14.
I20260709 12:24:57.367449  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.352480  9483 consensus_queue.cc:1059] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:24:57.349876  9197 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:57.386935  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.387575  5149 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:57.361825  9416 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:57.378798  5041 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:57.378674  5220 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 10: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:57.390182  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.370398  9355 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:24:57.411424  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.414983  5146 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:57.418167  5147 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:57.416960  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:57.464500  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 47655.3us
I20260709 12:24:57.419301  9416 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:57.470371  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.386272  9153 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:57.423756  9419 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:57.423969  9355 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:57.427170  9501 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:57.393695  9197 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:57.433246  5076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:24:57.481035  5076 raft_consensus.cc:606] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:57.437323  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:57.442183  9516 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:24:57.445864  5900 leader_election.cc:336] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.272s (SENT)
I20260709 12:24:57.483084  5900 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:57.426024  5149 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:57.485164  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.452597  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.486399  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.453924  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.454758  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.455595  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.457868  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:57.498478  5947 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:57.458905  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.414243  9298 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.460466  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.460800  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.512421  5941 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 6.
I20260709 12:24:57.513633  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.461565  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.472466  9413 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:57.474818  9419 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:57.462189  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 333016us
I20260709 12:24:57.465379  5041 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:57.420241  5145 raft_consensus.cc:1275] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:57.465649  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.466955  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:57.518299  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 53009.8us
I20260709 12:24:57.518790  5217 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
W20260709 12:24:57.467525  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 248043us
W20260709 12:24:57.470614  5224 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.025s	sys 0.028s
I20260709 12:24:57.471570  5147 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:57.472921  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 910) took 1663 ms. Trace:
W20260709 12:24:57.520002  5224 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.025s	sys 0.029s
I20260709 12:24:57.499265  5948 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:57.423146  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 172864us
I20260709 12:24:57.469456  9365 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:57.458993  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.515232  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:57.522837  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:57.509819  5947 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:24:57.514680  5942 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
W20260709 12:24:57.521670  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.519040  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.519090  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:57.555620  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.531410  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.520390  5068 rpcz_store.cc:276] 0709 12:24:55.804298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.804420 (+   122us) service_pool.cc:224] Handling call
0709 12:24:55.814135 (+  9715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.227509 (+413374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.466970 (+1239461us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400327654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:57.467169 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":886894080}
I20260709 12:24:57.558943  5149 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:57.498706  9322 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.564572  5222 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:57.567039  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.570449  5146 raft_consensus.cc:2376] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13. Re-sending same reply.
I20260709 12:24:57.571451  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.572392  5146 raft_consensus.cc:2368] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8.
I20260709 12:24:57.567438  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.558346  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:57.571333  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.574023  5147 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:57.567301  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.575070  5149 raft_consensus.cc:2376] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11. Re-sending same reply.
W20260709 12:24:57.575408  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.578997  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.579151  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:57.579947  5941 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:57.580197  5146 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:24:57.580935  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:57.581955  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 829) took 1450 ms. Trace:
I20260709 12:24:57.582412  5218 rpcz_store.cc:276] 0709 12:24:56.132126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.132345 (+   219us) service_pool.cc:224] Handling call
0709 12:24:57.581764 (+1449419us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b5400852354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:57.581893 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":750156544}
I20260709 12:24:57.586895  9322 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:57.587968  9122 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:57.589694  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.590764  9416 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:57.591459  9526 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.591979  9526 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:57.592723  9520 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:24:57.593672  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:57.594113  9313 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:57.594277  5940 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.528931  9312 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:57.509004  9355 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:57.600358  5941 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:57.602161  5143 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:57.581820  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:57.579249  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 133441us
I20260709 12:24:57.596762  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:57.608811  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 50565.7us
W20260709 12:24:57.611831  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 50584.8us
I20260709 12:24:57.517383  9413 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.617909  5948 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:57.621526  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:57.557252  9419 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:57.622870  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.626446  5190 leader_election.cc:400] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14.
I20260709 12:24:57.627004  5190 leader_election.cc:403] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:57.628620  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:57.518169  9298 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:57.641939  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:57.653079  5068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:24:57.654691  5068 raft_consensus.cc:606] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:57.655129  5068 raft_consensus.cc:614] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:57.625622  9413 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:57.662693  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:57.629199  9312 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:57.677335  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.668843  9499 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:57.684754  9499 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:57.687130  9499 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:57.687435  9169 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:24:57.688907  9529 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:24:57.692684  9365 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:57.649312  9153 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:57.705857  5068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:24:57.667480  9297 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:57.714246  5068 raft_consensus.cc:606] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:57.716698  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.718045  5941 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:24:57.699615  9169 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:24:57.695510  9499 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:57.718322  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:57.705240  9153 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:57.681604  9313 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:24:57.696048  9312 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:57.720060  9499 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:57.745743  5946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 1090) took 1576 ms (client timeout 1536 ms). Trace:
W20260709 12:24:57.746934  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:57.747476  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.748636  5944 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 12:24:57.746188  5946 rpcz_store.cc:269] 0709 12:24:56.168819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.175926 (+  7107us) service_pool.cc:224] Handling call
0709 12:24:57.745463 (+1569537us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b5400005354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:57.745642 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1926235904}
I20260709 12:24:57.743034  9474 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:57.769088  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:57.772897  5216 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:57.770782  5115 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:57.748636  9499 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:57.747121  9197 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:57.750595  9297 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:24:57.783592  9439 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:57.785023  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.786278  5073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:24:57.787032  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1397) took 1238 ms. Trace:
I20260709 12:24:57.787459  5215 rpcz_store.cc:276] 0709 12:24:56.549317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.583070 (+ 33753us) service_pool.cc:224] Handling call
0709 12:24:56.583513 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:56.758894 (+175381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.758939 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:56.774857 (+ 15918us) log.cc:844] Serialized 26 byte log entry
0709 12:24:56.974803 (+199946us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:56.974836 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.786588 (+811752us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54006f77d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:57.786649 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:57.786702 (+    53us) raft_consensus.cc:1685] finished
0709 12:24:57.786720 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.786965 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13061,"raft.queue_time_us":14877,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":6748,"spinlock_wait_cycles":371707136,"thread_start_us":109732,"threads_started":1,"wal-append.queue_time_us":179568}
I20260709 12:24:57.791411  5216 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:24:57.774080  9271 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:57.776924  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:57.777218  9521 log.cc:927] Time spent T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:57.798569  9298 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:57.777575  9532 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:57.778959  9312 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:57.779827  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:57.760727  9356 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:57.802114  9197 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.744258  9153 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:57.802805  9436 raft_consensus.cc:3055] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:57.797195  9499 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:57.801565  9532 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.803411  9436 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:57.803718  9532 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:57.804685  9153 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:57.804519  9436 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:57.804968  9526 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:57.796746  9322 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:57.802735  9197 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:57.812541  5940 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
W20260709 12:24:57.837965  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 62753.5us
I20260709 12:24:57.839159  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:57.852100  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:57.853889  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:57.854457  5222 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:57.857400  5899 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:57.857839  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:57.858506  5216 raft_consensus.cc:2376] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9. Re-sending same reply.
I20260709 12:24:57.866398  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:57.868336  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:57.902334  5218 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:24:57.902423  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:57.902561  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:57.902046  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:57.907704  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:57.911775  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 72745.2us
I20260709 12:24:57.916371  4895 catalog_manager.cc:5697] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 7 to 10, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 10 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:57.906476  5067 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
W20260709 12:24:57.921682  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 43928.8us
I20260709 12:24:57.920524  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.928057  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1392) took 1389 ms. Trace:
I20260709 12:24:57.875924  9485 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.930932  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:57.879065  9122 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:57.923597  5899 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:57.931838  5068 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:57.932092  5223 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:57.850759  9436 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:57.875700  9322 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.853667  9501 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:57.933961  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:57.931030  5224 rpcz_store.cc:276] 0709 12:24:56.538456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.538619 (+   163us) service_pool.cc:224] Handling call
0709 12:24:56.551499 (+ 12880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.551729 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:56.847026 (+295297us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.878755 (+1031729us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.927727 (+ 48972us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:57.927778 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:57.927827 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.927889 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.928003 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91040,"prepare.queue_time_us":430366,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2891,"raft.queue_time_us":117452,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"thread_start_us":1222662,"threads_started":3,"wal-append.queue_time_us":621338}
I20260709 12:24:57.949882  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.952491  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:24:57.925268  9482 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.626s	user 0.012s	sys 0.014s
I20260709 12:24:57.954926  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:57.990583  5218 raft_consensus.cc:2376] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10. Re-sending same reply.
I20260709 12:24:57.950982  4895 catalog_manager.cc:5697] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 7 to 8, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 8 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:57.954869  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 328624us
I20260709 12:24:57.949932  9271 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:24:57.950419  9122 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.992609  9122 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:57.992549  4895 catalog_manager.cc:5697] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 7 to 10, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 10 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:24:57.992949  5190 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:57.950687  9485 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:57.995096  4895 catalog_manager.cc:5697] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 7 to 8, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 8 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:24:57.976972  5215 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:24:57.960494  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:24:57.994422  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.913866  9153 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:57.998423  9439 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.998875  9153 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:57.998970  9439 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.001066  9153 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:58.001145  9439 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:58.002272  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:58.003859  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 45551.9us
W20260709 12:24:58.005774  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.005904  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:58.005415  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:24:58.007746  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 47494.9us
I20260709 12:24:57.930414  9489 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.997357  4895 catalog_manager.cc:5697] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 5 to 6, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 6 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:57.994747  9482 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.695s	user 0.016s	sys 0.019s
W20260709 12:24:57.951339  9483 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.011s	sys 0.007s
W20260709 12:24:58.030717  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 511658us
I20260709 12:24:57.997798  9485 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:58.017856  4895 catalog_manager.cc:5697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:58.032372  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:57.955780  5217 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:24:58.024559  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:58.031268  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 389421us
I20260709 12:24:57.935305  9219 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:58.031962  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:58.037138  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.037185  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:58.008407  9525 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:24:58.050034  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:58.003645  9312 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:58.054095  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:58.053287  4895 catalog_manager.cc:5697] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:24:58.066295  5224 raft_consensus.cc:2410] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:58.014628  9215 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:24:58.068639  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.069085  5939 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
W20260709 12:24:58.043035  9483 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.011s	sys 0.007s
W20260709 12:24:58.090679  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 58412us
I20260709 12:24:58.089375  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:58.089711  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:58.089094  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:58.089238  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 53503.1us
I20260709 12:24:58.090370  5939 pending_rounds.cc:85] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 9
I20260709 12:24:58.032994  9540 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:58.095932  9542 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:58.034369  9489 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.096720  9474 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:58.097007  9356 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:58.096571  4895 catalog_manager.cc:5697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 5 to 9, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 9 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:24:58.101568  5939 pending_rounds.cc:107] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:24:58.100227  5944 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:58.131768  5939 raft_consensus.cc:2889] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:58.076320  9215 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.108078  5190 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:58.125970  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:58.142179  5068 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 15: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:57.965051  9322 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:58.134042  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.146075  5071 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:58.149979  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48258.9us
I20260709 12:24:58.096722  9493 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:58.103142  9153 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.103899  9540 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:58.172078  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:58.116108  9312 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:58.111616  9319 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:58.095957  9419 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:58.221247  4898 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43222 (request call id 143) took 23388 ms (client timeout 15000 ms). Trace:
I20260709 12:24:58.153640  9215 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:58.231033  9219 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:58.237294  5223 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:58.185966  9197 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:58.237203  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 129210us
I20260709 12:24:58.249218  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:58.237399  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 60992.3us
I20260709 12:24:58.125260  9352 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 12:24:58.250607  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:58.210916  9540 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:58.192013  9517 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:24:58.260597  9551 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:58.263062  9439 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.266815  9352 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:58.267262  9197 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:58.267990  9312 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:58.249826  9219 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:58.259925  9544 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:58.195648  9483 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:24:58.279160  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:58.243929  9552 raft_consensus.cc:2804] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:58.153476  9356 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.275492  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:58.281365  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.283907  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:58.284803  5076 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 16: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:58.284042  5189 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:58.242604  9485 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.209707  9153 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:58.210533  9493 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:24:58.292533  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 358597us
W20260709 12:24:58.295800  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 280097us
I20260709 12:24:58.314570  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:58.314994  5900 leader_election.cc:336] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.531s (SENT)
W20260709 12:24:58.298856  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 196011us
I20260709 12:24:58.133754  9542 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.304342  5899 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:58.270483  9548 raft_consensus.cc:993] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: : Instructing follower 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:24:58.294019  9499 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:58.324052  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:58.331106  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:58.340078  5115 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:58.332834  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:58.308502  9365 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:58.357410  5068 raft_consensus.cc:2410] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:24:58.295305  9439 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:58.341300  9534 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.325s	user 0.002s	sys 0.000s
I20260709 12:24:58.364702  5073 raft_consensus.cc:606] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:58.358251  4887 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.672s	user 0.005s	sys 0.012s
I20260709 12:24:58.335929  9355 raft_consensus.cc:1081] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Signalling peer 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:24:58.333914  9474 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:58.344583  9532 consensus_meta.cc:287] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.540s	user 0.017s	sys 0.006s
I20260709 12:24:58.334817  9555 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:58.347280  9169 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:24:58.390705  9547 log.cc:927] Time spent T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:24:58.391911  9545 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:24:58.327325  9485 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.365384  9271 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:24:58.397109  5222 consensus_meta.cc:287] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.522s	user 0.014s	sys 0.007s
I20260709 12:24:58.437808  5222 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
W20260709 12:24:58.398841  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 141536us
I20260709 12:24:58.400183  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:58.439337  5069 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:24:58.390528  9419 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:58.400766  9413 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.717s	user 0.011s	sys 0.017s
I20260709 12:24:58.411962  9489 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:58.235544  4898 rpcz_store.cc:269] 0709 12:24:34.619216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.619520 (+   304us) service_pool.cc:224] Handling call
0709 12:24:34.857592 (+238072us) spinlock_profiling.cc:243] Waited 8.2 ms on lock 0x7b54006a5754. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a5090 00007fd8a8543f3a 00007fd8ae2f9e9c 00007fd8ae3f804e 00007fd8ac8b92a4 00007fd8ac8b9232 00007fd8ac8b9161 00007fd8ac8b90dc 00007fd8ac8b8392 00007fd8a687648c 00007fd8a68758c6
0709 12:24:57.961739 (+23104147us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:49.127800 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:49.210122 (+ 82322us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:49.225805 (+ 15683us) write_op.cc:435] Acquired schema lock
0709 12:24:49.228652 (+  2847us) tablet.cc:662] Decoding operations
0709 12:24:49.290201 (+ 61549us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:49.290332 (+   131us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:49.290360 (+    28us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:24:49.372564 (+ 82204us) tablet.cc:701] Row locks acquired
0709 12:24:49.372936 (+   372us) write_op.cc:260] PREPARE: finished
0709 12:24:49.373128 (+   192us) write_op.cc:270] Start()
0709 12:24:49.373296 (+   168us) write_op.cc:276] Timestamp: P: 1783599889373047 usec, L: 0
0709 12:24:49.373339 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 12:24:49.382297 (+  8958us) log.cc:844] Serialized 8234 byte log entry
0709 12:24:50.318077 (+935780us) op_driver.cc:464] REPLICATION: finished
0709 12:24:51.513826 (+1195749us) write_op.cc:301] APPLY: starting
0709 12:24:51.529771 (+ 15945us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:51.765773 (+236002us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:51.765949 (+   176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:52.213999 (+448050us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:52.214407 (+   408us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=42
0709 12:24:52.214476 (+    69us) write_op.cc:312] APPLY: finished
0709 12:24:52.215012 (+   536us) log.cc:844] Serialized 144 byte log entry
0709 12:24:52.402516 (+187504us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:52.528100 (+125584us) write_op.cc:454] Released schema lock
0709 12:24:52.528293 (+   193us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:52.718175 (+189882us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:52.725156 (+  6981us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1749750,"reported_tablets":42,"spinlock_wait_cycles":18861824,"child_traces":[["op",{"apply.queue_time_us":1157813,"apply.run_cpu_time_us":31439,"apply.run_wall_time_us":1076066,"num_ops":21,"prepare.queue_time_us":342367,"prepare.run_cpu_time_us":43550,"prepare.run_wall_time_us":574445,"raft.queue_time_us":365956,"raft.run_cpu_time_us":171136,"raft.run_wall_time_us":735000,"replication_time_us":944513,"thread_start_us":1713195,"threads_started":5,"wal-append.queue_time_us":1017110,"wal-append.run_cpu_time_us":31689,"wal-append.run_wall_time_us":2477016}]]}
I20260709 12:24:58.448899  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:58.452598  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 177217us
W20260709 12:24:58.453383  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 169405us
I20260709 12:24:58.477816  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:58.478693  5074 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:58.480647  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 140673us
I20260709 12:24:58.442000  9555 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:58.484051  5115 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:58.414332  9532 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.411222  9474 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:58.484990  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 152184us
I20260709 12:24:58.498406  9493 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:58.499830  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:58.515784  5075 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.502348  9356 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:58.474066  9297 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.693s	user 0.017s	sys 0.006s
I20260709 12:24:58.494015  9365 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:58.450479  9322 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:58.525833  5071 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:58.444280  9413 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:58.515687  9122 consensus_meta.cc:287] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.523s	user 0.006s	sys 0.006s
W20260709 12:24:58.535916  9537 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:24:58.537623  9485 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:58.539147  9542 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:58.497318  9563 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 12:24:58.534090  9559 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:24:58.543257  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:58.535072  9558 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:24:58.511127  9284 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:58.542340  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:58.555203  5070 raft_consensus.cc:2368] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17.
I20260709 12:24:58.516252  9551 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:58.515198  9544 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:58.515050  9271 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:58.515446  9555 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:58.584460  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:58.541254  9297 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14.
I20260709 12:24:58.590078  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:58.590787  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.592267  5072 raft_consensus.cc:2368] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16.
W20260709 12:24:58.665570  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.642966  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 80387.4us
I20260709 12:24:58.558924  9122 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:58.670061  5942 raft_consensus.cc:2393] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:58.554628  9568 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:58.671209  5944 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:58.642488  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 81232.5us
I20260709 12:24:58.648669  5071 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:58.622293  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:58.558442  9572 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:24:58.584436  9169 raft_consensus.cc:697] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:58.654186  9574 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.653558  5075 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 8.
W20260709 12:24:58.654677  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 93812.6us
I20260709 12:24:58.643162  9153 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:58.674000  5946 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:24:58.654438  9551 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.685063  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:58.689373  5941 raft_consensus.cc:2393] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:58.690860  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:58.675522  9567 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:58.680791  9568 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:58.685911  9566 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:58.690619  9551 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:58.663318  9365 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.709884  9439 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:58.703306  9567 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:58.697093  9576 consensus_queue.cc:1059] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:24:58.711124  9563 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:58.741984  9563 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.742344  9549 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:58.712574  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 837) took 1278 ms. Trace:
I20260709 12:24:58.743062  9549 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.688771  9574 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:58.669806  9570 consensus_queue.cc:1059] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:24:58.745816  9413 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:58.745996  9574 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:58.746773  9574 raft_consensus.cc:3055] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:58.747198  9574 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:58.714665  5221 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:24:58.712430  9532 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:58.721524  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 212887us
I20260709 12:24:58.681104  9572 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.727068  9501 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:58.727751  9361 raft_consensus.cc:3055] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:58.730847  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 1473) took 1084 ms. Trace:
I20260709 12:24:58.730515  9215 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.731622  9474 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:58.732496  9568 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.755266  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:58.734505  9548 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.708384  9271 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:58.736783  9482 raft_consensus.cc:3055] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:58.736435  9544 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:24:58.738662  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 282106us
I20260709 12:24:58.739379  9169 consensus_queue.cc:237] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:58.740331  9560 log.cc:927] Time spent T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:24:58.740940  9122 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:58.735240  4887 maintenance_manager.cc:643] P ce6fc7638c8549faa2535fb5774c32c1: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 7.300s	user 0.217s	sys 0.109s Metrics: {"bytes_written":30239,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":396917,"dirs.run_cpu_time_us":5492,"dirs.run_wall_time_us":22021,"drs_written":1,"lbm_read_time_us":91605,"lbm_reads_1-10_ms":2,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":1,"lbm_write_time_us":210396,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":6,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":362836,"threads_started":2,"update_count":21,"wal-append.queue_time_us":621859}
I20260709 12:24:58.741647  9566 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.742523  9563 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:58.711827  9557 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.717844  5069 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
W20260709 12:24:58.750376  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 72923us
I20260709 12:24:58.745986  9570 raft_consensus.cc:3055] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:58.748504  9567 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:58.743125  5149 rpcz_store.cc:276] 0709 12:24:57.434277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.581108 (+146831us) service_pool.cc:224] Handling call
0709 12:24:57.581575 (+   467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:57.582395 (+   820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.582434 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:57.583165 (+   731us) log.cc:844] Serialized 52 byte log entry
0709 12:24:57.842572 (+259407us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:57.883016 (+ 40444us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.920165 (+ 37149us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.920490 (+   325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.920616 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.096162 (+175546us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b1800016300. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:24:58.712310 (+616148us) raft_consensus.cc:1685] finished
0709 12:24:58.712334 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.712500 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32680,"prepare.queue_time_us":195527,"prepare.run_cpu_time_us":138,"prepare.run_wall_time_us":4645,"raft.queue_time_us":20866,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":18661,"spinlock_wait_cycles":14739840,"thread_start_us":257830,"threads_started":1,"wal-append.queue_time_us":717167}
I20260709 12:24:58.754772  5076 rpcz_store.cc:276] 0709 12:24:57.646748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.646983 (+   235us) service_pool.cc:224] Handling call
0709 12:24:58.730578 (+1083595us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b540058af54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:58.730751 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407515392}
I20260709 12:24:58.748157  9574 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.733677  9555 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:58.759279  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:58.772758  5074 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:24:58.745358  9549 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:58.803725  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.754557  9361 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:58.753901  9501 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:58.819967  5223 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:58.821178  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:58.810053  9436 consensus_meta.cc:287] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.853s	user 0.010s	sys 0.012s
I20260709 12:24:58.760833  9482 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:58.854100  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:58.855177  5218 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:24:58.833578  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 327255us
I20260709 12:24:58.784967  9474 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:58.844492  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 85337.2us
I20260709 12:24:58.784651  9271 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:58.819895  9517 consensus_queue.cc:1059] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:24:58.842717  9570 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:58.851181  9585 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:58.873920  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:58.863021  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:58.876355  5215 raft_consensus.cc:2410] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:24:58.878288  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:58.879496  5898 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:58.880787  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:58.881791  5149 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:58.882056  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:58.882691  5074 raft_consensus.cc:2376] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13. Re-sending same reply.
I20260709 12:24:58.885919  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56034 (request call id 3) took 1091 ms. Trace:
I20260709 12:24:58.886435  5073 rpcz_store.cc:276] 0709 12:24:57.785944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.786095 (+   151us) service_pool.cc:224] Handling call
0709 12:24:58.877589 (+1091494us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54004493d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a8535853 00007fd8ad50360a 00007fd8a6a897d4 00007fd8a6a89762 00007fd8a6a89691 00007fd8a6a8960c 00007fd8a6a888c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:58.877705 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1328462592}
W20260709 12:24:58.888103  9541 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.793s	user 0.001s	sys 0.003s
I20260709 12:24:58.865813  9570 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:58.872732  9416 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:58.890793  9570 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:58.865556  9585 raft_consensus.cc:697] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:58.892882  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:58.893950  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:58.894619  5219 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:58.896117  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:58.896775  5220 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:58.897944  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:58.898123  9284 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:58.898726  9284 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:58.853662  9574 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:58.900729  5900 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.747s (SENT)
I20260709 12:24:58.901286  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:58.830641  9557 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:58.904291  9557 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:58.844591  9532 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:24:58.906567  5223 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:58.846005  9153 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:24:58.910802  5076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c86395378750473e80c76902ad29fbe7"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:24:58.911664  5076 raft_consensus.cc:606] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:58.912036  5076 raft_consensus.cc:614] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:58.862033  9571 consensus_queue.cc:1059] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:24:58.915563  9215 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:58.916512  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:58.917092  9572 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:58.917315  9219 consensus_meta.cc:287] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.591s	user 0.006s	sys 0.006s
I20260709 12:24:58.917963  9582 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:58.918067  9219 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:58.919402  9582 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:58.921104  9219 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:24:58.921552  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.923187  9122 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:58.925417  9474 raft_consensus.cc:697] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:58.926142  9566 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:58.927265  9271 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:58.874352  9569 consensus_queue.cc:1059] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:24:58.961014  5215 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:24:58.927479  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:58.890794  9416 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:58.970611  9416 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:58.971836  9416 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:58.937933  9557 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.985471  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:58.985971  9557 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:58.986618  5076 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:58.985724  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:58.988469  9557 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:58.991554  9474 consensus_queue.cc:237] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:58.950696  9312 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.993927  9312 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:58.996400  9312 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:58.965459  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:58.892483  9585 consensus_queue.cc:237] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:58.929751  9582 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:58.938284  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:58.940657  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:58.940832  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:58.931605  9153 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:58.950687  5223 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:24:58.935043  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:58.957397  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:58.958268  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:59.008289  5219 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:24:58.959401  5220 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:24:58.963141  5218 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:59.010183  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:58.965277  9361 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:24:59.004418  9540 consensus_meta.cc:287] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.712s	user 0.014s	sys 0.000s
I20260709 12:24:59.020435  5941 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:24:59.022637  5946 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:24:59.030992  5216 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:59.031330  5218 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:24:58.956112  9551 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.039227  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:59.048266  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:59.049217  5944 raft_consensus.cc:2410] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:24:59.052827  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 925) took 1003 ms. Trace:
I20260709 12:24:59.053257  5070 rpcz_store.cc:276] 0709 12:24:58.049576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.049763 (+   187us) service_pool.cc:224] Handling call
0709 12:24:59.052637 (+1002874us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54000bf4d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:59.052767 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135031680}
I20260709 12:24:59.054927  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.055616  5072 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:59.055675  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.056274  5150 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:24:59.056808  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.057446  5067 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:59.045262  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.042816  9570 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.062598  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.043107  9312 consensus_queue.cc:1048] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:24:59.075351  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.077600  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:59.079873  5115 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.035729  9361 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.083227  5217 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:59.085417  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.087354  5216 raft_consensus.cc:2410] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:24:59.087538  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.088313  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.085254  9526 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:59.090649  9551 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:59.085615  9322 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:59.099997  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.085798  9557 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:59.083763  9188 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.093174  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.056939  9284 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:59.094528  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.018426  9501 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:59.108330  5142 raft_consensus.cc:2368] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13.
I20260709 12:24:59.026047  9540 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.096069  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.096004  9312 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:59.098007  5944 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:39870
I20260709 12:24:59.123922  5944 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:59.098030  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.085418  9570 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.098891  9526 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.034840  9563 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.123296  5942 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:59.124300  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:59.107005  9501 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:59.102180  9322 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.135893  9322 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:59.126135  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.138022  5218 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:59.101912  9271 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:59.129590  9526 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.133885  5188 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.146575  5218 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:59.131937  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:59.126768  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:59.127667  5150 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:24:59.103106  9574 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:59.129807  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.124348  5944 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:59.130293  5067 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:59.132097  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.150238  5142 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 8.
I20260709 12:24:59.105494  9284 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:59.135284  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 96133.5us
I20260709 12:24:59.151201  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:24:59.154436  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.154973  5900 leader_election.cc:400] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8.
I20260709 12:24:59.155372  5900 leader_election.cc:403] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:59.158136  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.165354  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.166324  5150 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:24:59.167387  5900 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.171725  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.172518  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:59.174705  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:59.172699  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.176168  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.176856  5150 raft_consensus.cc:2376] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15. Re-sending same reply.
I20260709 12:24:59.177250  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:59.178120  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.107141  9188 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.175537  5142 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:59.178833  5150 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:59.193141  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:59.183317  9570 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:59.183509  9540 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:59.130698  9557 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:59.197827  5944 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:59.154811  9271 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:59.196950  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:59.200448  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.162657  9215 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.203828  5939 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:24:59.172420  9576 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 7. Current term is 8.
I20260709 12:24:59.180600  9555 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:59.130107  9312 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.179348  9583 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:59.187821  9169 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:59.219380  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.214354  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.221868  5150 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:59.219806  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.223048  5142 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:59.214120  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:59.224330  5944 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:59.195039  9574 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:59.199294  9585 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:59.202687  9436 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:59.229653  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:59.215027  9570 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:59.232784  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:59.221546  9169 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:59.235069  5939 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:24:59.235765  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.228132  9540 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:59.240983  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.241820  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.240381  5144 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:59.243947  5150 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 15: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:59.228704  9557 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:59.244515  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.244793  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.245518  5144 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:59.243285  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.247669  5114 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.247778  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:59.249250  5114 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.251035  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.252950  9555 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:59.254820  9188 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:59.255575  9579 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:59.251256  5114 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.256974  9579 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:59.238061  9319 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:59.240692  9169 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:59.257812  9319 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:59.258270  9319 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:59.258822  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.259862  9555 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:59.239542  9570 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:59.262471  9570 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.263137  9555 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.263659  9555 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.242954  9542 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:59.262629  9474 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:59.266497  9542 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.267349  9542 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:59.268508  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:59.257474  9579 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:24:59.277318  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.278074  5221 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:24:59.278892  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:24:59.282842  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.243573  9540 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.284880  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.286917  5221 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:59.286320  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:59.288794  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:59.289342  5221 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:59.294696  5149 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 15: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:59.297925  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.299294  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.300168  5150 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:24:59.302021  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.269713  9585 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.279501  9474 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:24:59.310212  9598 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:24:59.312338  9169 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:59.315531  5946 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:59.330045  9563 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:59.333036  9169 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:59.334326  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.334894  5221 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:24:59.335963  5221 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:59.336475  5943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 1113) took 1268 ms (client timeout 1546 ms). Trace:
I20260709 12:24:59.339293  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:59.340199  9548 consensus_meta.cc:287] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.007s
I20260709 12:24:59.340570  5145 raft_consensus.cc:2410] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:24:59.340713  9548 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:59.341789  5073 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:24:59.302451  9574 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:24:59.346789  9553 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 1.111s	user 0.003s	sys 0.001s
I20260709 12:24:59.305891  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:59.336920  5943 rpcz_store.cc:269] 0709 12:24:58.068438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.089454 (+ 21016us) service_pool.cc:224] Handling call
0709 12:24:59.336280 (+1246826us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b5400240c54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:59.336414 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1667881856}
I20260709 12:24:59.349870  9579 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:59.352773  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.354048  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.294397  9540 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.354768  5148 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:24:59.355933  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.356565  5148 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:24:59.354808  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.357434  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.358353  5145 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:59.363022  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.363783  5070 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:24:59.364956  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.365927  5071 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:24:59.368222  5148 raft_consensus.cc:2393] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:59.370415  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.354632  9312 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:59.373126  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:59.379467  5042 leader_election.cc:400] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13.
I20260709 12:24:59.379343  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.379853  5042 leader_election.cc:403] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:59.380396  5075 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:24:59.322536  5072 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:59.386217  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:24:59.389559  9552 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.025s	user 0.010s	sys 0.002s
I20260709 12:24:59.390003  9605 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:59.390053  9552 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.010s	sys 0.002s
I20260709 12:24:59.392653  9552 raft_consensus.cc:697] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:59.394376  9552 consensus_queue.cc:237] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.391692  9605 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:59.388712  5188 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:59.397413  5943 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:59.398777  5042 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:59.399959  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.386873  9569 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.411265  5145 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:24:59.358078  9585 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:59.366605  9474 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:59.431959  5149 raft_consensus.cc:2393] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 10: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:59.376360  9563 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.436198  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.437785  5147 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:24:59.439133  5145 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:24:59.440117  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.440701  5145 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:24:59.441583  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.433703  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.443084  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:59.437271  5076 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:59.336112  9526 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.339386  9322 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.449308  5042 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:59.451591  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:24:59.446062  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 57441.5us
I20260709 12:24:59.455519  5188 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:59.457520  5150 raft_consensus.cc:2376] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15. Re-sending same reply.
I20260709 12:24:59.458432  5042 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:59.431237  9361 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13.
I20260709 12:24:59.456959  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.436867  9153 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:59.461757  5114 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:59.467607  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 1148) took 1271 ms. Trace:
I20260709 12:24:59.470737  5068 rpcz_store.cc:276] 0709 12:24:58.196060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.250146 (+ 54086us) service_pool.cc:224] Handling call
0709 12:24:59.450808 (+1200662us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b540047ca54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:24:59.467547 (+ 16739us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236673920}
I20260709 12:24:59.468667  5149 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:59.468735  5067 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:24:59.477730  5224 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:24:59.477900  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.477923  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.478899  5148 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:24:59.471422  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:59.453437  9416 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:59.486001  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.446481  9501 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.482481  9215 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:59.497514  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.498894  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.499445  5148 raft_consensus.cc:2376] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16. Re-sending same reply.
I20260709 12:24:59.499828  9271 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:59.472404  9352 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:59.500449  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.500768  9352 raft_consensus.cc:697] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:59.457720  9565 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:59.501370  9565 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.463835  9571 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:59.502308  9571 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:59.502743  9571 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:59.502791  5145 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:59.456837  9355 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:59.491459  9499 raft_consensus.cc:3055] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:59.492918  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.480031  9153 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:59.501791  9352 consensus_queue.cc:237] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.459822  9539 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:59.504442  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.469161  9419 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:59.505455  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.515278  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.512408  5147 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:24:59.513245  5145 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:24:59.516332  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:59.517263  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.517963  5145 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 14: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:24:59.436690  9569 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.452930  9563 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.501860  9565 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:59.534125  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.526675  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.488039  9579 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:59.535202  5147 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:24:59.535339  5145 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:24:59.537674  9579 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:59.512079  9416 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:59.512018  9499 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:59.525624  9271 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:59.544818  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.555579  5222 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:59.511793  9607 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:59.557626  5147 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:59.562095  5141 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:59.544407  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.525818  9356 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:59.563413  5146 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:59.510761  9501 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.513921  9355 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:59.551402  9579 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:59.550876  9605 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:24:59.574816  4888 maintenance_manager.cc:419] P ce6fc7638c8549faa2535fb5774c32c1: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7472 bytes on disk
I20260709 12:24:59.578070  9416 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:59.580998  9501 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:59.595589  9501 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:59.582095  9604 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:24:59.583228  9542 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.584378  9356 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:59.563453  9607 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:59.590468  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 121707us
I20260709 12:24:59.605257  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:24:59.621273  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 106097us
I20260709 12:24:59.613049  5141 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:24:59.624223  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.623943  4887 maintenance_manager.cc:643] P ce6fc7638c8549faa2535fb5774c32c1: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.028s	user 0.001s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":444,"lbm_reads_lt_1ms":4}
I20260709 12:24:59.624944  5141 raft_consensus.cc:2376] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16. Re-sending same reply.
I20260709 12:24:59.627157  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.627153  5190 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:59.631657  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.633085  5223 raft_consensus.cc:2410] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 17 because replica has last-logged OpId of term: 14 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 12:24:59.543263  9319 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.643392  5216 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:24:59.634725  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:59.641208  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:59.594977  9355 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:59.645406  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.646574  5042 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.646643  5141 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:59.648582  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.656390  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.556473  9153 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.665445  5190 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.612243  9607 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.663228  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:24:59.664315  5042 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:59.667570  5042 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:24:59.668165  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 85567.4us
I20260709 12:24:59.668619  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:24:59.638571  9271 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:59.668959  5042 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:59.646374  9419 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:59.661350  9215 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.659438  9483 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:59.677687  5141 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:24:59.684437  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.677748  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:59.667717  9355 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:59.679169  9219 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:59.681834  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.682312  9322 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:59.670184  9609 raft_consensus.cc:2764] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:24:59.683084  9356 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.667529  9552 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:59.684329  9542 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:59.685672  5940 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:24:59.675737  9361 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:59.675012  9548 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:59.677500  9526 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:59.665618  9538 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:59.622347  9499 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:59.716879  9538 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:59.717130  9499 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:59.717905  9538 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.720182  9555 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:59.696099  9474 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.722370  5145 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:59.723349  9474 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:59.670936  9585 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:59.680372  9419 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:59.723729  9571 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:59.724417  9419 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:59.726801  9319 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:59.703691  5150 raft_consensus.cc:2376] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14. Re-sending same reply.
I20260709 12:24:59.705361  5220 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:24:59.706125  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:59.709223  5042 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:24:59.690559  9197 raft_consensus.cc:3055] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:59.705595  9355 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:59.705672  9413 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:59.709625  9493 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:59.730494  9197 raft_consensus.cc:740] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:59.739310  9493 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:59.739943  9493 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:59.736977  9355 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.740726  9609 consensus_queue.cc:1059] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:59.742556  9609 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:59.742906  9609 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:59.743664  9609 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:59.716180  9552 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:24:59.744726  9413 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:59.733047  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:59.746302  9482 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.746958  9319 raft_consensus.cc:2764] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:24:59.747336  9482 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.749411  5899 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:24:59.749785  9197 consensus_queue.cc:260] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.750849  9197 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.751901  5899 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:59.752045  9542 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:24:59.753561  9413 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.753644  9542 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:59.754896  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.756865  9542 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:24:59.746829  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 528952us
I20260709 12:24:59.744889  9609 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:59.771252  5068 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:59.759696  5900 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.731215  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:24:59.760267  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:59.712419  9361 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:59.774101  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 349909us
I20260709 12:24:59.774962  5898 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.775861  5900 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.758924  9585 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:24:59.778723  9586 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:59.714550  9548 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:24:59.780550  5069 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:24:59.781407  5069 pending_rounds.cc:85] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 6
I20260709 12:24:59.781891  5069 pending_rounds.cc:107] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:24:59.782212  5069 raft_consensus.cc:2889] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:59.787986  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:24:59.753893  9439 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.793296  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.796911  5143 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:59.749136  9565 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.759588  9483 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:59.780467  9361 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:59.808225  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 55406.5us
I20260709 12:24:59.811237  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.788709  9548 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:59.813230  5143 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:24:59.815296  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:24:59.809767  5899 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:24:59.822234  5900 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:24:59.795895  9570 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.795100  9568 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:59.825709  5898 leader_election.cc:400] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17.
I20260709 12:24:59.834853  5190 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:59.813666  9581 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:24:59.844954  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.790519  9572 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:24:59.813372  9563 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:24:59.846320  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:59.814983  9483 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:59.822139  9312 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:59.830794  9576 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:59.833323  9365 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:59.866998  9413 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:59.870345  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:24:59.840416  9568 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.806155  9439 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.892071  9586 raft_consensus.cc:697] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:59.849519  9526 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:59.839291  9581 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.901005  5947 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:24:59.850135  9582 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:59.844393  9169 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:59.866992  9483 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.865406  9574 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:59.900079  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:59.870733  9312 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.916255  5941 raft_consensus.cc:1275] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:59.915930  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:24:59.943667  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 121542us
I20260709 12:24:59.878870  9122 raft_consensus.cc:2764] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:24:59.809772  9552 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:59.881744  9365 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.842936  9570 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:24:59.897840  5144 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:24:59.915243  9630 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:24:59.913260  9153 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:59.927552  5942 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:24:59.953692  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:24:59.922669  9581 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:59.930303  9619 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:24:59.930483  9439 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:24:59.931303  9624 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:24:59.934042  9586 consensus_queue.cc:237] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:24:59.935303  9571 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:24:59.920259  9526 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:00.008170  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:00.008757  9526 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:00.009858  9570 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:24:59.961052  9617 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:24:59.939359  9215 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:00.014261  9355 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:00.016647  9355 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:59.945276  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:24:59.950702  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 141031us
I20260709 12:24:59.961360  9539 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:00.018239  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 73054.6us
I20260709 12:25:00.019069  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:00.019507  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:00.020097  9569 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:00.020433  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.021214  5943 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:59.920951  9413 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:59.923915  9582 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:00.056874  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:24:59.907490  9568 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:59.956858  5115 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:00.058401  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 101607us
I20260709 12:25:00.058420  5069 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:24:59.914040  9499 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:24:59.950739  9365 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:59.945417  9312 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:59.947521  9574 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:24:59.923739  9585 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:24:59.909943  9607 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:00.084487  9574 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.085443  5143 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 10: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:00.086006  9607 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:24:59.925202  9532 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 12:24:59.965705  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.087538  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:24:59.967497  9622 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:25:00.090010  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1394) took 1089 ms. Trace:
I20260709 12:25:00.090914  9618 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:00.091503  9618 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.091871  9618 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:59.967882  5222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1406) took 1167 ms. Trace:
I20260709 12:25:00.073932  9579 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:00.094893  5149 raft_consensus.cc:2393] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:00.094213  5222 rpcz_store.cc:276] 0709 12:24:58.800171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.800315 (+   144us) service_pool.cc:224] Handling call
0709 12:24:58.800662 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.561792 (+761130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.967592 (+405800us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b54003cafd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:59.967812 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1735420288}
I20260709 12:25:00.095909  9579 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:00.096074  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1265) took 1137 ms. Trace:
I20260709 12:25:00.096418  5149 rpcz_store.cc:276] 0709 12:24:58.958862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.472026 (+513164us) service_pool.cc:224] Handling call
0709 12:25:00.095882 (+623856us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400239954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:00.096018 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398304000}
I20260709 12:25:00.097641  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:24:59.968811  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1385) took 1008 ms. Trace:
I20260709 12:24:59.960507  9153 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:24:59.959574  9563 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:00.099506  9563 raft_consensus.cc:697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:00.099675  9153 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:24:59.978411  9548 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:24:59.953605  9474 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:24:59.991516  5148 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:59.993552  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.001469  9284 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.001890  9565 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:00.004577  9540 consensus_meta.cc:287] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.001s
I20260709 12:25:00.005978  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.958693  9630 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.007386  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:00.021339  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.022181  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:00.022969  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:00.023514  9436 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:00.023741  5220 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:00.024098  9566 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:00.024601  9569 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:00.025501  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:24:59.881771  9576 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:00.024101  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:00.030088  5114 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:00.034381  9608 log.cc:927] Time spent T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.373s	user 0.001s	sys 0.004s
I20260709 12:25:00.034654  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:24:59.988327  9419 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:00.047153  5900 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:00.056370  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:24:59.954413  5941 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:00.057717  5067 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:00.064244  5216 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:00.067791  5071 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:00.068629  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:00.065948  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:00.073046  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:00.074707  9629 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.063337  9582 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:00.146986  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 99934.3us
W20260709 12:25:00.147948  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 117999us
I20260709 12:25:00.148270  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:00.148584  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:00.149111  5224 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:00.104781  5042 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:00.088172  5143 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:00.098999  5145 rpcz_store.cc:276] 0709 12:24:58.960464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.537398 (+576934us) service_pool.cc:224] Handling call
0709 12:24:59.537722 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.722754 (+185032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.968559 (+245805us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54004fae54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:59.968746 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424203008}
I20260709 12:24:59.984915  5147 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:59.989264  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1405) took 1189 ms. Trace:
I20260709 12:25:00.099038  5222 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:00.149107  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:00.123575  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.098887  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:00.154839  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:00.133513  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:00.141575  5073 raft_consensus.cc:2410] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:00.142385  5943 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:00.142728  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:00.155021  5114 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:00.143035  5067 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:00.090689  5146 rpcz_store.cc:276] 0709 12:24:59.000988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.568495 (+567507us) service_pool.cc:224] Handling call
0709 12:24:59.568900 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.616752 (+ 47852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.616790 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.617449 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.781508 (+164059us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:59.880968 (+ 99460us) log.cc:844] Serialized 14 byte log entry
0709 12:24:59.881292 (+   324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.881400 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.089668 (+208268us) raft_consensus.cc:1685] finished
0709 12:25:00.089702 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.089920 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":84924,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":35895,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":17843,"thread_start_us":230198,"threads_started":3,"wal-append.queue_time_us":140093}
W20260709 12:25:00.145804  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.157924  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:00.158502  5071 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:00.152020  5217 rpcz_store.cc:276] 0709 12:24:58.799698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.799883 (+   185us) service_pool.cc:224] Handling call
0709 12:24:58.828370 (+ 28487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.086288 (+257918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.988989 (+902701us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54002c1ad4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:24:59.989197 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":585393024}
I20260709 12:25:00.160534  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:00.164186  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:00.168776  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:00.169283  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 858) took 1124 ms. Trace:
I20260709 12:25:00.169514  5145 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:00.169694  5143 rpcz_store.cc:276] 0709 12:24:59.045227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.168373 (+1123146us) service_pool.cc:224] Handling call
0709 12:25:00.168747 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.168955 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.168982 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.169025 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:00.169054 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.169105 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.169223 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.175391  5146 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.177078  5149 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: [12.7-13.8]
W20260709 12:25:00.181887  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 159752us
I20260709 12:25:00.183665  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:00.183131  5041 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:00.182085  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 77406.7us
I20260709 12:25:00.185860  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:00.188082  5042 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:00.188688  5149 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:25:00.188572  5223 raft_consensus.cc:1240] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:25:00.189837  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:00.191601  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:00.193121  5074 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:00.193579  5070 raft_consensus.cc:2393] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:00.194396  5941 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:00.195472  5216 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:00.198891  5149 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:00.131203  9532 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:00.210628  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:00.210757  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:00.212280  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:00.204646  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:00.175469  5067 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:00.213907  5067 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:00.206156  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:00.207701  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:00.149942  9576 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:00.209184  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:00.221401  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:00.223140  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:00.147123  9583 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.226456  5071 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:25:00.226900  5073 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:00.226452  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:00.230288  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1387) took 1269 ms. Trace:
I20260709 12:25:00.230768  5147 rpcz_store.cc:276] 0709 12:24:58.961142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.562844 (+601702us) service_pool.cc:224] Handling call
0709 12:24:59.574378 (+ 11534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.151960 (+577582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.230057 (+ 78097us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b5400238f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:00.230222 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":943066880}
I20260709 12:25:00.159179  9439 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.168422  9197 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.233984  5215 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:00.159416  9585 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:00.250536  9122 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:00.159832  9570 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:00.251535  9197 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:00.252224  9583 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.253746  9197 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:00.254773  9583 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:00.255519  5143 raft_consensus.cc:1240] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:00.225574  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:00.261415  5943 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:00.243081  5075 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:00.261266  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:00.262379  5145 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:25:00.243232  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:00.245273  9548 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:00.245510  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:00.263193  5947 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:00.245829  9615 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:25:00.175139  9538 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:25:00.246490  9579 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:00.191543  9526 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:25:00.207937  9551 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:00.247550  9581 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.203004  9482 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:00.209012  9539 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:00.209350  9571 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:25:00.249841  9572 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:00.234474  5224 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:00.100546  9563 consensus_queue.cc:237] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:00.271876  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:00.274322  5147 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:25:00.203063  9355 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:00.314282  5189 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:00.317173  5942 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:25:00.136919  9284 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:00.253602  9630 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:00.210625  9532 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:00.162329  9540 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.159889  9569 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:00.233771  9416 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:00.239261  9483 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:00.287254  5071 raft_consensus.cc:3055] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:00.090103  9413 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:00.170441  9629 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:00.290760  5149 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:00.191434  9271 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:00.233350  9632 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:00.292307  5146 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:00.294697  5067 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:00.210273  9493 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.207356  9501 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:25:00.202447  9605 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:00.336870  9416 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:00.328482  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 122400us
I20260709 12:25:00.257850  9439 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:00.331436  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:00.370819  5073 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:00.323426  9312 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.288597  9548 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:00.332218  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:00.321523  9567 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:00.334796  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:00.282042  9585 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 12:25:00.335932  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 138232us
I20260709 12:25:00.338577  5071 raft_consensus.cc:740] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:00.339571  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 127399us
W20260709 12:25:00.341643  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 179065us
I20260709 12:25:00.344609  5221 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:00.284512  9542 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:00.346328  5070 raft_consensus.cc:3055] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:25:00.355877  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 145355us
I20260709 12:25:00.338174  9483 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:00.279651  9551 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:00.362617  9632 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:00.373328  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:00.364116  9361 consensus_meta.cc:287] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.000s
I20260709 12:25:00.391212  5071 consensus_queue.cc:260] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:00.399459  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 64757.6us
I20260709 12:25:00.369249  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 865) took 1294 ms. Trace:
I20260709 12:25:00.407653  5146 rpcz_store.cc:276] 0709 12:24:59.068858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.174934 (+1106076us) service_pool.cc:224] Handling call
0709 12:25:00.175268 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.362809 (+187541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.363012 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.393033  5070 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:00.371232  9416 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:00.407044  5147 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:00.409304  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:00.413059  5071 raft_consensus.cc:3060] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:00.413623  5217 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:25:00.410146  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:00.412678  9361 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:00.389299  9628 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:00.415858  9642 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:00.417403  9583 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:25:00.415647  9626 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.428128  9312 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:00.428371  9626 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:00.402314  9551 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:00.430763  9626 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:00.402983  9548 consensus_queue.cc:1059] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:25:00.390743  9153 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:25:00.399827  9483 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:00.410667  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:00.390669  9356 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:00.417943  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:00.444679  5146 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:00.402012  9419 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:00.396925  9542 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:00.365239  9271 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:00.452751  5148 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
W20260709 12:25:00.460409  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 84819.5us
I20260709 12:25:00.409040  9526 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:00.472043  5146 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:00.477574  5071 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:00.406451  9632 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:00.477775  5900 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:00.414057  9643 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:00.421234  9636 log.cc:927] Time spent T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:00.483616  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:00.434262  9416 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:00.468938  9639 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.099s	user 0.002s	sys 0.000s
I20260709 12:25:00.420288  9439 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:00.490470  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 91579.1us
I20260709 12:25:00.498102  5041 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:00.420924  9361 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:00.499858  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.499800  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 89284.4us
I20260709 12:25:00.469185  9581 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:00.504088  9638 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:25:00.539444  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:00.540285  5942 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:25:00.541215  5075 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:00.504563  9284 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:00.479269  9526 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:00.389777  9532 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:00.481456  9356 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:00.451181  9626 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:25:00.476186  9153 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:25:00.500636  9312 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:00.544312  9215 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
W20260709 12:25:00.556226  9637 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 12:25:00.556499  9215 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:00.556921  9215 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:00.557462  9569 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:00.555612  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 857) took 1510 ms. Trace:
W20260709 12:25:00.558022  9609 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.778s	user 0.006s	sys 0.007s
I20260709 12:25:00.557811  9215 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.559106  9215 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:00.548671  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:00.549026  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:00.595046  5943 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:25:00.550242  5946 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:25:00.550359  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:00.558395  5150 rpcz_store.cc:276] 0709 12:24:59.044956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.057643 (+1012687us) service_pool.cc:224] Handling call
0709 12:25:00.072015 (+ 14372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.099937 (+ 27922us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:00.100264 (+   327us) log.cc:844] Serialized 14 byte log entry
0709 12:25:00.362002 (+261738us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:00.391943 (+ 29941us) log.cc:844] Serialized 26 byte log entry
0709 12:25:00.392121 (+   178us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:00.392366 (+   245us) log.cc:844] Serialized 14 byte log entry
0709 12:25:00.410018 (+ 17652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.410090 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.555374 (+145284us) raft_consensus.cc:1685] finished
0709 12:25:00.555397 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.555545 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59211,"prepare.queue_time_us":4044,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":84,"raft.queue_time_us":18450,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":6405,"thread_start_us":261114,"threads_started":1,"wal-append.queue_time_us":315192}
I20260709 12:25:00.608991  5073 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:00.609156  5220 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
W20260709 12:25:00.551628  9634 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.093s	user 0.002s	sys 0.000s
I20260709 12:25:00.611421  9365 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.612227  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1400) took 1582 ms. Trace:
I20260709 12:25:00.612747  5141 rpcz_store.cc:276] 0709 12:24:59.029451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.703049 (+673598us) service_pool.cc:224] Handling call
0709 12:24:59.703477 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.715048 (+ 11571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.715068 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.721837 (+  6769us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.971206 (+249369us) spinlock_profiling.cc:243] Waited 8.84 ms on lock 0x7b18001cae60. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db0d1 00007fd8a84f6a0f
0709 12:25:00.242006 (+270800us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:00.263279 (+ 21273us) log.cc:844] Serialized 14 byte log entry
0709 12:25:00.263534 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.263604 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.611873 (+348269us) raft_consensus.cc:1685] finished
0709 12:25:00.611992 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.612148 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11105,"prepare.queue_time_us":253,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":11342,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":14718,"spinlock_wait_cycles":20328704,"thread_start_us":270565,"threads_started":1,"wal-append.queue_time_us":610163}
I20260709 12:25:00.551867  9356 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:00.615999  5941 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:00.616433  9618 consensus_meta.cc:287] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 12:25:00.616853  9618 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:00.552989  9641 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.619154  9641 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:00.554289  9532 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:00.524654  9542 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.624440  9542 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:00.624814  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:00.555697  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:00.525835  9499 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:00.626238  5071 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:00.547345  9526 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:00.627234  9319 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:00.628746  9319 consensus_queue.cc:1059] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:00.630241  9538 raft_consensus.cc:3055] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:00.630820  9538 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:00.631856  9538 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:00.633239  9538 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:00.574463  9271 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:00.644416  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:00.574729  9416 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:00.575085  9629 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:00.543630  9572 consensus_queue.cc:1059] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:25:00.547360  9581 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:00.536998  9539 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:00.667492  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1267) took 1458 ms. Trace:
I20260709 12:25:00.668082  5145 rpcz_store.cc:276] 0709 12:24:59.208842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.170397 (+961555us) service_pool.cc:224] Handling call
0709 12:25:00.170756 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.262802 (+ 92046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.667173 (+404371us) spinlock_profiling.cc:243] Waited 91.4 ms on lock 0x7b540036e554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:00.667427 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210140032}
I20260709 12:25:00.577962  9493 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:00.669937  9365 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:00.647428  9413 consensus_queue.cc:1059] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:25:00.637435  9641 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:00.580114  9568 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.503s	user 0.005s	sys 0.005s
I20260709 12:25:00.579185  5070 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:00.581260  9586 raft_consensus.cc:3055] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:00.617718  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:00.633738  9646 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.528967  9642 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:00.639719  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:00.679786  5067 raft_consensus.cc:2410] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:00.641026  5071 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:00.641743  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:00.641883  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:00.682334  5069 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:00.642266  5942 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:00.645128  9284 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:00.646013  9548 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:00.638995  9566 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:00.651202  5150 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:00.657691  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:00.660854  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:00.691171  5072 raft_consensus.cc:2410] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:25:00.691013  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:00.658938  9609 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.694556  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:00.665182  9654 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:25:00.665393  9581 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:00.561225  9540 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:00.532585  9643 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.618470  9482 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:00.544720  9439 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:25:00.686168  5070 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:00.603233  9576 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:00.692440  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:00.680174  9586 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:00.664754  9572 raft_consensus.cc:3055] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:00.695390  5073 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:00.672853  5041 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:00.700619  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:25:00.702554  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:00.720471  5943 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:00.728473  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:00.695147  9219 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:00.693274  9642 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:25:00.872522  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 170085us
I20260709 12:25:00.726675  9618 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:00.686239  9582 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:00.740700  9572 raft_consensus.cc:740] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:00.686852  9646 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:00.718083  9609 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:00.865017  5076 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:00.865383  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 164830us
I20260709 12:25:00.680733  9568 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:00.867343  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 194593us
I20260709 12:25:00.891217  5070 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:00.891294  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:00.727180  9628 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.141s
I20260709 12:25:00.868377  9569 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:00.671236  9551 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:00.871541  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:00.868261  5939 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:00.886245  5146 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 15: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:00.886629  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:00.889887  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:00.890491  5898 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:00.997458  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.977205  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.032752  5150 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:01.006527  5945 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:01.040910  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:01.041975  5147 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:01.045120  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:01.046556  5147 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:01.050117  5940 raft_consensus.cc:2393] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 17: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:00.888788  9626 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:00.890682  9565 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:01.071060  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 857) took 1832 ms. Trace:
I20260709 12:25:01.035458  9419 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:01.071666  5221 rpcz_store.cc:276] 0709 12:24:59.238938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.336705 (+ 97767us) service_pool.cc:224] Handling call
0709 12:24:59.370579 (+ 33874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.392630 (+1022051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.070647 (+678017us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54001437d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:01.070948 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239623424}
I20260709 12:25:01.048452  9628 consensus_queue.cc:1243] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:00.895139  9436 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.176s
W20260709 12:25:01.056318  9662 log.cc:927] Time spent T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:25:01.005381  9319 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:25:00.868302  9632 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:01.122323  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 232486us
I20260709 12:25:01.067844  9626 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:00.893867  9483 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:01.133999  9551 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:01.138377  9581 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:01.178071  9284 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:01.178442  5188 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:01.179118  9632 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:01.179505  5114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.009s	sys 0.015s
I20260709 12:25:01.179648  9284 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:01.181083  9284 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:01.181459  9563 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:01.183811  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:01.184566  5224 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:01.139190  9644 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.077739  9271 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:01.141250  9568 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:01.188371  9644 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:01.142771  9526 consensus_meta.cc:287] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.003s
I20260709 12:25:01.189292  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1133) took 1154 ms. Trace:
I20260709 12:25:01.189747  9526 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:01.189886  5945 rpcz_store.cc:276] 0709 12:25:00.034884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.036404 (+  1520us) service_pool.cc:224] Handling call
0709 12:25:00.072666 (+ 36262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.035686 (+963020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.188998 (+153312us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b540000a854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:01.189224 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2146936192}
I20260709 12:25:01.190732  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:01.191942  9526 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:01.066610  9572 consensus_queue.cc:260] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:01.193051  9572 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:01.195148  9644 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:01.195910  9169 consensus_queue.cc:1175] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" responder_term: 12 status { last_received { term: 12 index: 8 } last_committed_idx: 7 last_received_current_leader { term: 12 index: 8 } }
I20260709 12:25:01.060129  9586 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:01.144382  9629 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.198410  9629 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:01.199172  9671 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.130082  9628 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:25:01.091589  9416 raft_consensus.cc:3055] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:01.145921  9439 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:01.200515  9416 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:01.200778  9439 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.201231  9439 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:01.149967  9642 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:01.153708  9582 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:01.155768  9576 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:01.154435  5150 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:25:01.214360  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:01.134078  9643 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:01.183128  9284 raft_consensus.cc:697] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:01.160432  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 906) took 1054 ms. Trace:
I20260709 12:25:01.092661  9419 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.154949  9609 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:01.170861  5939 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:01.173484  9668 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:01.173789  9356 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:01.168299  9626 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:01.175787  9483 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:01.176329  9585 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:01.176964  9215 consensus_meta.cc:287] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.000s
I20260709 12:25:01.177181  9319 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:01.188443  9271 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:01.183601  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:25:01.138577  9664 log.cc:927] Time spent T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.249s	user 0.000s	sys 0.000s
I20260709 12:25:01.152237  9646 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:01.202296  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:01.210215  5220 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:01.183451  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:01.216964  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:01.159257  9542 consensus_meta.cc:287] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.000s
W20260709 12:25:01.223692  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 175972us
I20260709 12:25:01.223250  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:01.224079  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:01.450752  5150 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:01.229489  5114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.011s	sys 0.016s
W20260709 12:25:01.451465  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 1.25828e+06us
I20260709 12:25:01.452359  5114 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:01.225627  5948 rpcz_store.cc:276] 0709 12:25:00.105642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.235861 (+130219us) service_pool.cc:224] Handling call
0709 12:25:00.236279 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.433866 (+197587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.433898 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:00.434374 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:25:00.509621 (+ 75247us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:00.509655 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.160006 (+650351us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54002413d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:01.160058 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.160107 (+    49us) raft_consensus.cc:1685] finished
0709 12:25:01.160120 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.160367 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36842,"raft.queue_time_us":35081,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":27315,"spinlock_wait_cycles":382412032,"thread_start_us":74198,"threads_started":1,"wal-append.queue_time_us":129790}
I20260709 12:25:01.211189  9169 raft_consensus.cc:3055] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:01.171315  9667 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:25:01.236248  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:01.549908  5141 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
W20260709 12:25:01.553187  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 102473us
I20260709 12:25:01.467818  9169 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:01.391007  5147 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:01.228958  9419 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:01.395762  9319 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:01.395948  9539 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.406250  5221 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:01.430459  5217 raft_consensus.cc:2410] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:25:01.441053  9271 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:01.400036  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 509580us
I20260709 12:25:01.443857  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1215) took 1004 ms. Trace:
I20260709 12:25:01.444743  9629 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:01.601735  5215 rpcz_store.cc:276] 0709 12:25:00.439232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.439488 (+   256us) service_pool.cc:224] Handling call
0709 12:25:00.480343 (+ 40855us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:00.480801 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.480838 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:00.507438 (+ 26600us) log.cc:844] Serialized 52 byte log entry
0709 12:25:00.632278 (+124840us) spinlock_profiling.cc:243] Waited 9.88 ms on lock 0x7b48002211d8. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ad464aa1 00007fd8ad45bb37 00007fd8ad45b9ca 00007fd8ad55326f 00007fd8ad552f37 00007fd8ad56755c 00007fd8ad567508 00007fd8ad567488 00007fd8ad567443 00007fd8ad5666ec 00007fd8ac490a96
0709 12:25:00.684478 (+ 52200us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:00.684523 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.684598 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.443446 (+758848us) raft_consensus.cc:1685] finished
0709 12:25:01.443473 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.443751 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4089,"raft.queue_time_us":1021,"raft.run_cpu_time_us":2309,"raft.run_wall_time_us":146920,"spinlock_wait_cycles":22720256,"thread_start_us":143925,"threads_started":1,"wal-append.queue_time_us":188539}
W20260709 12:25:01.448510  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.199889  9671 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:01.460193  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:01.461118  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:01.543991  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.610513  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:01.467994  9644 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.398638  9585 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:01.601147  9568 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:01.672670  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:01.652465  5945 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:01.563690  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:01.675796  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:01.552407  9538 consensus_meta.cc:287] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.901s	user 0.008s	sys 0.003s
I20260709 12:25:01.468012  9643 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:01.601289  5075 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1510) took 1271 ms. Trace:
I20260709 12:25:01.671634  5899 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:01.657608  9676 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:25:01.606003  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:01.688234  5220 raft_consensus.cc:2393] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:01.606536  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:01.689458  5113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.017s	sys 0.021s
W20260709 12:25:01.689837  5113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.017s	sys 0.021s
I20260709 12:25:01.689926  5216 raft_consensus.cc:2410] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:25:01.606686  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:01.468192  9557 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:01.235872  9356 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:01.229269  9416 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:01.602026  5189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.029s	sys 0.021s
W20260709 12:25:01.722754  5189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.030s	sys 0.021s
I20260709 12:25:01.724068  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:01.650491  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:01.650229  5073 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:01.651768  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:01.651768  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:01.652423  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:01.652624  9648 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:01.653383  9319 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:01.560770  9646 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 12:25:01.611115  9539 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:01.602541  9284 consensus_queue.cc:237] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:01.610003  5150 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:01.676543  5071 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:01.564886  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 875) took 1125 ms. Trace:
I20260709 12:25:01.746444  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 872) took 1328 ms. Trace:
I20260709 12:25:01.746950  5150 rpcz_store.cc:276] 0709 12:25:00.418218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.215695 (+797477us) service_pool.cc:224] Handling call
0709 12:25:01.746363 (+530668us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.690583  9581 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:01.688624  9613 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:01.678916  5222 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:01.670753  9644 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:01.682000  5075 rpcz_store.cc:276] 0709 12:25:00.329931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.330082 (+   151us) service_pool.cc:224] Handling call
0709 12:25:00.339750 (+  9668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.541758 (+202008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.601014 (+1059256us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54003570d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:01.601220 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462868352}
I20260709 12:25:01.607849  5070 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:01.648077  9271 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:01.731259  9643 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:01.719745  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:01.672629  9585 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:01.737481  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:01.782370  5069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.013s	sys 0.013s
I20260709 12:25:01.782493  5945 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:25:01.782886  5073 raft_consensus.cc:2393] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 10: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:01.685096  9667 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:01.785918  5215 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:01.671216  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:01.680794  5224 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:01.673242  9568 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:01.665417  9419 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:01.762146  5220 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:01.762133  5216 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:01.764657  5221 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:01.795141  5224 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:01.777870  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:01.782958  5069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.013s	sys 0.013s
I20260709 12:25:01.745105  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:01.784016  5114 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:01.798842  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:01.763293  9676 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:01.720899  9557 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:01.749379  9219 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:25:01.815373  5945 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:25:01.742267  9648 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:01.741443  9538 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:01.746833  5147 rpcz_store.cc:276] 0709 12:25:00.439620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.044809 (+605189us) service_pool.cc:224] Handling call
0709 12:25:01.564793 (+519984us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.749857  9581 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:01.670643  9169 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:01.601997  9582 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.165s
I20260709 12:25:01.746312  9645 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:01.768558  9416 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:01.602002  9483 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.797187  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:01.816036  5945 pending_rounds.cc:85] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 6
I20260709 12:25:01.829161  5945 pending_rounds.cc:107] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:25:01.829455  5945 raft_consensus.cc:2889] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:01.821182  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:01.809669  9676 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:01.783912  9319 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:01.821172  9581 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:01.803877  9555 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:01.804112  9585 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:01.750846  9605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.022s	sys 0.033s
I20260709 12:25:01.825289  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:01.871719  5147 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:01.806695  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:01.876359  5215 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:25:01.833806  9642 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.835322  5075 raft_consensus.cc:2410] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:01.837719  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:01.966265  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:01.834270  9169 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:01.862022  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 60321.5us
I20260709 12:25:01.970125  5147 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:01.970842  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1284) took 1431 ms. Trace:
I20260709 12:25:01.790069  9643 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:01.971148  5147 rpcz_store.cc:276] 0709 12:25:00.539366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.824955 (+1285589us) service_pool.cc:224] Handling call
0709 12:25:01.970778 (+145823us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.972150  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:01.972724  5147 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:01.973625  9643 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:01.974534  9684 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:25:01.793196  9667 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:01.821836  9538 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.869802  5141 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:01.828095  9219 raft_consensus.cc:3055] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:01.979601  9219 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:01.980727  9219 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:01.981961  9219 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:01.982378  9568 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:01.967110  9674 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:01.983623  9674 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:01.985790  9419 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:01.961558  9605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.022s	sys 0.034s
I20260709 12:25:01.962153  9646 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:01.987666  9646 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:01.969599  9679 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:01.991375  9679 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:01.967109  5073 raft_consensus.cc:2410] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:25:01.788728  9271 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:01.768781  9613 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:01.996788  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:01.999464  5189 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:01.977159  9667 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:01.863547  9197 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:01.998121  5945 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
I20260709 12:25:01.817466  9675 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.155s
I20260709 12:25:02.003405  5069 raft_consensus.cc:1275] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:25:02.003648  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 200280us
I20260709 12:25:02.005600  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:02.012853  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:02.013646  9585 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:02.015118  5074 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:02.015506  5074 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:02.016328  5074 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:02.017258  5074 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:02.018446  9644 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:02.018668  9675 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:01.834591  9645 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:02.018903  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:01.833837  9483 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:02.008375  5898 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:02.009702  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:02.007861  5147 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:02.025224  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 883) took 1373 ms. Trace:
I20260709 12:25:02.024358  5141 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:02.026561  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:02.027726  5946 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:02.029301  5073 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:02.025998  5147 rpcz_store.cc:276] 0709 12:25:00.651450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.971872 (+1320422us) service_pool.cc:224] Handling call
0709 12:25:02.025150 (+ 53278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.997092  9538 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:02.011777  9683 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:25:02.035255  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:02.040138  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 874) took 1601 ms. Trace:
I20260709 12:25:02.040562  5147 rpcz_store.cc:276] 0709 12:25:00.439198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.039474 (+1600276us) service_pool.cc:224] Handling call
0709 12:25:02.039744 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.039903 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.039919 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.039951 (+    32us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:02.039972 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.040008 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.040083 (+    75us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.019481  9675 raft_consensus.cc:3055] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:02.043265  9675 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:01.966024  9642 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.072127  5069 raft_consensus.cc:2410] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:02.044302  9675 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:02.080544  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 81177.9us
I20260709 12:25:02.065299  9539 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:02.081866  5189 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.65:37253 timed out after 2.631s (SENT) [suppressed 36 similar messages]
W20260709 12:25:02.065719  9439 consensus_meta.cc:287] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.853s	user 0.007s	sys 0.006s
I20260709 12:25:02.082772  9439 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:02.084944  5074 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:25:02.085439  5074 pending_rounds.cc:85] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 7
I20260709 12:25:02.085780  5074 pending_rounds.cc:107] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:25:02.022305  9197 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.032410  9568 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:02.086048  5074 raft_consensus.cc:2889] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:02.086648  9568 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:02.086831  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1515) took 1447 ms. Trace:
I20260709 12:25:02.087218  9568 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.088117  9197 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:02.067534  9646 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:02.088449  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:02.089116  9646 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:02.068819  9572 consensus_meta.cc:287] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.875s	user 0.008s	sys 0.004s
I20260709 12:25:02.069244  9645 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:02.070395  9681 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:25:02.090654  9568 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.078245  5945 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:02.070101  9688 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:02.091373  5945 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:02.012426  9667 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:02.092275  5945 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.092779  9667 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:02.087203  5074 rpcz_store.cc:276] 0709 12:25:00.639535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.639999 (+   464us) service_pool.cc:224] Handling call
0709 12:25:00.640352 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.015016 (+1374664us) spinlock_profiling.cc:243] Waited 56.5 ms on lock 0x7b540030ee20. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a84b2dc0 00007fd8a853d1c0 00007fd8a853e231 00007fd8a853ad8f 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c
0709 12:25:02.086540 (+ 71524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.086769 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129983744}
I20260709 12:25:02.095445  9667 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:02.090718  9689 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:02.089910  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:02.037870  9644 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.112345  5150 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:02.116125  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1286) took 1576 ms. Trace:
I20260709 12:25:02.117074  5147 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:02.116799  5150 rpcz_store.cc:276] 0709 12:25:00.539946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.097385 (+1557439us) service_pool.cc:224] Handling call
0709 12:25:02.097870 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.115668 (+ 17798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.116031 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.118011  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1285) took 1578 ms. Trace:
I20260709 12:25:02.118582  5147 rpcz_store.cc:276] 0709 12:25:00.539552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.072711 (+1533159us) service_pool.cc:224] Handling call
0709 12:25:02.107797 (+ 35086us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.117649 (+  9852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.117947 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.119297  5150 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:25:02.033690  9671 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:02.124377  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 877) took 1562 ms. Trace:
I20260709 12:25:02.125005  5150 rpcz_store.cc:276] 0709 12:25:00.561893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.118369 (+1556476us) service_pool.cc:224] Handling call
0709 12:25:02.118891 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.119961 (+  1070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.124279 (+  4318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.127389  5150 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:25:02.127924  5150 pending_rounds.cc:85] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 6
I20260709 12:25:02.085266  9439 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:02.132738  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 97632.6us
I20260709 12:25:00.447079  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:02.135470  9692 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:25:02.135797  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 123025us
I20260709 12:25:02.136397  9692 consensus_queue.cc:1243] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:25:02.133354  5150 pending_rounds.cc:107] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:25:02.137765  5150 raft_consensus.cc:2889] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:02.138590  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 879) took 1557 ms. Trace:
I20260709 12:25:02.075325  9356 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.139050  5150 rpcz_store.cc:276] 0709 12:25:00.581283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.126223 (+1544940us) service_pool.cc:224] Handling call
0709 12:25:02.126796 (+   573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.138311 (+ 11515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.138526 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.144629  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 878) took 1582 ms. Trace:
I20260709 12:25:02.145087  5150 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:02.076181  9679 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.145905  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 881) took 1564 ms. Trace:
I20260709 12:25:02.145346  5147 rpcz_store.cc:276] 0709 12:25:00.562355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.119876 (+1557521us) service_pool.cc:224] Handling call
0709 12:25:02.143690 (+ 23814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.144103 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.144148 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.144220 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:02.144283 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.144370 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.144539 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:02.109999  9153 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:25:02.149585  5150 rpcz_store.cc:276] 0709 12:25:00.581892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.144681 (+1562789us) service_pool.cc:224] Handling call
0709 12:25:02.144944 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.145630 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.145849 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.101994  9689 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:02.113623  9667 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:02.155701  5150 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:25:02.161306  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1415) took 1521 ms. Trace:
I20260709 12:25:02.161886  5150 rpcz_store.cc:276] 0709 12:25:00.641074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.154842 (+1513768us) service_pool.cc:224] Handling call
0709 12:25:02.155360 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.160827 (+  5467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.161227 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.164135  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1416) took 1522 ms. Trace:
I20260709 12:25:02.164587  5150 rpcz_store.cc:276] 0709 12:25:00.641523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.162953 (+1521430us) service_pool.cc:224] Handling call
0709 12:25:02.163415 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.163702 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.163742 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.163807 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:02.163854 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.163926 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.164063 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.168428  5150 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 12:25:02.139531  9439 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:02.176209  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:02.169314  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1417) took 1527 ms. Trace:
W20260709 12:25:02.152143  9626 consensus_meta.cc:287] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.778s	user 0.007s	sys 0.005s
I20260709 12:25:02.171221  5147 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:02.187914  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1412) took 1548 ms. Trace:
I20260709 12:25:02.188632  5147 rpcz_store.cc:276] 0709 12:25:00.639116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.150822 (+1511706us) service_pool.cc:224] Handling call
0709 12:25:02.169007 (+ 18185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.187478 (+ 18471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.187831 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:02.172813  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.191113  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.177963  5150 rpcz_store.cc:276] 0709 12:25:00.641953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.165787 (+1523834us) service_pool.cc:224] Handling call
0709 12:25:02.166156 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.168968 (+  2812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.169246 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.180994  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.182725  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.192849  5223 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:02.183504  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.151268  9675 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:02.195495  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:02.131748  9644 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.159618  9689 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.185493  9197 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:25:02.185019  9645 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:25:02.185583  9122 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.026367  9676 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:02.223090  9676 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:02.195978  9648 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:02.234479  9628 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:02.235296  9628 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:02.236825  9552 raft_consensus.cc:3055] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:02.237493  9552 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:02.194566  9626 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:02.238037  9628 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:02.238675  9552 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:02.240085  9626 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:02.240435  9552 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:02.243803  9690 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:25:02.246526  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 1042) took 1696 ms. Trace:
I20260709 12:25:02.247076  5943 rpcz_store.cc:276] 0709 12:25:00.550025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.660478 (+110453us) service_pool.cc:224] Handling call
0709 12:25:00.660877 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.741807 (+ 80930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.246159 (+1504352us) spinlock_profiling.cc:243] Waited 59.2 ms on lock 0x7b540000c654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:02.246431 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136178432}
I20260709 12:25:02.248850  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:02.174391  9439 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:02.255234  9439 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:02.256930  9439 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:02.261011  9356 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.261837  9356 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.174904  9687 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:02.206698  9483 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:02.207917  9685 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 12:25:02.208992  9680 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 12:25:02.209400  9693 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:25:02.214264  9613 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:02.214737  9679 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.216507  9271 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.218359  9674 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:02.219151  9197 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:02.204842  9557 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:02.221685  9122 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.270396  9667 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.362478  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1134) took 2327 ms. Trace:
I20260709 12:25:02.282646  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:02.257092  5146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 871) took 1839 ms (client timeout 1658 ms). Trace:
W20260709 12:25:02.274945  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.366094  5947 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:02.380736  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1519) took 1187 ms. Trace:
I20260709 12:25:02.338179  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1271) took 2720 ms. Trace:
I20260709 12:25:02.338691  5149 raft_consensus.cc:1275] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:02.339253  5144 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:25:02.366410  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.339293  5943 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:02.393978  5221 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:02.394309  5220 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:02.394661  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:02.340389  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.395396  5942 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:02.395610  5222 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:02.346410  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:02.395862  5943 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:02.383858  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:02.344452  5141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1283) took 1805 ms (client timeout 1959 ms). Trace:
I20260709 12:25:02.349874  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:02.364519  5939 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 1125) took 2345 ms (client timeout 2631 ms). Trace:
I20260709 12:25:02.347335  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:02.393129  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.398607  5219 raft_consensus.cc:2410] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:02.394879  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.345878  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:02.339458  5148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1403) took 3125 ms (client timeout 2025 ms). Trace:
I20260709 12:25:02.390506  5941 rpcz_store.cc:276] 0709 12:25:00.035234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.233484 (+198250us) service_pool.cc:224] Handling call
0709 12:25:00.243984 (+ 10500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.616557 (+372573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.362187 (+1745630us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54001536d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:02.362384 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":855055232}
I20260709 12:25:02.396960  5067 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:25:02.390798  5146 rpcz_store.cc:269] 0709 12:25:00.418049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.483292 (+ 65243us) service_pool.cc:224] Handling call
0709 12:25:02.256818 (+1773526us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b540019f854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:02.257000 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":824791296}
W20260709 12:25:02.400008  5148 rpcz_store.cc:269] 0709 12:24:59.213662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.500168 (+286506us) service_pool.cc:224] Handling call
0709 12:25:02.278169 (+2778001us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54000e2254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:02.339356 (+ 61187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1127770368}
I20260709 12:25:02.363823  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1511) took 2024 ms. Trace:
I20260709 12:25:02.392325  5143 rpcz_store.cc:276] 0709 12:24:59.617248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.173968 (+556720us) service_pool.cc:224] Handling call
0709 12:25:00.174292 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.256024 (+ 81732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.275419 (+2019395us) spinlock_profiling.cc:243] Waited 81 ms on lock 0x7b54007aadd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:02.338063 (+ 62644us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186272128}
I20260709 12:25:02.402621  5076 rpcz_store.cc:276] 0709 12:25:00.339200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.339505 (+   305us) service_pool.cc:224] Handling call
0709 12:25:00.361095 (+ 21590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.889746 (+528651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.363550 (+1473804us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b5400219b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:02.363754 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1157911296}
I20260709 12:25:02.404093  5068 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:25:02.398408  5939 rpcz_store.cc:269] 0709 12:25:00.019329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.023882 (+  4553us) service_pool.cc:224] Handling call
0709 12:25:02.364288 (+2340406us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b5400004454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:02.364431 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1670980864}
I20260709 12:25:02.404953  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.406229  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:02.391948  5072 rpcz_store.cc:276] 0709 12:25:01.192986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.196446 (+  3460us) service_pool.cc:224] Handling call
0709 12:25:01.196832 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:01.197225 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:01.197258 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:01.197796 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:25:01.687133 (+489337us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:01.687183 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.687282 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.380415 (+693133us) raft_consensus.cc:1685] finished
0709 12:25:02.380440 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.380625 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13543,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":12400,"thread_start_us":488876,"threads_started":1,"wal-append.queue_time_us":630782}
I20260709 12:25:02.408196  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:02.407790  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:02.397022  5141 rpcz_store.cc:269] 0709 12:25:00.539138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.554119 (+1014981us) service_pool.cc:224] Handling call
0709 12:25:02.344224 (+790105us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b540036f454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:02.344369 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373943040}
I20260709 12:25:02.407836  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:02.361197  9416 consensus_meta.cc:287] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.001s
I20260709 12:25:02.409763  5939 raft_consensus.cc:2376] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12. Re-sending same reply.
I20260709 12:25:02.414543  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.409271  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.390486  9643 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:02.383934  9661 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 1.259s	user 0.020s	sys 0.026s
I20260709 12:25:02.409017  5941 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:02.348953  9607 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.384902  5042 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:02.411407  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:02.384135  9197 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.412150  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:02.349898  9618 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.358178  9642 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.339159  9356 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.416450  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:02.433132  5071 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
W20260709 12:25:02.437655  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52864.8us
I20260709 12:25:02.350147  9648 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:02.350018  9579 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.413864  9567 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.361936  9644 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:02.432039  9613 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.431754  9271 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.430117  9557 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.337920  9538 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.455341  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:02.359297  9581 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:02.382145  9679 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:02.349763  9474 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:02.416728  9540 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:25:02.447108  9416 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:02.408527  9667 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:02.384006  9687 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:02.446278  9197 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.446082  9219 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:02.583287  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 65085.4us
I20260709 12:25:02.450814  9618 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:02.438405  9643 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:02.569957  9661 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:25:02.447925  9352 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.132s
I20260709 12:25:02.461397  9215 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:02.644120  5221 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:02.488587  9567 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:02.704236  9352 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.704849  9352 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:02.705694  9567 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:02.706261  9567 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.706677  9567 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.707140  9352 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:02.461120  9693 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.239s
I20260709 12:25:02.709118  9499 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:02.709688  9499 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:02.698081  9671 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:25:02.651793  5215 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 11: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:25:02.652225  9319 consensus_meta.cc:287] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.788s	user 0.012s	sys 0.000s
I20260709 12:25:02.487818  9579 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:02.659516  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.459185  9555 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:02.692454  5942 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:02.445492  9607 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:02.578591  9644 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.695196  9219 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:02.465091  9482 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.227s
I20260709 12:25:02.659528  5945 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:02.717700  5900 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.008s	sys 0.023s
I20260709 12:25:02.731956  9215 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:02.721076  5073 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.073s	sys 0.152s
W20260709 12:25:02.728708  5075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.018s	sys 0.015s
W20260709 12:25:02.771335  5073 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.073s	sys 0.152s
W20260709 12:25:02.771517  5075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.019s	sys 0.015s
I20260709 12:25:02.729948  5224 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:25:02.654897  9581 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:02.656327  9679 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:02.653126  9538 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.466208  9648 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:02.732575  5040 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.011s	sys 0.020s
W20260709 12:25:02.785857  5040 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.012s	sys 0.020s
I20260709 12:25:02.695806  9416 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:02.650373  9678 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:02.578641  9271 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:02.604555  9474 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:02.769586  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.770804  5900 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.008s	sys 0.023s
W20260709 12:25:02.718206  5041 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.023s	sys 0.036s
I20260709 12:25:02.701392  9483 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:02.582980  9557 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:02.794265  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.807003  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.804180  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 678957us
W20260709 12:25:02.804373  5041 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.023s	sys 0.036s
I20260709 12:25:02.615801  9613 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:02.782608  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.777005  9571 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.041s	sys 0.056s
I20260709 12:25:02.728819  9555 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.770398  9219 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:02.826683  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.827198  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 1047) took 1214 ms. Trace:
W20260709 12:25:02.806003  9565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.676s	user 0.008s	sys 0.049s
I20260709 12:25:02.816972  5900 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:25:02.820762  9571 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.042s	sys 0.056s
I20260709 12:25:02.768220  9644 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:02.791597  9284 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:02.790638  9567 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.795994  9678 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:02.830806  9693 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:02.830893  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 369502us
I20260709 12:25:02.902771  9693 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:02.836089  5900 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:02.840184  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:02.904026  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 68127.4us
I20260709 12:25:02.831722  5946 rpcz_store.cc:276] 0709 12:25:01.612874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.076756 (+463882us) service_pool.cc:224] Handling call
0709 12:25:02.077241 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.077523 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:02.443432 (+365909us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.826777 (+383345us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.826852 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:02.826907 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.826997 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.827134 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":673793,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":6988,"raft.queue_time_us":265,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":1779,"thread_start_us":709626,"threads_started":2}
I20260709 12:25:02.904846  5900 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:02.905177  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:02.905776  5900 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:02.906473  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:02.839394  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 383032us
I20260709 12:25:02.907513  9626 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:02.908545  9642 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:02.909469  9439 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:02.834470  9552 consensus_meta.cc:287] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.593s	user 0.012s	sys 0.006s
W20260709 12:25:02.830998  9565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.702s	user 0.008s	sys 0.049s
I20260709 12:25:02.933398  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.829391  9555 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:02.864135  9557 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:02.865084  5075 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1175) took 1408 ms. Trace:
I20260709 12:25:02.830346  9413 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.784945  9499 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:02.861919  9699 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:02.963778  9706 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:25:02.874701  9416 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:02.969043  9687 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:02.970844  9644 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:02.935474  9667 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:02.933434  9565 raft_consensus.cc:697] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:02.954820  9552 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.974236  9613 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:02.974049  9552 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.950412  9688 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:02.975514  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:02.976343  5068 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:25:02.976127  9688 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.976500  9552 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.975559  9648 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.938723  9482 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:02.955456  9618 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:02.977994  9482 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:02.978848  9474 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.979398  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:02.816301  9483 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.832043  9702 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:02.842226  9284 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:02.884311  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:02.868397  9122 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:02.892972  5071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 951) took 1216 ms. Trace:
I20260709 12:25:02.986997  5945 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:25:02.819531  9540 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.904062  5942 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:02.845763  5939 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:02.916381  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:02.920171  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:02.924067  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:02.950445  5075 rpcz_store.cc:276] 0709 12:25:01.456612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.994907 (+538295us) service_pool.cc:224] Handling call
0709 12:25:02.020859 (+ 25952us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.033086 (+ 12227us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:02.033142 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.047872 (+ 14730us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.772197 (+724325us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:02.772274 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.772394 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.864868 (+ 92474us) raft_consensus.cc:1685] finished
0709 12:25:02.864890 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.865022 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11677,"raft.queue_time_us":60917,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":93,"thread_start_us":643040,"threads_started":1,"wal-append.queue_time_us":719667}
I20260709 12:25:02.953521  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:02.996500  5076 raft_consensus.cc:2410] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:25:02.865767  9542 raft_consensus.cc:3055] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:02.993383  5939 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:02.973925  9565 consensus_queue.cc:237] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:02.977092  9574 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:02.877133  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:02.842546  9632 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:03.020237  5941 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:25:02.879532  9689 consensus_meta.cc:287] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.645s	user 0.004s	sys 0.009s
I20260709 12:25:03.045403  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 1052) took 1381 ms. Trace:
W20260709 12:25:02.990056  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.049790  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:02.979451  9548 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.873834  9215 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:02.980784  5068 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:25:02.986793  5221 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:25:02.985826  5071 rpcz_store.cc:276] 0709 12:25:01.676922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.777505 (+100583us) service_pool.cc:224] Handling call
0709 12:25:02.892744 (+1115239us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b5400162954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:02.892897 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463608192}
I20260709 12:25:02.936837  9693 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:02.991791  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 164280us
I20260709 12:25:02.978603  9688 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:02.963781  9635 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 2.632s	user 0.003s	sys 0.025s
I20260709 12:25:02.999164  9499 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:03.051209  5943 rpcz_store.cc:276] 0709 12:25:01.664155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.434395 (+770240us) service_pool.cc:224] Handling call
0709 12:25:02.434779 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.435070 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:02.881483 (+446413us) log.cc:844] Serialized 14 byte log entry
0709 12:25:03.044920 (+163437us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:03.044988 (+    68us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:03.045044 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.045150 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.045322 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":368673,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"thread_start_us":595721,"threads_started":2}
I20260709 12:25:03.096719  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:02.960898  9413 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:02.990650  9579 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:02.988224  5945 pending_rounds.cc:85] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 8
I20260709 12:25:03.020933  9632 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.094213  5114 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:03.104441  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:02.999593  9542 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:03.107507  9542 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:03.109810  9474 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:03.110433  9474 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.110908  9474 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:03.111531  5115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.001s	sys 0.019s
W20260709 12:25:03.111910  5115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.001s	sys 0.019s
W20260709 12:25:03.112273  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 657025us
I20260709 12:25:03.113288  5115 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:03.118319  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:03.098871  9676 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.873s	user 0.014s	sys 0.008s
I20260709 12:25:02.932739  9678 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:03.102123  5945 pending_rounds.cc:107] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:25:03.125018  5945 raft_consensus.cc:2889] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:03.125113  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 214499us
W20260709 12:25:03.102339  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 198753us
I20260709 12:25:03.091604  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:03.129453  5071 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:25:03.119824  5114 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:03.018443  9540 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:03.126026  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 1048) took 1512 ms. Trace:
I20260709 12:25:03.094758  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:03.131132  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:03.130649  5945 rpcz_store.cc:276] 0709 12:25:01.613154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.077745 (+464591us) service_pool.cc:224] Handling call
0709 12:25:02.078095 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.658903 (+580808us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b58006f59d0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a84970aa 00007fd8a8519447 00007fd8a8531fe7 00007fd8a85337de 00007fd8a853db49 00007fd8a853e416 00007fd8a853ad8f 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4
0709 12:25:03.125649 (+466746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.125942 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627451776}
I20260709 12:25:03.132463  9579 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.985172  9699 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:03.109316  9542 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:03.097126  9439 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:03.138703  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:03.139997  5073 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:03.147536  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:03.148123  5076 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:25:03.148530  5076 pending_rounds.cc:85] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 8
I20260709 12:25:03.148864  5076 pending_rounds.cc:107] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:25:03.149192  5076 raft_consensus.cc:2889] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:03.103863  9643 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:03.117318  9632 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:03.104013  9607 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:03.000597  9702 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:03.141259  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.157801  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:03.128171  9493 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:03.152655  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:03.078675  9689 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:03.091320  9548 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:03.162919  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:03.144722  9641 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:25:03.165606  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:03.182121  5067 raft_consensus.cc:2410] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:03.142848  9618 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:03.136258  9626 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:03.171458  5072 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:03.178696  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:03.182500  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:03.185747  5221 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:03.151700  9710 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:03.188958  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:03.187192  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 92542.1us
I20260709 12:25:03.199276  5067 raft_consensus.cc:2410] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:25:03.199689  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:03.201022  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:03.201990  5222 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:03.202607  5215 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:25:03.200480  5221 raft_consensus.cc:2376] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15. Re-sending same reply.
I20260709 12:25:03.187288  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:03.208823  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.149506  9642 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:03.211253  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1395) took 4210 ms. Trace:
I20260709 12:25:03.213419  5141 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:03.211279  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.119503  9284 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:03.187455  9618 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.146299  9439 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:03.180823  9356 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:03.230686  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.232175  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.196332  9607 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:03.165947  9675 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:25:03.247699  5150 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:03.251322  5216 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:03.218405  5142 rpcz_store.cc:276] 0709 12:24:59.001145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.686858 (+685713us) service_pool.cc:224] Handling call
0709 12:24:59.687253 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.710971 (+ 23718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.711000 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.974071 (+263071us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b2c00193f58. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:24:59.974526 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 12:25:00.176508 (+201982us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:00.227617 (+ 51109us) log.cc:844] Serialized 14 byte log entry
0709 12:25:00.340570 (+112953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.340694 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.963110 (+2622416us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:03.210993 (+247883us) raft_consensus.cc:1685] finished
0709 12:25:03.211020 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.211188 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30013,"prepare.queue_time_us":50815,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":47932,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":16077,"spinlock_wait_cycles":13279616,"thread_start_us":301860,"threads_started":2,"wal-append.queue_time_us":304777}
I20260709 12:25:03.217463  9730 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:03.256807  9540 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:03.257005  9730 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:03.257469  9730 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:03.259101  9699 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:03.218039  9710 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:03.260778  9710 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:03.261298  9728 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.261910  9728 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:03.248407  9729 consensus_queue.cc:1175] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ceec344dd00f40c18ef58414ce147016" responder_term: 13 status { last_received { term: 11 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: [12.8-12.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:03.262812  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1410) took 2858 ms. Trace:
I20260709 12:25:03.263249  5149 rpcz_store.cc:276] 0709 12:25:00.404495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.405075 (+   580us) service_pool.cc:224] Handling call
0709 12:25:02.338268 (+1933193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.393151 (+ 54883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.262536 (+869385us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b54007aadd0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:03.262733 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4408261504}
I20260709 12:25:03.265213  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1140) took 2727 ms. Trace:
I20260709 12:25:03.271052  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:03.270936  5947 rpcz_store.cc:276] 0709 12:25:00.537450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.541441 (+  3991us) service_pool.cc:224] Handling call
0709 12:25:02.365666 (+1824225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.391350 (+ 25684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.264948 (+873598us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b5400004450. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:03.265139 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4166828928}
I20260709 12:25:03.180686  9711 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:25:03.265827  9676 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.266403  9725 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:03.263249  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.239058  9618 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:03.242422  9607 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.282614  5141 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:03.266844  9413 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:03.306533  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.325070  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.312776  5146 raft_consensus.cc:2410] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:03.267891  9312 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:03.239789  9722 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:03.335711  5145 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:03.238379  9567 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:03.324438  9720 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:25:03.250021  9284 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:03.320448  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 193583us
I20260709 12:25:03.323135  5944 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:03.331002  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.332348  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.334758  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.342368  5141 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:03.338743  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:03.317914  5940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 901) took 3418 ms (client timeout 2858 ms). Trace:
I20260709 12:25:03.344764  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.325801  9607 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:03.314949  5041 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:03.345738  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.343146  5149 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
W20260709 12:25:03.343652  5940 rpcz_store.cc:269] 0709 12:24:59.899090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.899305 (+   215us) service_pool.cc:224] Handling call
0709 12:25:03.317589 (+3418284us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400008554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:03.317812 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2543026560}
I20260709 12:25:03.346742  5943 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:03.325333  9483 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:03.348858  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.317211  9676 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:03.252816  9356 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:03.377665  9579 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:03.327781  9312 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:25:03.378212  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.399907  5141 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:03.393991  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:03.419914  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 198957us
W20260709 12:25:03.359122  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.421870  5940 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:25:03.421386  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.421963  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:03.430027  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.432231  5149 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:03.439996  5947 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:03.440845  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.322161  9626 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:03.348263  9724 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 12:25:03.439510  5899 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:03.442875  5217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 868) took 2249 ms (client timeout 1926 ms). Trace:
W20260709 12:25:03.450955  5217 rpcz_store.cc:269] 0709 12:25:01.181223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.183275 (+  2052us) service_pool.cc:224] Handling call
0709 12:25:03.429976 (+2246701us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b540031d8d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:03.430137 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76679936}
I20260709 12:25:03.444978  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.446767  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.446954  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.448423  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.454110  5943 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:03.418773  9714 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:03.462414  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.413406  9747 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:03.465106  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.469961  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.476665  5150 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:25:03.345427  9722 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:03.477923  5941 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
W20260709 12:25:03.448308  9708 log.cc:927] Time spent T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.290s	user 0.001s	sys 0.001s
I20260709 12:25:03.476665  5145 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:03.483407  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.484830  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:03.481031  9571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.012s	sys 0.023s
I20260709 12:25:03.442023  9312 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.486383  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.487259  5943 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:03.489845  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.490556  5942 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:03.490664  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.460639  9677 raft_consensus.cc:3055] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:03.491676  5943 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:03.479736  9748 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:03.391798  9740 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.130s
I20260709 12:25:03.380828  9642 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:03.485716  9571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.012s	sys 0.024s
W20260709 12:25:03.506322  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.480633  9747 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:03.485180  9722 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:03.489704  9548 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:03.518813  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.519600  5146 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:03.493170  9677 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:03.497450  9579 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.500928  9689 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:03.534070  9728 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:03.537187  9356 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:03.538493  9735 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 12:25:03.540097  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.507997  9197 consensus_meta.cc:287] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.810s	user 0.015s	sys 0.001s
I20260709 12:25:03.495004  9748 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.528643  5947 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:03.531141  9677 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:03.512854  9739 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.120s	user 0.002s	sys 0.000s
I20260709 12:25:03.551107  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.552778  5146 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:03.554980  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1185) took 1207 ms. Trace:
I20260709 12:25:03.555560  5074 rpcz_store.cc:276] 0709 12:25:02.347846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.396647 (+ 48801us) service_pool.cc:224] Handling call
0709 12:25:02.396975 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.397412 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:02.397454 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.415920 (+ 18466us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.866266 (+450346us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:02.866330 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.866429 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:03.554660 (+688231us) raft_consensus.cc:1685] finished
0709 12:25:03.554696 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.554897 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":9047,"thread_start_us":407959,"threads_started":1,"wal-append.queue_time_us":635943}
W20260709 12:25:03.558665  9737 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:25:03.486460  9746 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
W20260709 12:25:03.559689  9736 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
I20260709 12:25:03.559435  9474 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:03.480656  9714 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:03.550165  9748 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:03.553874  9197 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:03.542630  9579 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:03.514607  9747 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:03.594547  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.595289  5146 raft_consensus.cc:2376] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14. Re-sending same reply.
W20260709 12:25:03.613775  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.592206  9746 consensus_queue.cc:1243] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:25:03.596422  9555 consensus_meta.cc:287] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.640s	user 0.014s	sys 0.000s
I20260709 12:25:03.590617  9677 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:03.628894  5142 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:03.631076  5146 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 16. Current term is 17. Ops: [14.6-16.7]
I20260709 12:25:03.635797  5115 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:03.643935  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:03.646679  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.656525  9709 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:25:03.657606  9542 consensus_meta.cc:287] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.000s
I20260709 12:25:03.638134  9566 raft_consensus.cc:3055] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:03.658218  9579 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:03.658407  9566 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:03.657616  9709 consensus_queue.cc:1243] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:03.659490  9566 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:03.648844  9605 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:03.640444  9571 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:03.663101  9571 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:03.664007  9571 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:03.665043  9571 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:03.668567  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.669335  5217 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:25:03.661325  9566 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.645077  9734 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:03.638425  9555 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:03.682874  5141 raft_consensus.cc:2393] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:03.687810  5115 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:03.688907  9605 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:03.692617  5115 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:03.693599  9605 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:03.694029  9474 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:03.694211  9605 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:03.695324  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.695160  9605 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:03.695405  9197 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:03.697633  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.698911  5146 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:03.699075  5150 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:03.702373  9555 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:03.706775  9352 raft_consensus.cc:3055] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:03.707548  9352 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:03.709686  9219 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.871s	user 0.010s	sys 0.005s
I20260709 12:25:03.710299  9219 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:03.720263  5150 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:25:03.723189  9632 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.004s
I20260709 12:25:03.723739  9632 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:03.724996  9714 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:03.725759  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1146) took 2122 ms. Trace:
W20260709 12:25:03.726179  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.729864  9730 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:03.732430  9730 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:03.733767  5148 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:03.733424  5188 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:03.733306  9284 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:03.735195  9284 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:03.735548  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.736167  5945 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:25:03.736413  9714 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:03.737691  5946 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:25:03.710927  9352 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:03.726228  5944 rpcz_store.cc:276] 0709 12:25:01.603199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.794375 (+191176us) service_pool.cc:224] Handling call
0709 12:25:03.322851 (+1528476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:03.348619 (+ 25768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.725511 (+376892us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400008550. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:03.725695 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3476709248}
I20260709 12:25:03.713115  9618 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:03.754347  5216 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:03.754813  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:03.744270  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:03.739631  9352 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:03.735716  9704 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.895s	user 0.002s	sys 0.000s
I20260709 12:25:03.700306  9312 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:03.748111  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 855) took 4703 ms. Trace:
I20260709 12:25:03.741914  5041 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:03.757061  5144 rpcz_store.cc:276] 0709 12:24:59.044419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.025100 (+980681us) service_pool.cc:224] Handling call
0709 12:25:02.338897 (+2313797us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.393517 (+ 54620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.747902 (+1354385us) spinlock_profiling.cc:243] Waited 2.31 s on lock 0x7b54000e2250. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:03.748049 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5320313728}
I20260709 12:25:03.758924  5150 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: [12.7-13.8]
I20260709 12:25:03.761072  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:03.761775  5041 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:03.766335  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:03.766830  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:03.767892  5223 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:25:03.768817  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:03.768144  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:03.773859  5216 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:03.773401  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:03.762480  9658 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:03.776595  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.777403  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:03.777352  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:03.783030  5215 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 13: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:03.784444  9542 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:25:03.792870  9312 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:03.794189  9702 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:03.781647  9565 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:03.766479  9153 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 12:25:03.795641  9678 consensus_meta.cc:287] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.652s	user 0.014s	sys 0.000s
I20260709 12:25:03.779057  9483 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:03.796157  9678 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:03.802096  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.803448  9271 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:03.804736  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 1180) took 1802 ms. Trace:
I20260709 12:25:03.805138  5069 rpcz_store.cc:276] 0709 12:25:02.002524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.012592 (+ 10068us) service_pool.cc:224] Handling call
0709 12:25:03.804537 (+1791945us) spinlock_profiling.cc:243] Waited 51.4 ms on lock 0x7b540047d954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:03.804673 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118286720}
I20260709 12:25:03.796172  9483 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:03.806699  9483 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:03.802403  9632 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:03.772111  9762 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.811280  9762 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:03.813788  9219 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:03.784204  9658 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.814898  9658 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.815328  9678 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:03.818953  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 45624.2us
I20260709 12:25:03.819965  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:03.783646  9607 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:03.827554  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.827960  9499 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:03.828368  9499 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:03.828756  9499 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:03.829005  9607 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:03.829919  5143 raft_consensus.cc:2368] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16.
I20260709 12:25:03.832614  9734 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:03.832835  9482 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:03.835471  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.836318  5144 raft_consensus.cc:2368] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13.
W20260709 12:25:03.836723  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.837396  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.838068  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.839510  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.843466  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.846974  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.847970  5144 raft_consensus.cc:2376] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12. Re-sending same reply.
W20260709 12:25:03.854362  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.857239  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.860847  5150 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:03.858840  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.862716  5146 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:25:03.866699  5948 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:03.868423  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.870409  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:03.871114  5944 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:03.873032  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:03.852262  9762 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:03.878437  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.860692  9532 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:03.876685  9674 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.898532  5944 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:03.900413  9532 raft_consensus.cc:697] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:03.905097  9618 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:03.906100  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:03.909657  5141 raft_consensus.cc:2376] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11. Re-sending same reply.
I20260709 12:25:03.875425  9748 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:03.913722  9766 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:03.914350  9766 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.914683  9766 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:03.918035  5946 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:25:03.918324  5942 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:03.919862  9748 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:03.918541  5946 pending_rounds.cc:85] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 7
I20260709 12:25:03.921873  5946 pending_rounds.cc:107] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:25:03.922117  5946 raft_consensus.cc:2889] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:03.922659  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:03.921895  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.924008  5146 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:03.931039  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:03.932197  5070 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 12:25:03.932804  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.933204  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:03.934482  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:03.934980  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:03.935204  5070 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:03.934716  5899 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:03.935438  5071 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:03.940223  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:03.940709  5069 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:03.942004  9419 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:03.942301  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 1221) took 1552 ms. Trace:
I20260709 12:25:03.942533  9419 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:03.942727  5224 rpcz_store.cc:276] 0709 12:25:02.389634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.404532 (+ 14898us) service_pool.cc:224] Handling call
0709 12:25:03.942112 (+1537580us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b5400163d54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:03.942239 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":744617216}
I20260709 12:25:03.943185  9419 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:03.907409  9674 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:03.945717  9658 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:03.947942  9674 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:03.948489  9658 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:03.938650  9539 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:25:03.949172  5939 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:03.949506  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:03.953893  9532 consensus_queue.cc:237] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:03.954437  5224 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:03.954377  5939 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:03.949177  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:03.961447  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.960799  5041 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:03.966207  5041 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:03.966889  5188 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:03.968829  5067 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:03.976064  5939 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:25:03.976382  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:03.976691  5939 pending_rounds.cc:85] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 9
I20260709 12:25:03.975848  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:03.976959  5939 pending_rounds.cc:107] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:25:03.976900  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:03.977938  5939 raft_consensus.cc:2889] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:03.978001  5114 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:03.978992  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 1058) took 1134 ms. Trace:
I20260709 12:25:03.979382  5939 rpcz_store.cc:276] 0709 12:25:02.844805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.845054 (+   249us) service_pool.cc:224] Handling call
0709 12:25:02.845491 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:03.948798 (+1103307us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b4c00091780. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6807a20 00007fd8a6818a29 00007fd8a67de644 00007fd8a6805f2d 00007fd8a6805b93 00007fd8a853618b 00007fd8a8536cb5 00007fd8a8531f30 00007fd8a85337de 00007fd8a853db49 00007fd8a853e416
0709 12:25:03.978727 (+ 29929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.978937 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15134080}
I20260709 12:25:03.979833  5114 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:03.984293  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:03.985013  5141 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:25:03.985319  5219 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:03.989981  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:03.990897  5219 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.929004  9747 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:03.997471  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:04.001010  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:04.007654  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:04.008047  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1448) took 1115 ms. Trace:
I20260709 12:25:04.005709  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:04.003095  5224 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:25:04.011816  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:03.959502  9548 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:04.019140  5190 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:04.008674  5220 rpcz_store.cc:276] 0709 12:25:02.892851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.893006 (+   155us) service_pool.cc:224] Handling call
0709 12:25:02.893565 (+   559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:03.163665 (+270100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:03.163713 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:03.169402 (+  5689us) log.cc:844] Serialized 26 byte log entry
0709 12:25:03.428604 (+259202us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:03.428693 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.986838 (+558145us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54008502d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:03.986892 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:03.986939 (+    47us) raft_consensus.cc:1685] finished
0709 12:25:03.986953 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.007927 (+ 20974us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2315,"raft.queue_time_us":2772,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":6268,"spinlock_wait_cycles":614622080,"thread_start_us":217856,"threads_started":1,"wal-append.queue_time_us":372253}
W20260709 12:25:04.001864  9728 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 12:25:03.969069  9352 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.143651  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.144454  5219 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
W20260709 12:25:04.144201  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 139695us
I20260709 12:25:03.983853  9569 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:04.038738  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:04.153060  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:04.154245  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:04.154482  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:04.154989  5067 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 8.
I20260709 12:25:04.155792  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:03.995086  9678 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:04.151288  5146 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:04.000120  9650 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:04.002707  9734 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:04.019896  9215 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:04.157194  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 138132us
I20260709 12:25:04.003396  9705 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:04.153088  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 133892us
W20260709 12:25:04.186753  5115 connection.cc:256] RPC call timeout handler was delayed by -1.3672491710003669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
I20260709 12:25:04.163374  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:04.188699  5216 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:04.142957  9548 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:03.996371  9153 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:04.168596  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:04.190050  5219 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:04.191038  5222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1445) took 1301 ms. Trace:
I20260709 12:25:04.191632  5222 rpcz_store.cc:276] 0709 12:25:02.889503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.889685 (+   182us) service_pool.cc:224] Handling call
0709 12:25:02.890044 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:03.202412 (+312368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.190718 (+988306us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b540012a7d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:04.190958 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716870528}
I20260709 12:25:04.156772  9730 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.006157  9671 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.174s
I20260709 12:25:04.176658  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:04.006462  9542 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.172s
I20260709 12:25:04.201683  5219 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:04.179414  9650 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.180986  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:04.181672  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:04.152047  9679 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.318s	user 0.029s	sys 0.052s
W20260709 12:25:04.208364  9679 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.376s	user 0.030s	sys 0.052s
I20260709 12:25:04.165668  9352 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:03.995626  9493 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.162s
I20260709 12:25:04.143105  9565 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:04.003077  9219 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.169s
I20260709 12:25:04.189499  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:04.187884  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:04.213182  5223 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:04.197870  9548 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:04.217978  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.219771  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.170256  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:04.157123  9569 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.005939  9539 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.140239  9571 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.156402  9675 raft_consensus.cc:2764] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:25:04.006327  9729 consensus_queue.cc:1048] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.174s
I20260709 12:25:04.174759  9284 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:04.180016  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:04.207650  9650 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:04.178568  9734 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:04.186167  9705 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:04.247627  9705 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:04.249130  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:04.211282  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:04.211671  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:04.179041  9483 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:04.174463  9499 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:04.259102  5145 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:04.229290  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:04.260414  5223 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:04.227458  9271 raft_consensus.cc:2764] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:25:04.230324  9352 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:04.230571  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.272836  5218 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:04.230581  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:04.274798  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:04.275450  5074 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:04.276291  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:04.234827  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 54892.7us
I20260709 12:25:04.277237  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:04.237258  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.238875  5070 raft_consensus.cc:1217] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Deduplicated request from leader. Original: 10.7->[11.8-13.9]   Dedup: 11.8->[13.9-13.9]
I20260709 12:25:04.279475  5070 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 12:25:04.280457  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:04.281193  5070 raft_consensus.cc:2410] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:25:04.241770  5149 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:25:04.242666  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:04.204402  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:25:04.282433  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 78160.5us
I20260709 12:25:04.244458  5142 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:04.251281  5067 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:25:04.284883  5190 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:04.250451  9734 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:04.294401  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:04.295187  9499 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:04.259840  9493 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.297911  9493 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:04.301851  9676 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:04.212932  9565 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:04.305965  5073 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:04.203738  9730 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:04.266597  9729 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:04.226930  9569 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.228026  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:04.200193  9153 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:04.236857  9284 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:04.286214  5141 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:04.257507  5219 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:04.286691  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:04.271785  9215 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.306349  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.297955  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:04.305168  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:04.308040  5074 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:25:04.238458  9571 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:04.308439  5041 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:04.297436  9483 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:04.312204  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:04.262101  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:04.346685  5074 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:04.327597  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:04.329290  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:04.235848  9539 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:04.286230  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:04.349429  5147 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:04.338157  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:04.350196  5219 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:04.352267  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.353716  5074 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:04.355463  5074 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:04.340909  5148 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:04.308691  5067 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:25:04.356384  5219 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:04.315085  9776 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:04.343389  5223 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:04.331034  9729 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:04.340368  5145 raft_consensus.cc:2376] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16. Re-sending same reply.
I20260709 12:25:04.347025  5076 raft_consensus.cc:1217] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Deduplicated request from leader. Original: 14.8->[15.9-15.9]   Dedup: 15.9->[]
I20260709 12:25:04.348322  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:04.348357  5149 raft_consensus.cc:2376] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17. Re-sending same reply.
I20260709 12:25:04.366734  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.314373  9352 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:04.325996  9542 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260709 12:25:04.340301  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.362376  5070 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:04.375890  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:04.330729  9671 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:04.379135  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:04.317138  9565 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:04.381088  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:04.381250  5115 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:04.381747  5141 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.336302  9284 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:04.367715  5068 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:04.368099  5076 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:25:04.352905  5146 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:04.375049  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:04.367354  5149 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:04.353470  9651 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:04.379930  5145 raft_consensus.cc:2376] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15. Re-sending same reply.
I20260709 12:25:04.384369  5223 raft_consensus.cc:2368] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13.
I20260709 12:25:04.386448  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:04.386348  5142 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:04.387753  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:04.390622  5041 leader_election.cc:400] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13.
I20260709 12:25:04.390969  5041 leader_election.cc:403] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:25:04.392431  9585 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:04.394443  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.369606  9729 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:04.347434  9215 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:04.397612  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:04.370998  9747 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:04.399390  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.401593  5146 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:25:04.376152  9709 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:04.400578  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.397269  5148 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
W20260709 12:25:04.410359  9771 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:25:04.421656  9542 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:04.384881  9605 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:04.422219  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.423098  9539 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:04.423417  9772 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:25:04.360715  9702 consensus_queue.cc:1059] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:04.390126  9762 consensus_queue.cc:1059] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:25:04.427234  5148 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 12:25:04.483920  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.424582  5145 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
W20260709 12:25:04.485509  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 97849.5us
I20260709 12:25:04.488745  5146 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:04.488986  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 204149us
I20260709 12:25:04.395694  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:04.484612  9499 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:04.497896  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:04.498656  5149 raft_consensus.cc:2410] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:04.507025  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 889) took 1040 ms. Trace:
I20260709 12:25:04.519202  5217 rpcz_store.cc:276] 0709 12:25:03.467196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.470739 (+  3543us) service_pool.cc:224] Handling call
0709 12:25:03.489434 (+ 18695us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:03.669906 (+180472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.506716 (+836810us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54001437d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:04.506953 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413050624}
I20260709 12:25:04.518422  5076 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:25:04.523939  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:04.525770  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:04.534188  5141 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
W20260709 12:25:04.534489  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:04.535364  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:04.535979  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:04.536651  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.538920  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:04.541887  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:04.548043  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:04.549151  5941 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
W20260709 12:25:04.511436  9770 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.286s	user 0.002s	sys 0.000s
W20260709 12:25:04.554868  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.555873  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:04.487313  9542 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:04.553194  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.553782  5148 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:25:04.560931  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.489125  9730 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:04.489614  9709 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:04.566855  9709 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:04.567241  9605 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:04.567488  5142 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:04.567909  9605 raft_consensus.cc:697] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:04.569056  5941 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:04.567317  9709 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.571009  9709 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:04.574944  9215 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:04.575707  9419 consensus_meta.cc:287] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.632s	user 0.007s	sys 0.005s
I20260709 12:25:04.576221  9419 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:04.582675  9766 consensus_meta.cc:287] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.004s
I20260709 12:25:04.582876  9419 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:04.583384  9766 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:04.512204  9352 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:04.552363  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 617662us
I20260709 12:25:04.586858  9153 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:04.587044  9352 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:04.589411  9782 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:25:04.594015  9671 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.594858  9671 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:04.601677  9574 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.602722  5072 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:04.602562  9574 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:04.606449  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 92925.8us
I20260709 12:25:04.558481  5146 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:04.495733  9585 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:04.514276  9651 raft_consensus.cc:697] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:04.557359  9779 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:25:04.616231  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:04.617100  5939 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:04.563419  9571 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:04.610857  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:04.620951  5068 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:04.614269  5071 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:25:04.611945  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:25:04.621021  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 89177.5us
I20260709 12:25:04.619366  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:04.622265  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:04.623145  5939 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:04.628939  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:04.632018  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:04.633272  5114 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:04.630555  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:04.606758  9674 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.638320  5941 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15. Ops: [12.5-14.6]
I20260709 12:25:04.634775  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:04.627631  9671 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:04.628278  9740 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.656452  5944 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:04.656291  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:04.620239  9776 raft_consensus.cc:697] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:04.665725  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:04.665823  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 140140us
I20260709 12:25:04.634649  9532 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:25:04.669682  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 113930us
I20260709 12:25:04.671026  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:04.673333  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 87136.4us
I20260709 12:25:04.626003  9585 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:04.674947  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:04.632033  9738 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:04.627135  9709 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:04.672722  9748 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:04.681381  9650 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:04.672952  9740 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:04.684001  9722 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:04.678066  9787 raft_consensus.cc:3055] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:04.675055  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:04.721747  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 46782.7us
I20260709 12:25:04.722741  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:04.656940  9679 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:04.699226  5219 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:25:04.723692  5219 pending_rounds.cc:85] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 7
I20260709 12:25:04.724025  5219 pending_rounds.cc:107] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:25:04.724282  5219 raft_consensus.cc:2889] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:04.695492  5224 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:04.699160  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:04.727200  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:04.569011  9605 consensus_queue.cc:237] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:04.716912  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:04.737224  5216 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:04.740821  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:04.741623  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:04.744340  5898 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:04.717850  5900 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:04.726580  5941 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:25:04.745393  5941 pending_rounds.cc:85] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 6
I20260709 12:25:04.745800  5941 pending_rounds.cc:107] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:25:04.746169  5941 raft_consensus.cc:2889] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:04.687143  9651 consensus_queue.cc:237] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:04.732611  5222 raft_consensus.cc:2393] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:04.674304  9574 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:04.651257  9565 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:04.673076  9566 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:04.645268  9674 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:04.755595  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:04.778909  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:04.771260  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 53533.9us
I20260709 12:25:04.775409  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:04.781386  5900 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:04.772406  5067 raft_consensus.cc:2393] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:04.670789  9734 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:04.732311  9724 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.748251  9762 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:04.663341  9542 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:04.793148  9705 consensus_meta.cc:287] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.003s
I20260709 12:25:04.749214  9738 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:04.734050  9679 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.754652  9122 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:04.734724  9787 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:04.800820  5218 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:04.732637  9581 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:04.794524  5222 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:25:04.762562  9629 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:04.762310  9436 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:04.687779  9532 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:04.807819  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 63686.3us
I20260709 12:25:04.798710  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:04.774915  9766 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:04.784322  9626 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:04.788791  9566 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:04.845487  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 46871.9us
I20260709 12:25:04.846908  9566 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:04.847467  5900 leader_election.cc:400] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16.
I20260709 12:25:04.847884  9566 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:04.849228  5943 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
W20260709 12:25:04.806357  9784 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:25:04.824956  5222 pending_rounds.cc:85] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 9
I20260709 12:25:04.819743  5216 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:04.800608  9574 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:04.850540  5222 pending_rounds.cc:107] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:25:04.776886  9783 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:25:04.850904  5222 raft_consensus.cc:2889] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:04.849378  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:04.856204  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:04.860575  9574 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:04.813455  9676 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:04.866539  9793 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:04.806468  9776 consensus_queue.cc:237] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:04.824451  9122 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:04.824469  9679 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:04.822962  9762 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:04.833719  9581 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:04.818569  9705 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:04.816308  9779 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:25:04.866472  9574 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:04.869597  9566 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.806970  9724 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:04.881278  9762 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.879664  9499 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13.
I20260709 12:25:04.968101  9532 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:04.893121  9566 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:04.872098  9642 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:25:04.960021  5215 raft_consensus.cc:1240] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:04.890931  9585 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:04.810148  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 139205us
I20260709 12:25:04.877810  9122 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:04.900144  9569 consensus_meta.cc:287] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.002s
W20260709 12:25:04.969312  9548 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.666s	user 0.013s	sys 0.000s
W20260709 12:25:04.960942  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 236578us
I20260709 12:25:04.869267  9626 consensus_queue.cc:1059] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:25:04.885634  9581 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:04.981493  9626 raft_consensus.cc:3055] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:04.982115  9626 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:04.983882  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:04.979272  9548 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:04.988355  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:04.988422  9585 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:04.989804  9548 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:04.979136  9569 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:04.997020  9566 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:04.971992  9565 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:04.981289  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:05.002426  9569 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:04.873692  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:04.989683  9585 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:05.006904  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 133179us
I20260709 12:25:05.008630  9585 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:05.008960  9574 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:05.009668  9574 raft_consensus.cc:697] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:05.010232  9585 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:05.010931  9585 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:05.011382  9574 consensus_queue.cc:237] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:05.012495  9585 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:04.984421  9626 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:05.015693  9642 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:05.016932  9626 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:05.018055  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:05.026149  9565 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:05.030042  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.899610  9729 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:05.042595  9565 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:04.903424  9709 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:05.066161  9798 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:04.897395  9762 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.914705  9734 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:04.899267  9738 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:04.909044  9740 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:04.889341  9779 consensus_queue.cc:1243] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 15.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:04.890110  9787 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:05.100980  9787 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:05.034482  9674 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:04.878971  9793 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:05.103708  9793 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:05.053599  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.062763  9729 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:05.064189  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 55237.6us
W20260709 12:25:05.065006  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 75452.1us
I20260709 12:25:05.067950  9798 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:05.069636  9783 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:05.069274  9797 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:05.079145  9705 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:05.079939  9790 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:05.087738  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 82358.4us
I20260709 12:25:05.094494  5223 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:05.096618  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:05.141384  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:05.143020  9806 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:05.144268  9806 raft_consensus.cc:697] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:05.113689  9798 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.149926  9790 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:05.146083  9806 consensus_queue.cc:237] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.128790  5071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1545) took 1164 ms. Trace:
I20260709 12:25:05.120450  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:05.125075  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:05.147979  9493 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:05.141561  5076 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
W20260709 12:25:05.105500  9284 consensus_meta.cc:287] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.709s	user 0.009s	sys 0.016s
I20260709 12:25:05.131294  5900 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:05.125089  9565 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:05.122431  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:05.124001  9724 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:05.170644  9586 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:05.163975  9284 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.172037  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:05.172437  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1224) took 1200 ms. Trace:
W20260709 12:25:05.174194  5070 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.010s	sys 0.018s
I20260709 12:25:05.175088  9439 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.166028  5067 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:05.178915  5215 rpcz_store.cc:276] 0709 12:25:03.971831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.972000 (+   169us) service_pool.cc:224] Handling call
0709 12:25:04.018852 (+ 46852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.993805 (+974953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.171942 (+178137us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54004835d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:05.172343 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162356992}
I20260709 12:25:05.154430  5071 rpcz_store.cc:276] 0709 12:25:03.964114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.964266 (+   152us) service_pool.cc:224] Handling call
0709 12:25:03.964657 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.622046 (+657389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.128070 (+506024us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b5400429354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:05.128561 (+   491us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1482693504}
W20260709 12:25:05.184674  5070 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.011s	sys 0.018s
I20260709 12:25:05.187323  9675 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:05.188694  9774 raft_consensus.cc:3055] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:05.189242  9774 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:05.190799  9774 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:05.193038  9774 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:05.195391  9439 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:05.153856  9790 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:05.203486  9724 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:05.214996  9781 log.cc:927] Time spent T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.795s	user 0.002s	sys 0.000s
I20260709 12:25:05.215466  9439 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:05.205185  5076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:05.201503  9581 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:05.231364  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:05.242751  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 111890us
I20260709 12:25:05.250321  9783 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:05.228456  9419 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:05.233937  5942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 1163) took 1499 ms (client timeout 1850 ms). Trace:
I20260709 12:25:05.256615  9419 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:05.257345  5942 rpcz_store.cc:269] 0709 12:25:03.734419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.735207 (+   788us) service_pool.cc:224] Handling call
0709 12:25:05.233294 (+1498087us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400006254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:05.233625 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":418333952}
I20260709 12:25:05.256386  9783 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.227984  9483 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:25:05.259591  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:05.260463  5942 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:05.263424  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:05.264041  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:05.236254  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:05.265266  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:05.265290  5946 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:05.261214  9783 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:05.268800  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.269265  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:05.271103  5946 raft_consensus.cc:2410] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:25:05.272541  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:05.282471  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 635348us
I20260709 12:25:05.224923  9284 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:05.220556  5076 raft_consensus.cc:606] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:05.286954  5076 raft_consensus.cc:614] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:05.223834  9581 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:05.273519  9650 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:05.288760  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:05.289978  9419 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:05.294142  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:05.283802  9675 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.296502  9675 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.299301  9675 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:05.283802  9585 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.300631  9585 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.302073  9812 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:05.303249  9419 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:05.303579  9585 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:05.307148  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:05.200687  9122 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:05.311569  5946 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:05.311821  9493 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:05.312335  9493 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.312731  9493 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:05.314534  9797 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:05.318699  5942 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:25:05.320036  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:05.318051  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:05.323774  9797 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:05.217523  9762 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:05.334859  5075 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1535) took 1404 ms. Trace:
I20260709 12:25:05.324366  9797 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:05.338016  5075 rpcz_store.cc:276] 0709 12:25:03.930127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.943790 (+ 13663us) service_pool.cc:224] Handling call
0709 12:25:03.944138 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.142632 (+198494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.142669 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.142722 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:04.142757 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.334334 (+1191577us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540047de54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:05.334417 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.334650 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455732992}
I20260709 12:25:05.338730  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1547) took 1373 ms. Trace:
I20260709 12:25:05.339082  5900 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:05.339246  5072 rpcz_store.cc:276] 0709 12:25:03.964811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.965043 (+   232us) service_pool.cc:224] Handling call
0709 12:25:03.965315 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.603314 (+637999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.338243 (+734929us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b5400048654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:05.338561 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1465352064}
I20260709 12:25:05.340831  5900 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:05.341097  9626 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:05.341995  9626 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:05.343195  9626 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:05.343240  9797 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:05.349571  9284 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:05.350208  9284 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.350693  9284 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:05.354825  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1461) took 1041 ms. Trace:
I20260709 12:25:05.355358  5218 rpcz_store.cc:276] 0709 12:25:04.311396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.311581 (+   185us) service_pool.cc:224] Handling call
0709 12:25:04.311949 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.824266 (+512317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.352350 (+528084us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b5400483ad4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:05.352796 (+   446us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123723008}
I20260709 12:25:05.356609  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:05.356925  9581 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:05.357697  9581 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:05.358840  9581 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.361596  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:05.362768  5215 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:05.363682  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:05.343297  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 904) took 1126 ms. Trace:
I20260709 12:25:05.367026  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:05.366977  5067 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:05.369875  5076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:05.366566  5147 rpcz_store.cc:276] 0709 12:25:04.216888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.219423 (+  2535us) service_pool.cc:224] Handling call
0709 12:25:05.342836 (+1123413us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54000e2254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:05.343048 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296349056}
I20260709 12:25:05.348959  9797 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:05.373588  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:05.362103  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:05.375795  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:05.376497  9679 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:05.384140  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:05.385141  5948 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:05.386049  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:05.386426  9679 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:05.386782  9769 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 1.174s	user 0.004s	sys 0.000s
I20260709 12:25:05.389490  9730 raft_consensus.cc:3055] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:05.390050  9730 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:05.390151  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:05.391060  5940 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:05.392063  9745 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.392609  9745 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:05.393327  5218 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:05.391135  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:05.396636  9780 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.975s	user 0.002s	sys 0.005s
I20260709 12:25:05.396905  9650 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:05.372100  9798 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.397962  9798 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.398407  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1307) took 1064 ms. Trace:
I20260709 12:25:05.382622  5222 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:05.377705  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:05.337044  9493 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:05.404876  5219 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:05.398893  5142 rpcz_store.cc:276] 0709 12:25:04.333489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.395459 (+ 61970us) service_pool.cc:224] Handling call
0709 12:25:05.397992 (+1002533us) spinlock_profiling.cc:243] Waited 50.2 ms on lock 0x7b540036e554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:05.398201 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115461504}
I20260709 12:25:05.382442  5076 raft_consensus.cc:606] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:05.412285  5942 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:05.406240  9679 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:05.419528  9728 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:05.421775  5216 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:05.425937  5946 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:05.403108  9122 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:05.433459  5215 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:05.435519  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:05.436221  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:05.440735  9679 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.440321  5041 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:25:05.442940  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 60848.7us
I20260709 12:25:05.434757  9585 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.449800  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:05.391439  9730 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.457683  9811 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 12:25:05.478142  9762 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:05.479796  9798 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:05.480443  9738 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:05.481552  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1459) took 1489 ms. Trace:
I20260709 12:25:05.481895  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:05.481984  5221 rpcz_store.cc:276] 0709 12:25:03.957896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.958052 (+   156us) service_pool.cc:224] Handling call
0709 12:25:03.958357 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:03.958575 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:05.344530 (+1385955us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b180041bea0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a80e34bc 00007fd8a543148b 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8abf333c7 00007fd8a85459a7 00007fd8a85550bf
0709 12:25:05.344747 (+   217us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.403823 (+ 59076us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:05.415352 (+ 11529us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:05.429517 (+ 14165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.435236 (+  5719us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.447420 (+ 12184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5765,"prepare.queue_time_us":187659,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":60,"raft.queue_time_us":19512,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":10964,"spinlock_wait_cycles":6967552,"thread_start_us":174706,"threads_started":2,"wal-append.queue_time_us":92530}
I20260709 12:25:05.484727  9678 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:05.485066  5946 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:05.486079  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:25:05.488989  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 78643.3us
I20260709 12:25:05.489954  5900 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:05.490084  9585 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.491333  9738 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:05.491858  9738 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:05.492513  9585 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:05.492901  9738 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:05.493757  9730 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:05.494417  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:05.495580  5150 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:05.465549  9812 consensus_queue.cc:1059] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:25:05.498407  9678 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:05.498643  9811 raft_consensus.cc:3055] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:05.499190  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:05.499473  9811 raft_consensus.cc:740] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:05.500377  5147 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:05.501201  9811 consensus_queue.cc:260] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:05.501856  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:05.505961  5143 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:25:05.515183  5222 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:05.502806  9811 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:05.502624  9585 consensus_queue.cc:1059] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:05.519444  9790 raft_consensus.cc:3055] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:05.520159  9790 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:05.519830  9565 consensus_queue.cc:1059] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:05.521282  9790 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:05.521930  9705 raft_consensus.cc:3055] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:05.522439  9705 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:05.522902  9790 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:05.523458  9705 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:05.524168  9493 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:05.525027  9705 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:05.533447  9585 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.534158  9679 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:05.534646  9585 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:05.525609  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1464) took 1101 ms. Trace:
I20260709 12:25:05.538700  5223 rpcz_store.cc:276] 0709 12:25:04.424662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.452226 (+ 27564us) service_pool.cc:224] Handling call
0709 12:25:04.482780 (+ 30554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.145305 (+662525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.525278 (+379973us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b54008502d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:05.525524 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1292321536}
I20260709 12:25:05.525960  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1544) took 1569 ms. Trace:
I20260709 12:25:05.537371  9585 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:05.528018  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:05.530236  5076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7be74bdf01fe4190992575e84da14996"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:05.546314  9671 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:05.546862  9671 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.547269  9671 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:05.519419  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:05.531314  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:05.549329  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:05.550241  5944 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:05.550578  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:05.551234  5070 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:05.551354  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:05.552085  5944 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:05.553208  5076 raft_consensus.cc:606] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:05.553809  9799 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:05.553687  5076 raft_consensus.cc:614] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:05.554394  9799 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:05.554914  9799 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:05.558498  5945 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:05.549885  5946 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:05.560993  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:05.561457  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.531834  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:05.532653  9665 consensus_queue.cc:1059] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:05.562309  5070 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:05.562831  9665 raft_consensus.cc:3055] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:05.563306  9665 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:05.564297  9665 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.489960  9499 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:05.565719  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:05.566085  9499 raft_consensus.cc:697] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:05.566402  5947 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:05.567178  9499 consensus_queue.cc:237] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:05.499176  9678 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:05.570029  9678 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:05.517525  5948 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:05.572659  5942 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:05.533010  9419 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:05.577899  9665 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:05.574638  5042 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:05.588665  9762 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:05.595494  5946 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:05.542960  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:05.582908  5948 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:05.597411  5068 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:05.593724  5115 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:05.541054  5069 rpcz_store.cc:276] 0709 12:25:03.955950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.958797 (+  2847us) service_pool.cc:224] Handling call
0709 12:25:03.959251 (+   454us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:03.959857 (+   606us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:03.994030 (+ 34173us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.314544 (+320514us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:04.315553 (+  1009us) log.cc:844] Serialized 52 byte log entry
0709 12:25:04.315785 (+   232us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:04.316181 (+   396us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.316541 (+   360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.316675 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.212750 (+896075us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:05.396084 (+183334us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b240050bd18. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a52d49c4 00007fd8a52d2acb 00007fd8a548ddc9 00007fd8a548da55 00007fd8a548d09d 00007fd8a80bf10e 00007fd8a80d688c 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221
0709 12:25:05.421658 (+ 25574us) spinlock_profiling.cc:243] Waited 900 us on lock 0x7b180044b4e0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:25:05.525430 (+103772us) raft_consensus.cc:1685] finished
0709 12:25:05.525454 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.525745 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8237,"prepare.queue_time_us":185016,"prepare.run_cpu_time_us":185,"prepare.run_wall_time_us":7595,"raft.queue_time_us":618,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":33129,"spinlock_wait_cycles":30897024,"thread_start_us":142460,"threads_started":1,"wal-append.queue_time_us":231280}
I20260709 12:25:05.595624  9581 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:05.627674  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.628728  9787 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:05.628868  5188 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:05.629494  9787 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:05.630133  9675 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:05.630693  9675 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:05.631633  9675 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:05.632161  9787 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:05.628775  9818 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:25:05.635133  5939 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:05.648464  9419 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:05.649674  9419 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:05.650033  9419 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:05.648839  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 117155us
I20260709 12:25:05.654923  9754 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:05.657637  9798 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:05.598785  9811 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:05.676662  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.676911  5945 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:25:05.698256  9352 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:25:05.698807  5940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 918) took 2261 ms (client timeout 1506 ms). Trace:
I20260709 12:25:05.699280  9352 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:05.702896  9542 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:05.706274  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.707294  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:05.708012  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:05.708806  5216 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:05.619056  9806 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.709438  9806 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:05.711143  5223 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:05.710881  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:05.712702  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 256111us
I20260709 12:25:05.713948  5042 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:05.714563  5217 raft_consensus.cc:2393] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:05.715660  9483 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:05.716257  9483 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:05.717329  9483 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:05.711870  9806 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:05.699441  5940 rpcz_store.cc:269] 0709 12:25:03.437374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.439261 (+  1887us) service_pool.cc:224] Handling call
0709 12:25:05.698474 (+2259213us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54002d78d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:05.698679 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4455089408}
I20260709 12:25:05.750640  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:05.749301  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 396970us
I20260709 12:25:05.754101  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 1202) took 1229 ms. Trace:
I20260709 12:25:05.763873  5068 rpcz_store.cc:276] 0709 12:25:04.524550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.124539 (+599989us) service_pool.cc:224] Handling call
0709 12:25:05.753858 (+629319us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b5400589b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:05.754023 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662379392}
I20260709 12:25:05.769053  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:05.759048  9822 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:25:05.774853  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:05.777668  5948 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:05.769063  9806 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.850229  5217 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:25:05.860853  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 107965us
I20260709 12:25:05.862387  9658 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:05.864610  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1537) took 1794 ms. Trace:
I20260709 12:25:05.865203  5073 rpcz_store.cc:276] 0709 12:25:03.930944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.931114 (+   170us) service_pool.cc:224] Handling call
0709 12:25:03.931448 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.306364 (+374916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.599899 (+1293535us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b540037f354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:05.725679 (+125780us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860887424}
W20260709 12:25:05.865816  5041 leader_election.cc:336] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Timed out: RequestConsensusVote RPC to 127.1.123.65:37253 timed out after 1.865s (SENT)
I20260709 12:25:05.867089  9352 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:05.867391  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:05.869272  9811 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:05.869858  9811 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:05.870443  9811 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:05.855093  9820 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:25:05.875083  5041 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:05.879850  9812 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:05.880486  9812 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.881044  9812 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:05.855978  9803 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 12:25:05.889140  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 295521us
I20260709 12:25:05.909942  9806 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:05.912098  9811 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:05.918606  9811 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:05.921741  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 327926us
I20260709 12:25:05.926553  5947 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
W20260709 12:25:05.895884  9284 consensus_meta.cc:287] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.544s	user 0.012s	sys 0.003s
W20260709 12:25:05.934796  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.864642  9658 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:05.928606  9797 consensus_meta.cc:287] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
I20260709 12:25:05.956631  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:05.960310  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:05.962327  5220 raft_consensus.cc:2410] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 12:25:05.968613  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 191204us
I20260709 12:25:05.947657  9284 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:05.949023  9650 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:25:05.971256  9745 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:05.992233  5222 raft_consensus.cc:2368] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18.
I20260709 12:25:05.982734  9499 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:05.994535  9215 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:05.994906  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.994561  9585 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:05.998454  5042 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:06.000759  9585 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:05.980317  9565 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:25:05.995229  9499 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:06.015115  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 67554.3us
I20260709 12:25:06.018324  9811 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:06.023191  5224 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:06.023500  5217 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:06.023553  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:05.993075  9650 consensus_queue.cc:1243] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:25:06.061976  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.996029  9678 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:06.066358  5948 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:06.067817  9678 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:06.069227  9671 consensus_meta.cc:287] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.521s	user 0.006s	sys 0.007s
I20260709 12:25:06.069746  9671 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:06.020499  9658 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:06.076666  9585 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:06.077487  9607 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:06.081624  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:06.082345  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.999522  9215 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:06.081384  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:06.091984  5947 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:06.092943  5946 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:06.093279  5944 consensus_meta.cc:287] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.541s	user 0.015s	sys 0.002s
I20260709 12:25:06.093765  5944 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:06.091701  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.097527  5943 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:25:06.099382  5940 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:06.101106  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:06.101511  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.101891  5142 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
W20260709 12:25:06.102973  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 87061us
I20260709 12:25:06.009373  9799 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:06.105264  5041 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:06.077574  9797 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:06.113108  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 116329us
I20260709 12:25:06.015758  9483 consensus_queue.cc:1059] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 12:25:06.114979  5147 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:06.122911  5215 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:06.123998  5941 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:06.126648  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:06.126748  9419 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:06.127763  5150 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:06.128651  9650 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:06.132579  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:06.134531  9650 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:06.136107  9499 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:06.133402  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:06.128459  9419 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:06.143628  9799 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:06.145589  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:06.147372  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:06.147770  5144 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:06.148164  5146 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:06.152472  5941 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:06.137331  9650 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:06.155357  9812 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:06.163815  9671 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:06.165789  5147 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:06.166800  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:06.171386  5150 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:06.125226  9797 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:06.187668  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:06.191612  9797 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:06.186594  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.193022  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:06.193702  5144 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:06.170994  9483 consensus_queue.cc:1059] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 12:25:06.199162  9705 consensus_meta.cc:287] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.665s	user 0.005s	sys 0.009s
I20260709 12:25:06.212214  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:06.218783  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.219853  5941 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:06.244477  9812 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:06.251916  5144 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:06.252547  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:06.261878  5147 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:06.272739  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.278270  5941 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:06.283028  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.284030  5947 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
W20260709 12:25:06.287644  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.280200  9812 raft_consensus.cc:2764] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
W20260709 12:25:06.294978  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 161621us
W20260709 12:25:06.297099  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 120086us
I20260709 12:25:06.298082  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:06.298859  9483 consensus_queue.cc:1243] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:06.309890  5148 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:06.310050  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 143414us
I20260709 12:25:06.310890  5947 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:25:06.304683  9650 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:06.305570  9499 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.325187  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.333400  5942 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:06.333971  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:06.318030  9419 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:06.338587  5149 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:25:06.339751  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:06.336200  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:06.345963  5144 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:06.325220  9812 raft_consensus.cc:3055] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:06.370021  9812 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:06.371487  9812 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:06.374030  5042 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:06.336198  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:06.354087  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:06.330891  9705 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.326787  9650 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:06.362370  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1434) took 2039 ms. Trace:
I20260709 12:25:06.362788  9797 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:06.390444  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:06.365345  5146 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:06.332930  9499 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:06.383113  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 62161.5us
I20260709 12:25:06.386077  5148 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:06.361604  9842 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:25:06.381592  4894 catalog_manager.cc:5697] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 6 to 11, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 11 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:06.389284  9705 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:06.387506  5145 rpcz_store.cc:276] 0709 12:25:04.322377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.516251 (+193874us) service_pool.cc:224] Handling call
0709 12:25:05.418723 (+902472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:05.419507 (+   784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.419537 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:05.420564 (+  1027us) log.cc:844] Serialized 52 byte log entry
0709 12:25:05.446966 (+ 26402us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:05.578856 (+131890us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.579258 (+   402us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.582710 (+  3452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.587621 (+  4911us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.861427 (+273806us) raft_consensus.cc:1685] finished
0709 12:25:05.861449 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.361947 (+500498us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b540036e550. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:06.362139 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4453,"prepare.queue_time_us":658192,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":8512,"raft.queue_time_us":10582,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":12807,"spinlock_wait_cycles":1820551040,"thread_start_us":135706,"threads_started":2,"wal-append.queue_time_us":49183}
I20260709 12:25:06.391464  5942 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:06.392549  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:06.318868  9483 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:25:06.397224  5146 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:06.394526  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:06.373433  9812 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:06.408840  5948 raft_consensus.cc:1217] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 10.7->[11.8-14.10]   Dedup: 11.8->[12.9-14.10]
I20260709 12:25:06.390864  9650 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:06.409576  5948 pending_rounds.cc:85] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 8
W20260709 12:25:06.418352  9830 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.407s	user 0.002s	sys 0.002s
I20260709 12:25:06.419098  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:06.420644  5144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf"
dest_uuid: "ceec344dd00f40c18ef58414ce147016"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49182
I20260709 12:25:06.421205  5144 raft_consensus.cc:606] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:06.429891  9799 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:06.430709  9834 log.cc:927] Time spent T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.419s	user 0.002s	sys 0.003s
I20260709 12:25:06.431401  9705 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:06.430800  9799 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:06.432554  9811 raft_consensus.cc:3055] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:06.432991  9284 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:06.433207  9811 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:06.434724  9811 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:06.436028  9811 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:06.437170  9671 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.438077  9671 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:06.398298  9850 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:25:06.459952  9215 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:06.460399  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:06.462019  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:06.462565  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:06.463581  5144 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 16: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:06.464396  5899 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:06.464761  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.464849  9806 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:06.413192  5948 pending_rounds.cc:107] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:25:06.468101  5220 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:06.477316  5948 raft_consensus.cc:2889] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:06.474491  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:06.477324  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.481588  5148 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
W20260709 12:25:06.486326  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.488359  9797 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.488224  9850 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:06.488538  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:06.489213  9797 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:06.491752  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.492971  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.493361  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.493214  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.494033  5939 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:06.494450  5946 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:06.495379  9797 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:06.497087  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:06.497982  5943 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:06.498925  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:06.494127  5947 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:25:06.507479  5069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:06.515461  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:06.517374  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:06.517948  5945 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:25:06.518375  5945 pending_rounds.cc:85] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 6
I20260709 12:25:06.518656  5945 pending_rounds.cc:107] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:25:06.518900  5945 raft_consensus.cc:2889] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:06.518975  5941 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:25:06.520169  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.520318  5220 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:06.484145  9581 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:06.519407  5941 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:06.496447  9849 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:06.523077  5224 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:06.522214  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:06.531524  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:06.539857  5939 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:06.535485  5223 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:06.545003  5222 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:06.546537  5942 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:06.547397  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:06.548084  5147 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:06.548383  5947 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:06.549003  9122 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:06.549588  9122 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.549960  9122 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:06.550577  9846 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:06.551183  9565 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:06.551636  9565 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:06.551731  5946 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:06.543279  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:06.534210  9843 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:25:06.560788  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.563609  5150 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:06.566977  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:06.578809  5945 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:06.568501  5940 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:25:06.584044  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:06.584546  5224 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:06.586017  5939 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:06.587180  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.559204  5941 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:06.538228  9607 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:06.591765  5041 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:06.577751  9419 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:06.543175  9849 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:06.614814  9419 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:06.616391  9849 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:06.617887  9499 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:06.618710  9499 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:06.619179  9499 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:06.620157  9499 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:06.621625  9565 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:06.597308  9797 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:06.624454  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 150038us
W20260709 12:25:06.626127  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 91521.2us
I20260709 12:25:06.623193  9797 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:06.610769  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 47510us
W20260709 12:25:06.631680  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.627615  9738 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:25:06.632386  9607 consensus_queue.cc:1059] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:06.601413  9799 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:06.600207  9843 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:06.637042  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:06.640456  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:06.641126  5899 leader_election.cc:400] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18.
I20260709 12:25:06.641609  5899 leader_election.cc:403] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:06.659070  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:06.660765  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:06.633460  9797 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:06.653750  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:06.632995  9671 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:06.665410  5072 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:06.669672  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 327598us
I20260709 12:25:06.643903  9705 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:06.645522  9738 raft_consensus.cc:3055] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:06.642625  9565 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:06.652715  9849 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:25:06.691088  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:06.694178  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.638332  9754 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:25:06.642377  9284 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:25:06.702863  5041 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:06.703591  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:06.707423  5067 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:06.698995  9658 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.023s	sys 0.050s
I20260709 12:25:06.694262  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.671377  9679 raft_consensus.cc:3055] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:06.700552  5942 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 9: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:06.638550  9806 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:25:06.681416  9671 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:06.687564  9799 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:06.651659  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:06.688697  9705 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.717593  5068 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:06.713101  9658 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.024s	sys 0.050s
I20260709 12:25:06.688460  9738 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:06.751417  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:06.716614  9679 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:06.694700  9856 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:25:06.722877  9671 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.780007  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:06.721740  9859 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 18
W20260709 12:25:06.794092  9852 log.cc:927] Time spent T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:25:06.725678  9854 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:06.797956  9854 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:06.799142  9854 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:06.800300  5943 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:06.802253  5945 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:06.804780  9679 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:06.726379  9853 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:06.808538  9658 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:06.809593  9658 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:06.809952  9658 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:06.810250  5067 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:06.810761  9658 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:06.787294  9846 raft_consensus.cc:993] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: : Instructing follower 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:06.728000  9705 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:06.771579  9799 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:06.789938  9665 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:06.816740  9665 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.820966  9665 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:06.823196  9846 raft_consensus.cc:1081] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Signalling peer 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:06.826658  9738 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:06.827350  9806 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.827903  9806 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:06.828220  9738 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:06.813946  9799 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.829999  9799 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:06.839143  9806 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:06.813035  9671 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:06.853439  5945 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:25:06.851271  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 190572us
W20260709 12:25:06.860584  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 201647us
I20260709 12:25:06.834486  9658 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:06.836181  9754 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:06.874847  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:06.730456  9843 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:06.847638  9790 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.870168  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.881958  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.883561  5944 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
W20260709 12:25:06.878454  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 277090us
I20260709 12:25:06.878645  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:06.885613  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:06.857007  5948 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:39870
I20260709 12:25:06.890455  5948 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:06.890931  5948 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:06.887070  9215 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:06.887912  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:06.898731  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.899744  5947 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:06.872051  9585 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.903112  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.909440  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:06.910439  5944 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:06.889074  5147 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:06.858949  9806 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.870105  9853 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:25:06.924960  9581 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:25:06.950398  9812 consensus_meta.cc:287] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.541s	user 0.012s	sys 0.005s
W20260709 12:25:06.950909  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 65448.1us
I20260709 12:25:06.959229  5069 raft_consensus.cc:606] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:06.959756  5069 raft_consensus.cc:614] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:06.960855  5115 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:06.961915  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:06.961923  9215 raft_consensus.cc:3055] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:06.963557  9215 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:06.966961  9864 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.967630  9864 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:06.970065  9857 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:25:06.902698  9754 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:06.954828  9867 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.964799  9215 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:06.990454  5948 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:06.932014  9585 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:06.935628  9790 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:07.000874  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.956883  9284 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:25:06.951319  9806 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:07.014073  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 136204us
I20260709 12:25:07.016374  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:07.014874  5948 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:07.017279  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 365711us
I20260709 12:25:07.003196  9864 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.026224  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:07.033216  5067 raft_consensus.cc:2410] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:07.042284  5068 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:07.042336  5224 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:25:06.991935  9866 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:25:07.013048  9859 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18.
I20260709 12:25:07.050329  5220 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:25:07.051084  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:07.051950  5224 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
W20260709 12:25:07.053332  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 193873us
W20260709 12:25:07.054747  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:07.055383  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.058316  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:07.059216  5224 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:07.064886  5223 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:07.064935  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:07.048971  9864 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.050434  9215 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.064422  9849 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:07.059145  9650 consensus_meta.cc:287] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.627s	user 0.012s	sys 0.005s
W20260709 12:25:07.083832  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 185294us
I20260709 12:25:06.996035  9867 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.074100  5899 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:07.082870  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 372924us
I20260709 12:25:07.086882  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:07.087910  4895 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43234 (request call id 126) took 40136 ms (client timeout 15000 ms). Trace:
W20260709 12:25:07.087950  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.042701  9862 consensus_queue.cc:1059] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
W20260709 12:25:07.115622  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.116109  5072 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
W20260709 12:25:07.117020  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 57210.1us
I20260709 12:25:07.118985  5190 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:07.121592  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 47617.1us
I20260709 12:25:07.088397  9863 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:07.132450  5942 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:07.134673  5070 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:07.136080  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:07.142920  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 84285.1us
I20260709 12:25:07.087999  9869 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:25:07.144611  5188 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:07.144712  9799 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.146441  5222 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:25:07.147476  9799 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:07.148663  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 186822us
I20260709 12:25:07.090384  9864 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:07.096546  9875 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:07.091979  9650 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:07.068140  9581 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:07.138113  9863 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:07.168074  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 82733us
W20260709 12:25:07.168591  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 81886.1us
I20260709 12:25:07.168900  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:07.161779  9122 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.611s	user 0.019s	sys 0.002s
W20260709 12:25:07.168229  5898 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.912s (SENT) [suppressed 36 similar messages]
I20260709 12:25:07.174228  9883 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.167169  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:07.178658  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:07.177626  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:07.153396  9859 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:07.187076  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 68222.3us
I20260709 12:25:07.188666  5072 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:07.190773  5223 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:07.164440  9875 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:07.192986  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:07.147581  9665 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.171818  9863 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:07.164970  9866 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:07.186089  9878 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:07.169751  9862 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:07.206992  9871 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:07.209306  5188 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:07.078269  4894 catalog_manager.cc:5697] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 12 to 15, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 15 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:07.214815  9705 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.215790  9870 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:07.216396  9870 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.215600  4894 catalog_manager.cc:5697] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:07.217700  9887 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:25:07.199422  9790 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.219396  9877 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:07.219545  9790 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:07.220185  9877 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:07.220372  9790 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:07.221804  9881 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.193658  9859 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.223526  9859 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:07.220842  9869 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:07.195273  9875 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:07.218621  9879 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:25:07.229116  5946 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 12:25:07.227757  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 60755.9us
I20260709 12:25:07.230026  5224 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:07.229154  9869 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:07.231204  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:07.204152  9806 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:07.189913  9122 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.232213  9869 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.234676  9869 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:07.235512  9122 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.212383  9585 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.236761  9585 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:07.177260  9894 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:25:07.238307  9585 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:07.238466  9889 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.238776  9585 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.239079  9889 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:07.239461  9585 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:07.239565  9483 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:07.233516  9806 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:07.206691  9862 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:07.249428  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 71953.2us
I20260709 12:25:07.250218  5189 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:25:07.222537  9881 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.238731  9894 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:25:07.088397  4895 rpcz_store.cc:269] 0709 12:24:26.920102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.920425 (+   323us) service_pool.cc:224] Handling call
0709 12:25:07.051655 (+40131230us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:58.745778 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:58.746483 (+   705us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:58.746564 (+    81us) write_op.cc:435] Acquired schema lock
0709 12:24:58.746586 (+    22us) tablet.cc:662] Decoding operations
0709 12:24:58.748019 (+  1433us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:58.748125 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:58.748151 (+    26us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:24:58.837572 (+ 89421us) tablet.cc:701] Row locks acquired
0709 12:24:58.849972 (+ 12400us) write_op.cc:260] PREPARE: finished
0709 12:24:58.918239 (+ 68267us) write_op.cc:270] Start()
0709 12:24:58.918542 (+   303us) write_op.cc:276] Timestamp: P: 1783599898863431 usec, L: 0
0709 12:24:58.918581 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:24:58.919370 (+   789us) log.cc:844] Serialized 4388 byte log entry
0709 12:24:59.071230 (+151860us) op_driver.cc:464] REPLICATION: finished
0709 12:24:59.176816 (+105586us) write_op.cc:301] APPLY: starting
0709 12:24:59.177091 (+   275us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:59.329022 (+151931us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:59.329176 (+   154us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:59.437424 (+108248us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:59.490776 (+ 53352us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 12:24:59.490828 (+    52us) write_op.cc:312] APPLY: finished
0709 12:24:59.491300 (+   472us) log.cc:844] Serialized 104 byte log entry
0709 12:24:59.491533 (+   233us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:59.491700 (+   167us) write_op.cc:454] Released schema lock
0709 12:24:59.491780 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:59.492240 (+   460us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:59.492269 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5436496,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":105260,"apply.run_cpu_time_us":22094,"apply.run_wall_time_us":315385,"cfile_cache_hit":42,"cfile_cache_hit_bytes":43869,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6216,"cfile_init":2,"lbm_read_time_us":25999,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":10,"num_ops":11,"prepare.queue_time_us":239729,"prepare.run_cpu_time_us":6391,"prepare.run_wall_time_us":173992,"replication_time_us":152441,"thread_start_us":254168,"threads_started":2}]]}
I20260709 12:25:07.256980  9881 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:07.242087  9867 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.216759  9870 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:07.243443  9806 raft_consensus.cc:697] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:07.191835  9883 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.259390  9806 consensus_queue.cc:237] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:07.262406  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 55436us
I20260709 12:25:07.263093  9866 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:07.261166  9886 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:07.274879  9886 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.274767  9866 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:07.275370  9886 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:07.226332  9871 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:07.200150  9799 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:07.202939  9878 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:07.209141  9864 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:07.293285  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 100400us
W20260709 12:25:07.312700  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44088.1us
I20260709 12:25:07.287452  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:07.241515  9889 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:07.221412  9877 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:07.289484  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:07.297953  9888 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:07.318019  9888 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:07.286752  4894 catalog_manager.cc:5697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 10 to 15, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 15 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:07.262006  9883 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:07.282683  9499 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:07.300428  9863 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:07.269436  9881 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:25:07.318421  9888 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:07.290144  9665 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.263535  9887 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:07.316767  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:07.285804  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 53651.6us
I20260709 12:25:07.336359  5189 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:07.336951  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:07.337883  5147 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:07.338115  5150 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:25:07.321231  9863 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.269068  9862 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:07.286569  9871 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.288830  9892 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:07.243112  9483 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:07.356297  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:07.324843  9499 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.357103  5146 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:07.359030  5147 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:25:07.334599  9889 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:07.360334  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:07.364585  5224 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 17: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:07.339636  9887 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.337634  9799 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:07.367347  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:07.368144  5223 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:07.372304  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:07.373505  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.374159  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.375207  5149 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
W20260709 12:25:07.373755  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 57082.7us
I20260709 12:25:07.293411  9859 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.297358  9671 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.374861  5142 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:07.389664  5145 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:07.389963  9895 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.363618  9499 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:07.390753  9895 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:07.393721  9867 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.394300  9867 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.364104  9889 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.397393  9889 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.398514  5941 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:07.399076  9790 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.376336  9650 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:07.401042  9863 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.362090  9665 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.401826  9863 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:07.402004  9665 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:07.404181  5146 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:07.404738  9896 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:07.344503  9883 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:25:07.375316  9862 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:25:07.407373  9862 consensus_queue.cc:1243] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 16.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:07.378322  9864 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:25:07.378639  9872 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:07.414561  9872 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:07.416913  9873 raft_consensus.cc:3055] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:07.450541  9873 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:07.452200  9873 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.335088  9894 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:25:07.453814  9894 consensus_queue.cc:1243] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:07.370023  9887 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:07.460959  5899 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:07.491271  5948 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:07.423324  9879 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:07.494397  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:07.424595  9754 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:07.425238  9896 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:07.382642  9878 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:07.428144  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:07.435544  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:07.435534  9849 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:07.443351  5069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:07.445339  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.525756  5147 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:07.526515  5948 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:25:07.447332  5145 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:07.377785  5141 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:07.418910  9679 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:07.354883  9871 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:07.320475  4894 catalog_manager.cc:5697] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 13 to 14, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 14 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:07.468883  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:07.555230  5224 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
W20260709 12:25:07.556238  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 228537us
I20260709 12:25:07.463831  9650 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.464526  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.478210  5220 pending_rounds.cc:85] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 6
W20260709 12:25:07.421721  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.560941  5220 pending_rounds.cc:107] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 16.7
I20260709 12:25:07.561332  5220 raft_consensus.cc:2889] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:07.468763  9738 consensus_meta.cc:287] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.640s	user 0.019s	sys 0.005s
W20260709 12:25:07.479653  9797 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.797s	user 0.012s	sys 0.003s
W20260709 12:25:07.504724  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 78054.2us
W20260709 12:25:07.481226  9565 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.778s	user 0.005s	sys 0.008s
I20260709 12:25:07.478708  9843 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:07.524555  5069 raft_consensus.cc:606] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:07.393465  9895 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:07.494009  9705 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:07.515939  9658 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
W20260709 12:25:07.531591  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.536327  9799 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.540107  9483 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.542963  9671 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:07.546209  9886 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:07.530123  9894 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:25:07.550200  5222 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 10: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:07.556545  5899 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:07.354676  9892 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:07.503654  9860 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:07.521802  9873 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:07.505837  9665 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:07.614943  9872 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.616052  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:07.616744  9215 consensus_meta.cc:287] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.535s	user 0.015s	sys 0.003s
W20260709 12:25:07.567654  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.529280  9849 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.574201  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.512319  9850 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:25:07.621958  5141 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:07.621671  9896 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:07.622488  9850 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:07.623588  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.624037  9585 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.624975  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.625192  5150 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:07.622964  9850 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:07.625977  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:07.626260  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.626757  5142 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.627127  9850 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.627635  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.628240  5143 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:07.628489  9850 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:07.629873  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.630003  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:07.631078  9585 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:07.631172  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.572754  9797 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.608556  9860 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.633273  9863 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.633636  9860 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:07.571836  9650 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:07.633908  9797 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:07.634769  5215 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 12:25:07.637977  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.638247  9863 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:07.638902  9863 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:07.583957  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.640516  5144 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:25:07.641705  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.642168  9650 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.643018  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:07.644011  9894 consensus_queue.cc:1243] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:07.645720  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:07.646399  5141 raft_consensus.cc:2376] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12. Re-sending same reply.
I20260709 12:25:07.646668  9878 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:07.647310  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:07.648626  9849 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:07.650279  5150 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:07.651166  9849 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.605985  9754 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:07.654489  9859 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.614194  9867 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.593605  9843 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:07.654881  9754 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.653992  4894 catalog_manager.cc:5697] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:07.655838  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:07.655900  9843 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:07.656975  5222 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:07.656667  4894 catalog_manager.cc:5697] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:07.629177  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:07.637825  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 45851.3us
I20260709 12:25:07.648635  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:07.661314  5141 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:25:07.659278  5143 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
W20260709 12:25:07.530193  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.609045  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:25:07.666862  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 197481us
W20260709 12:25:07.670801  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 347441us
I20260709 12:25:07.671352  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:07.672235  5224 raft_consensus.cc:2410] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:07.655788  9754 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:07.676729  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:07.680739  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.680958  5223 raft_consensus.cc:2410] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:25:07.683370  5898 leader_election.cc:336] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.505s (SENT)
I20260709 12:25:07.641494  9863 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:07.685992  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:07.688495  5142 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:25:07.690068  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:07.691740  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:07.700052  5189 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:07.693455  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:07.703265  5948 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:07.695708  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:07.670583  9797 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:07.705108  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.709492  5149 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:25:07.711359  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:07.712374  5948 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:07.713841  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:07.676710  9705 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.705890  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:07.651826  9889 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:07.722899  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:07.702824  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:07.685206  9859 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.718698  5189 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:07.727571  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:07.729836  5224 raft_consensus.cc:2393] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:25:07.730427  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 49800.9us
I20260709 12:25:07.732151  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:07.733155  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:07.734131  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:07.736903  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 62829.9us
I20260709 12:25:07.670579  9867 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:25:07.750324  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:07.750841  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 69266.9us
I20260709 12:25:07.754298  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:07.756283  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:07.798944  5941 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:25:07.758159  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:07.799559  5941 pending_rounds.cc:85] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 8
I20260709 12:25:07.766286  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:07.658339  4894 catalog_manager.cc:5697] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:07.767261  9877 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:07.721166  9849 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:25:07.801157  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:07.792126  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.751608  5115 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:07.799415  5946 raft_consensus.cc:2410] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:25:07.765911  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:07.802048  5945 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 17: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:07.725503  9565 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:07.808498  5218 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:07.800195  5943 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:07.800930  5219 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
W20260709 12:25:07.802050  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.734965  9895 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:07.731942  9875 consensus_meta.cc:287] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.501s	user 0.013s	sys 0.000s
I20260709 12:25:07.800089  5941 pending_rounds.cc:107] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 15.9
W20260709 12:25:07.813215  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 61728.7us
I20260709 12:25:07.813510  5941 raft_consensus.cc:2889] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:07.807885  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:07.807914  5940 pending_rounds.cc:85] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 4
I20260709 12:25:07.814570  5940 pending_rounds.cc:107] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 12:25:07.807885  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:07.814932  5940 raft_consensus.cc:2889] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:07.816692  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:07.796317  5222 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:07.819902  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 949) took 1306 ms. Trace:
I20260709 12:25:07.820569  5942 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:07.724889  9869 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:25:07.721608  9797 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:07.809132  9565 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.818778  9867 raft_consensus.cc:3055] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:25:07.825111  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 74883.4us
W20260709 12:25:07.834553  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.820554  5941 rpcz_store.cc:276] 0709 12:25:06.508245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.508377 (+   132us) service_pool.cc:224] Handling call
0709 12:25:06.518825 (+ 10448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.790460 (+271635us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b18000a2720. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a8576646 00007fd8a84b058e 00007fd8a8531f9e 00007fd8a85337de 00007fd8a853db49 00007fd8a853e416 00007fd8a853ad8f 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4
0709 12:25:07.398233 (+607773us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5800660ed0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a84970aa 00007fd8a8519447 00007fd8a8531fe7 00007fd8a85337de 00007fd8a853db49 00007fd8a853e416 00007fd8a853ad8f 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4
0709 12:25:07.814444 (+416211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.814670 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642864256}
I20260709 12:25:07.725345  9705 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:07.824674  5899 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:07.785673  9626 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:25:07.821332  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:07.819584  9870 consensus_meta.cc:287] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.003s
I20260709 12:25:07.741693  9859 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.815685  9665 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.820055  9893 raft_consensus.cc:2764] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:07.822574  9585 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:07.919276  5898 leader_election.cc:336] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.562s (SENT)
W20260709 12:25:07.923578  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 102325us
I20260709 12:25:07.945297  5898 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:07.933679  9888 consensus_meta.cc:287] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.588s	user 0.013s	sys 0.003s
I20260709 12:25:07.952358  9888 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.953562  9738 consensus_queue.cc:1059] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:07.953897  9745 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:07.850160  9899 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:07.955145  9899 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.955767  9899 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:07.955178  5215 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:07.875233  9565 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:07.958985  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:07.717094  9892 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:07.826282  9895 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:07.968631  5947 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:07.836131  9865 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:07.919369  9419 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.954778  9738 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:07.917414  9867 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:07.820149  9875 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:07.858547  9872 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 12:25:07.945174  9862 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.939807  9863 raft_consensus.cc:2764] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:25:07.827814  9887 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:07.945160  9585 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:07.967370  9901 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:25:07.979792  9888 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:07.986250  9877 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.945420  9665 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:07.950883  9483 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:25:07.938351  9870 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:08.040417  5144 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:07.958012  9790 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:25:08.003557  9896 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:08.007443  9806 raft_consensus.cc:3055] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:08.029973  9886 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:08.035418  9585 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:07.967285  9904 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:08.061349  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:08.050133  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:08.030987  9903 log.cc:927] Time spent T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.371s	user 0.002s	sys 0.000s
I20260709 12:25:08.011361  9738 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:07.988714  9850 raft_consensus.cc:3055] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:08.066602  5221 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:25:08.067023  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 120920us
I20260709 12:25:08.008309  9849 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:08.091300  5144 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:08.035951  9862 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:08.073700  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 216125us
I20260709 12:25:08.045780  9869 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.077363  9930 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:08.079524  5947 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
W20260709 12:25:08.080088  9908 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.415s	user 0.000s	sys 0.002s
I20260709 12:25:08.032601  9419 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:08.066638  9679 consensus_meta.cc:287] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.003s
W20260709 12:25:08.070695  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 118822us
I20260709 12:25:08.063963  9806 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:08.008306  4894 catalog_manager.cc:5697] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 7 to 14, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 14 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:08.075486  9856 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:08.052940  9877 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:08.161646  5141 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
W20260709 12:25:08.129365  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.080534  9754 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:08.091801  9849 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:08.101977  9889 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:08.119766  9790 consensus_queue.cc:1059] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:25:08.148684  5942 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:08.151793  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:08.153174  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:08.098778  9863 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:25:08.093180  9850 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:08.123459  9848 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:08.172288  9918 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:25:08.184985  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:08.118112  9869 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:08.173018  9483 consensus_queue.cc:1243] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:25:08.184082  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 84504.7us
I20260709 12:25:08.192348  9679 raft_consensus.cc:3055] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:25:08.181872  9931 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:25:08.152971  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:08.178993  9849 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:08.200706  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 47611.8us
W20260709 12:25:08.194783  9910 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.397s	user 0.002s	sys 0.000s
I20260709 12:25:08.170943  9898 raft_consensus.cc:2764] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 12:25:08.215045  5190 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.674s (SENT)
I20260709 12:25:08.180969  9856 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:08.218755  9875 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:08.203646  9889 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.223765  9899 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:08.224879  9895 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:08.226315  9665 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:08.196902  9797 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:08.227579  9873 consensus_meta.cc:287] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.613s	user 0.005s	sys 0.005s
W20260709 12:25:08.227859  9913 log.cc:927] Time spent T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:25:08.227260  9738 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:08.229235  5042 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:08.205017  9925 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.208706  9679 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:08.214148  9859 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:08.287487  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:08.288493  5070 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 10.
I20260709 12:25:08.292323  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:08.293326  5067 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:08.299228  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 70107.2us
I20260709 12:25:08.246608  9886 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:08.237497  9483 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:08.237843  9848 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:08.279328  9932 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:08.305588  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 54944.3us
W20260709 12:25:08.308431  9920 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
W20260709 12:25:08.319357  5114 leader_election.cc:336] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.519s (SENT)
I20260709 12:25:08.275405  9930 consensus_queue.cc:1059] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 12:25:08.297868  9499 consensus_meta.cc:287] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.906s	user 0.012s	sys 0.003s
W20260709 12:25:08.331576  5114 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.528s (SENT)
I20260709 12:25:08.327489  5067 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:08.329890  5115 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:08.249420  9889 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:08.280454  9895 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:25:08.336552  5114 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:08.357158  9886 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:08.392645  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 62942us
I20260709 12:25:08.234591  9806 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:08.329376  9856 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:08.399176  5217 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.009s	sys 0.033s
W20260709 12:25:08.403851  5217 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.009s	sys 0.033s
I20260709 12:25:08.395311  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:08.341604  9870 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:08.392969  9928 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
W20260709 12:25:08.392165  5187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.000s	sys 0.040s
I20260709 12:25:08.286489  9925 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:08.407619  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 76096.3us
W20260709 12:25:08.418702  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.419559  5114 leader_election.cc:336] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 2.052s (SENT)
W20260709 12:25:08.420235  5187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.000s	sys 0.041s
W20260709 12:25:08.422127  5189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.013s	sys 0.034s
W20260709 12:25:08.422853  5189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.013s	sys 0.034s
W20260709 12:25:08.423972  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.337769  9483 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:08.399140  9875 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:08.412613  9922 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
I20260709 12:25:08.212697  4894 catalog_manager.cc:5697] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 7 to 10, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 10 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:08.319357  9850 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:08.416497  9863 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:08.428619  9914 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.292s	user 0.002s	sys 0.000s
I20260709 12:25:08.339382  9932 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:08.425889  9862 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:08.422938  9938 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:08.460505  9945 raft_consensus.cc:3055] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:08.460939  9930 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:08.461658  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:08.462373  9917 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 12:25:08.431141  9806 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:08.478081  5218 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:08.429879  9942 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:08.375490  9499 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:08.466540  5224 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:25:08.467458  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 63160us
I20260709 12:25:08.469308  9886 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:08.448314  9924 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 12:25:08.473308  9848 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:08.466686  5147 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.042s	sys 0.065s
I20260709 12:25:08.477415  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:08.412042  9679 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:08.443764  9754 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:08.509418  5899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.012s	sys 0.015s
W20260709 12:25:08.512207  5899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.012s	sys 0.016s
W20260709 12:25:08.512310  5147 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.045s	sys 0.065s
I20260709 12:25:08.512234  5222 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
W20260709 12:25:08.513170  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 688555us
I20260709 12:25:08.470245  9945 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:08.475785  9877 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:08.474700  9930 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:08.520687  5223 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:08.518590  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:08.522473  5218 raft_consensus.cc:2393] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:08.518054  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:08.493974  9895 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.481925  9850 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:08.494292  9797 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:08.524317  5223 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 12:25:08.524856  9936 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:25:08.526134  9932 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:08.524407  9899 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:08.482493  9942 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:08.558149  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 162950us
I20260709 12:25:08.528563  9930 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:08.590299  5114 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.712s (SENT)
I20260709 12:25:08.591959  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:08.575275  5041 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:08.595520  5115 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:08.597477  5218 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:25:08.640338  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 44934.1us
I20260709 12:25:08.640929  5114 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:08.640072  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 100749us
I20260709 12:25:08.651321  5220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 920) took 1174 ms. Trace:
I20260709 12:25:08.651002  5217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 921) took 1173 ms. Trace:
I20260709 12:25:08.653460  5189 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:08.655158  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.591904  9866 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.011s	sys 0.021s
I20260709 12:25:08.593636  9938 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.653614  5220 rpcz_store.cc:276] 0709 12:25:07.477392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.477595 (+   203us) service_pool.cc:224] Handling call
0709 12:25:07.477981 (+   386us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:07.592603 (+114622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.592654 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:07.598640 (+  5986us) log.cc:844] Serialized 52 byte log entry
0709 12:25:07.832017 (+233377us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:07.832055 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.832119 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.641809 (+809690us) raft_consensus.cc:1685] finished
0709 12:25:08.641834 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.651243 (+  9409us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45171,"raft.queue_time_us":355,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":11247,"thread_start_us":173147,"threads_started":1,"wal-append.queue_time_us":299510}
I20260709 12:25:08.652114  5217 rpcz_store.cc:276] 0709 12:25:07.477844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.478570 (+   726us) service_pool.cc:224] Handling call
0709 12:25:07.500126 (+ 21556us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:07.514776 (+ 14650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.514840 (+    64us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:07.575880 (+ 61040us) spinlock_profiling.cc:243] Waited 8.14 ms on lock 0x7b2c000af528. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:25:07.599378 (+ 23498us) log.cc:844] Serialized 52 byte log entry
0709 12:25:07.732965 (+133587us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:07.849431 (+116466us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.404380 (+554949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.404456 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.598996 (+194540us) raft_consensus.cc:1685] finished
0709 12:25:08.640669 (+ 41673us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.650946 (+ 10277us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":299378,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":213145,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":22388,"spinlock_wait_cycles":28104448,"thread_start_us":711836,"threads_started":3,"wal-append.queue_time_us":184005}
I20260709 12:25:08.664237  5220 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:08.670544  9499 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:08.673488  9945 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:08.675129  9945 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:08.676126  9889 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:08.574721  9843 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:25:08.649784  4894 catalog_manager.cc:5697] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 10 to 13, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 13 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:08.679656  4894 catalog_manager.cc:5697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:08.596879  9949 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:08.685936  9871 consensus_meta.cc:287] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.115s	user 0.015s	sys 0.000s
I20260709 12:25:08.686914  9871 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:08.690147  9925 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:08.691826  9790 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:08.692443  9790 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:25:08.695964  9878 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.049s	user 0.009s	sys 0.004s
I20260709 12:25:08.700551  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35616 (request call id 1254) took 1030 ms. Trace:
I20260709 12:25:08.702093  5222 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:08.706786  9896 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.707434  9896 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:08.708326  5148 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:25:08.691896  9483 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:08.701067  5215 rpcz_store.cc:276] 0709 12:25:07.669809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.676405 (+  6596us) service_pool.cc:224] Handling call
0709 12:25:07.703302 (+ 26897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.956460 (+253158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.700322 (+743862us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54003c8f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:08.700483 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578819456}
W20260709 12:25:08.710016  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.711378  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.693486  9790 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:08.716439  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:08.716760  9790 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:08.717814  5076 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:08.718081  9898 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:08.718494  9898 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.718918  9898 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:08.721712  9565 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.734s	user 0.014s	sys 0.000s
I20260709 12:25:08.721940  9951 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.722193  9565 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:08.722476  9951 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:08.724152  9863 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:08.724397  9565 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:08.713197  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 561457us
I20260709 12:25:08.713302  9665 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:08.714140  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:08.737589  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 84215.5us
I20260709 12:25:08.714749  9859 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:08.715445  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:08.738745  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:08.724659  9951 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:08.725404  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:08.649590  9932 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:08.712697  9927 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.725883  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:08.657831  9866 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.015s	sys 0.021s
W20260709 12:25:08.729116  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.725031  9863 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:25:08.740135  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:08.598313  9942 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:08.745527  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 155281us
I20260709 12:25:08.593319  9848 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:25:08.538564  9895 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:08.727142  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:08.718602  9705 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:08.730639  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:08.795542  5947 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 9.
W20260709 12:25:08.738517  9860 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.104s	user 0.020s	sys 0.000s
I20260709 12:25:08.795359  9483 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.796712  9860 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:08.797487  9483 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:08.799209  9860 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:08.591511  9797 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:08.799865  9483 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:08.801469  9797 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:08.685921  9949 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:08.804093  9949 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:08.806560  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 923) took 1466 ms. Trace:
W20260709 12:25:08.806874  9585 consensus_meta.cc:287] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.715s	user 0.012s	sys 0.001s
I20260709 12:25:08.807017  5148 rpcz_store.cc:276] 0709 12:25:07.340572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.344550 (+  3978us) service_pool.cc:224] Handling call
0709 12:25:07.344929 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:08.708882 (+1363953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.806271 (+ 97389us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400238554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:08.806503 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3134876032}
I20260709 12:25:08.807381  9585 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:08.809549  9916 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.950s	user 0.000s	sys 0.004s
I20260709 12:25:08.796725  9871 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:08.747255  9889 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:08.777109  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 51772.5us
W20260709 12:25:08.783439  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.747664  9932 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:08.681401  4894 catalog_manager.cc:5697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 9 to 11, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 11 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:08.781921  9873 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:08.744503  9859 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:08.727473  9856 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:25:08.754338  9927 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:08.790172  5215 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:08.794169  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:08.820617  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:08.821869  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 81764.7us
W20260709 12:25:08.810695  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 159633us
I20260709 12:25:08.811086  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:25:08.777680  5114 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 2.425s (SENT)
I20260709 12:25:08.822944  5219 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:08.823807  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:08.825740  5222 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:08.826355  9927 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:08.827247  9943 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:08.827909  9943 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:08.828006  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50366.3us
I20260709 12:25:08.842836  9585 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:08.842540  4894 catalog_manager.cc:5697] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 8 to 11, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 11 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:08.844350  4894 catalog_manager.cc:5697] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 8 to 9, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 9 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:08.846288  5218 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:08.747385  9665 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:08.847182  5944 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:08.848636  9859 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:08.853025  9873 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:08.853662  9873 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:08.855538  5944 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:08.815587  5900 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:08.744119  9896 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:08.747603  9925 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:25:08.861517  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 46000.2us
I20260709 12:25:08.862789  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:08.754999  9865 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.870159  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:08.870347  9865 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:08.871570  5221 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:08.872757  9865 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:08.874956  5222 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:08.828869  9943 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:08.879828  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:08.880159  5070 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:08.880455  5072 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:08.880913  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:08.881063  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:08.880507  5940 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:25:08.881855  5074 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:08.860714  5223 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:08.866829  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:08.868266  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:08.883455  5076 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:08.868958  5215 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:08.868809  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1467) took 1232 ms. Trace:
I20260709 12:25:08.869269  9896 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:08.886941  5150 rpcz_store.cc:276] 0709 12:25:07.636582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.651111 (+ 14529us) service_pool.cc:224] Handling call
0709 12:25:07.651475 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:07.651865 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.651895 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:07.652421 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.722428 (+ 70007us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:07.722498 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.722628 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.808715 (+1086087us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:08.820039 (+ 11324us) raft_consensus.cc:1685] finished
0709 12:25:08.820070 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.868746 (+ 48676us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12262,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":3485,"spinlock_wait_cycles":40514816,"thread_start_us":69106,"threads_started":1,"wal-append.queue_time_us":151793}
I20260709 12:25:08.887594  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:08.818008  9932 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:08.869798  9925 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.886924  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1468) took 1250 ms. Trace:
I20260709 12:25:08.888557  5075 raft_consensus.cc:2410] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:25:08.886101  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:08.899758  5219 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
W20260709 12:25:08.889829  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.901352  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.893862  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:08.902319  5067 raft_consensus.cc:2393] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:25:08.888309  9849 consensus_meta.cc:287] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.663s	user 0.008s	sys 0.005s
I20260709 12:25:08.898231  5144 rpcz_store.cc:276] 0709 12:25:07.636823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.642747 (+  5924us) service_pool.cc:224] Handling call
0709 12:25:08.886680 (+1243933us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b54001b6554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:08.886837 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":912249088}
I20260709 12:25:08.903427  5071 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:08.904613  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:08.905597  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.905429  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:08.907140  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:08.908579  5941 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:08.907217  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:08.909849  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:08.910166  5072 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:08.911038  5942 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:25:08.739106  9938 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:08.868399  9860 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:08.923178  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:08.924005  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 57257.2us
I20260709 12:25:08.914218  5223 raft_consensus.cc:2393] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:08.916394  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:08.928490  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:08.929557  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:08.911749  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:08.921700  5941 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:08.892231  9665 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:08.929925  5219 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 15: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:08.891055  9896 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:08.941771  9895 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:08.943863  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 56334.8us
I20260709 12:25:08.944432  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:08.945773  9893 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:08.946266  9893 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.946587  4894 catalog_manager.cc:5697] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:08.948628  9483 consensus_queue.cc:1059] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:08.948268  4894 catalog_manager.cc:5697] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 10 to 12, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 12 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:08.949523  9483 consensus_queue.cc:1059] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:08.914906  9849 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:08.950340  9483 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:08.951696  9849 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:08.951849  9856 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:08.952343  9849 raft_consensus.cc:3055] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:08.952395  9869 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:08.953224  9856 raft_consensus.cc:3055] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:08.953460  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:08.953680  9856 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:08.923581  9860 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.954538  9856 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:08.933967  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:08.955652  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:08.956456  9658 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:08.957022  5073 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:08.954077  9939 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:08.961321  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:08.961587  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:08.931823  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:08.934368  9944 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:08.937878  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:08.941890  9896 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:08.952800  9849 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:08.955065  9860 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:08.902640  9925 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:08.956225  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:09.010882  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:08.931524  9947 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.934293  9951 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:08.954330  5939 raft_consensus.cc:2410] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:25:09.014249  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:08.894974  9898 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:08.922945  9745 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.966543  5070 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:09.020164  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:08.965094  5041 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:08.971222  5075 raft_consensus.cc:2393] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:08.946637  9893 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.016393  5071 raft_consensus.cc:2368] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14.
I20260709 12:25:08.918377  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.016866  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:09.018594  5219 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:25:09.021800  5188 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:08.967808  5940 raft_consensus.cc:2368] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16.
I20260709 12:25:09.015661  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:09.024387  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:08.957019  9658 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:08.941838  9904 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:25:09.021596  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 56586.9us
I20260709 12:25:09.032265  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.033990  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:09.012984  9671 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.069s	user 0.001s	sys 0.015s
I20260709 12:25:08.971371  9626 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:09.038278  5217 raft_consensus.cc:2393] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:09.040130  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:09.045598  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:09.046386  5216 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:09.043865  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.043298  5188 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:09.054018  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.055233  5072 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:09.057257  5075 raft_consensus.cc:2410] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:25:09.058503  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:09.059170  5067 raft_consensus.cc:2376] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14. Re-sending same reply.
I20260709 12:25:09.060690  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.030644  9872 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:09.041826  9671 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.002s	sys 0.015s
I20260709 12:25:09.073665  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.079741  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.050470  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:09.074795  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.036908  9904 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:09.036137  9856 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:09.080958  5067 raft_consensus.cc:2410] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 12:25:09.077203  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 803724us
I20260709 12:25:09.086606  5072 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:09.089457  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:09.049490  9889 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:09.093696  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.096904  5067 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
W20260709 12:25:09.075914  9875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.020s	sys 0.033s
I20260709 12:25:09.098500  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.031478  9745 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:09.100334  5070 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:09.105527  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.060972  9626 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:09.112280  5076 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:09.112382  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.116178  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.114614  9953 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:09.056281  9859 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:09.058924  9870 consensus_queue.cc:1059] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:25:09.077447  9872 raft_consensus.cc:697] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:09.118551  5070 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:09.073309  9806 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:25:09.122606  9958 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:09.123095  9969 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:09.123832  5067 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 13: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:09.128316  9938 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:09.096279  9862 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.139848  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:09.025187  9947 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:09.102164  9875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.020s	sys 0.033s
I20260709 12:25:09.164212  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.165115  5942 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:09.165607  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.121628  9626 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:09.097409  9650 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:09.152495  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 73428.8us
I20260709 12:25:09.154636  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:09.156569  5948 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.110734  9949 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.141654  9799 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.086751  9849 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:09.160043  9930 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.001s
I20260709 12:25:09.114683  9879 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
W20260709 12:25:09.199654  9961 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:25:09.199800  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.161070  9958 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:09.204440  5067 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:25:09.205154  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.207577  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.209405  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:09.215330  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:09.216825  5072 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:09.217509  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:09.217813  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.170662  9969 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:09.171829  9875 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:09.170190  9938 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:09.231222  9938 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:09.232261  9964 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:25:09.154213  9944 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:25:09.235394  5076 raft_consensus.cc:2410] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 12:25:09.235839  9966 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:25:09.222530  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.107564  9658 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:09.238451  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:09.239156  5947 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:09.239962  9947 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:09.240123  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:09.223917  5067 raft_consensus.cc:2376] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16. Re-sending same reply.
I20260709 12:25:09.241814  9975 raft_consensus.cc:3055] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:09.242630  5067 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:09.188901  9949 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:09.244100  5067 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:09.244390  5070 raft_consensus.cc:2368] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18.
I20260709 12:25:09.247018  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:09.222990  5946 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:09.195279  9799 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:09.226547  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.228248  9867 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:09.230158  9958 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:09.181663  9483 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.248914  5947 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:09.273674  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:09.266013  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:09.285753  5067 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:09.222257  9927 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.174437  9862 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:09.290343  5074 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:09.160965  9872 consensus_queue.cc:237] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:09.201390  9930 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:09.290993  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:09.374495  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.133868  4894 catalog_manager.cc:5697] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:09.603446  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.246371  9938 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:09.322304  9483 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:09.549703  9896 consensus_meta.cc:287] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.000s
W20260709 12:25:09.423195  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.615455  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:09.577913  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.316756  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1588) took 1040 ms. Trace:
W20260709 12:25:09.706257  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.611603  5947 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:09.667174  5067 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:09.705988  5188 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:25:09.659657  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.432687  9898 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:09.709765  9898 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.710289  9898 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:09.710966  9862 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:09.500818  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.605010  9925 consensus_meta.cc:287] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.002s
I20260709 12:25:09.603075  9934 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:09.673065  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.715310  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.568109  9873 consensus_meta.cc:287] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.714s	user 0.011s	sys 0.001s
I20260709 12:25:09.635046  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:09.617097  9745 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:09.309628  9961 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:25:09.676437  9887 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:09.677914  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:09.753418  5216 raft_consensus.cc:2410] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:25:09.679560  9892 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:09.681035  9799 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:09.242693  9975 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:09.706453  5069 rpcz_store.cc:276] 0709 12:25:08.276622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.276777 (+   155us) service_pool.cc:224] Handling call
0709 12:25:08.286985 (+ 10208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.455155 (+168170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.455201 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.455252 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:08.455297 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.316376 (+861079us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540041e1d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:09.316450 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.316662 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386124160}
I20260709 12:25:09.668785  9849 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:09.729784  5898 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.561s (SENT)
W20260709 12:25:09.776253  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 46517.4us
I20260709 12:25:09.332526  5948 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
W20260709 12:25:09.508334  9973 log.cc:927] Time spent T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 12:25:09.666209  9864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.681s	user 0.006s	sys 0.013s
I20260709 12:25:09.741276  5221 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 12:25:09.404266  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.733932  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 413335us
W20260709 12:25:09.746941  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.360503  9968 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.487844  9951 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:09.308912  9848 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:09.634069  9850 consensus_meta.cc:287] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.059s	user 0.008s	sys 0.005s
I20260709 12:25:09.587142  9949 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:09.313864  9419 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:09.753782  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:09.310286  9969 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:09.767887  9893 consensus_meta.cc:287] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.726s	user 0.012s	sys 0.000s
I20260709 12:25:09.726456  9947 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:25:09.773918  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.717092  9938 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:25:09.808594  9893 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:09.815542  9893 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:09.817268  5075 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:25:09.832561  5075 pending_rounds.cc:85] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 6
I20260709 12:25:09.832794  5075 pending_rounds.cc:107] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:25:09.833040  5075 raft_consensus.cc:2889] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:09.749938  9925 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:09.715325  9896 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:09.840579  9671 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:09.831391  4894 catalog_manager.cc:5697] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 1 to 14. New cstate: current_term: 14 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.841286  9671 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:09.842608  4894 catalog_manager.cc:5697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.843631  9671 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:09.505041  9875 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:09.789845  9864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.835s	user 0.007s	sys 0.015s
I20260709 12:25:09.729413  9934 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:09.808528  5943 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:09.851722  9934 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:09.803897  9951 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:09.806550  9898 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:09.750051  9873 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:09.331403  9927 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:09.865788  5898 leader_election.cc:336] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.605s (SENT)
I20260709 12:25:09.867910  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:09.844339  4894 catalog_manager.cc:5697] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:09.882653  5898 leader_election.cc:336] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.701s (SENT)
I20260709 12:25:09.808512  9947 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.875437  5215 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:09.875838  9864 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:09.872944  9934 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:09.888938  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:09.861850  9988 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.866727  9865 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.864976  9898 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:09.893237  5898 leader_election.cc:336] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.912s (SENT)
I20260709 12:25:09.801419  9968 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:09.874557  9867 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:09.869073  9863 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.805236  9848 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:09.909811  5898 leader_election.cc:336] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 2.076s (SENT)
W20260709 12:25:09.885680  9942 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.074s	user 0.012s	sys 0.002s
W20260709 12:25:09.918435  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 172084us
I20260709 12:25:09.921655  5945 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:09.925174  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:09.887490  9947 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:09.911255  9856 consensus_meta.cc:287] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.820s	user 0.011s	sys 0.003s
W20260709 12:25:09.893747  9904 consensus_meta.cc:287] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.796s	user 0.006s	sys 0.008s
I20260709 12:25:09.911775  9934 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:09.905912  9850 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:09.938869  5041 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:09.942656  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:09.915686  9859 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:09.914951  9930 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:09.947465  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:09.942958  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:09.949348  5942 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:09.868547  9975 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:09.927496  9626 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:09.961052  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:09.918661  9865 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:09.915299  9988 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:09.951865  9856 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:09.938608  9419 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:25:09.929359  9863 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:10.002122  9863 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:09.918969  9898 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:09.965785  9974 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.335s	user 0.000s	sys 0.002s
I20260709 12:25:09.931272  9942 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:09.993813  9978 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:09.987877  9975 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:09.991078  9856 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:09.997836  5898 leader_election.cc:336] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 2.200s (SENT)
W20260709 12:25:09.999735  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.962190  9938 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:09.948817  9904 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:09.964138  9483 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:10.025307  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.024189  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1462) took 2386 ms. Trace:
I20260709 12:25:10.015117  9978 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.038079  5218 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18. Ops: []
W20260709 12:25:10.026408  5114 leader_election.cc:336] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.703s (SENT)
I20260709 12:25:10.044373  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:10.046258  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 48496.5us
W20260709 12:25:10.050230  5114 leader_election.cc:336] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.765s (SENT)
I20260709 12:25:10.050993  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:10.036114  5147 rpcz_store.cc:276] 0709 12:25:07.624968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.632196 (+  7228us) service_pool.cc:224] Handling call
0709 12:25:07.632561 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:07.633063 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:07.719205 (+ 86142us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.519867 (+800662us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:08.530034 (+ 10167us) log.cc:844] Serialized 52 byte log entry
0709 12:25:08.538389 (+  8355us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:08.723824 (+185435us) spinlock_profiling.cc:243] Waited 44.2 ms on lock 0x7b18000e5540. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e522 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:25:09.362738 (+638914us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b18000e5540. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8abf333c7 00007fd8a85459a7 00007fd8a85550bf 00007fd8a8555038
0709 12:25:09.782098 (+419360us) log.cc:844] Serialized 14 byte log entry
0709 12:25:10.011230 (+229132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.011464 (+   234us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.011533 (+    69us) raft_consensus.cc:1685] finished
0709 12:25:10.011554 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.011752 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5990,"prepare.queue_time_us":1527647,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":82,"raft.queue_time_us":9314,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":18530,"spinlock_wait_cycles":106691200,"thread_start_us":1447819,"threads_started":4,"wal-append.queue_time_us":786203,"wal-append.run_cpu_time_us":2616,"wal-append.run_wall_time_us":1132720}
I20260709 12:25:10.052161  5223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1502) took 1028 ms. Trace:
I20260709 12:25:10.025805  9856 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:10.052600  5223 rpcz_store.cc:276] 0709 12:25:09.023545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.037101 (+ 13556us) service_pool.cc:224] Handling call
0709 12:25:09.037525 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.051610 (+ 14085us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.051640 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.073500 (+ 21860us) log.cc:844] Serialized 26 byte log entry
0709 12:25:09.197446 (+123946us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:09.197507 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.197625 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.051937 (+854312us) raft_consensus.cc:1685] finished
0709 12:25:10.051959 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.052097 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30355,"raft.queue_time_us":13716,"raft.run_cpu_time_us":5389,"raft.run_wall_time_us":21749,"thread_start_us":88204,"threads_started":1,"wal-append.queue_time_us":461512}
I20260709 12:25:10.054698  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:10.030583  9930 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:10.058737  5190 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.845s (SENT)
I20260709 12:25:10.073278  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:10.038991  9863 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.048981  9978 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:10.033246  9987 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.006093  9991 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 12:25:10.071014  5114 leader_election.cc:336] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.986s (SENT)
W20260709 12:25:10.088058  5114 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 2.081s (SENT)
I20260709 12:25:10.050055  9925 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:10.084148  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.081061  9968 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:10.070816  9873 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:10.074914  9875 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:10.085354  9863 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:10.102777  9989 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:25:10.104045  9863 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:10.106153  9927 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:10.111112  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 150151us
I20260709 12:25:10.082502  9930 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:10.005306  4894 catalog_manager.cc:5697] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:10.090646  9958 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:10.112737  5041 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:10.109290  9932 consensus_meta.cc:287] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 1.199s	user 0.012s	sys 0.001s
I20260709 12:25:10.084518  9896 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:10.122601  9419 raft_consensus.cc:2764] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:25:10.094923  9951 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:10.123838  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.111069  9968 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:10.134400  9934 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:10.135676  9865 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:10.136242  9988 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:10.119766  9983 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:25:10.129391  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:10.145489  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.139175  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.144866  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:10.147109  5219 raft_consensus.cc:2410] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:25:10.139618  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.139191  5223 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:25:10.143054  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 1210) took 1421 ms. Trace:
I20260709 12:25:10.147881  5944 rpcz_store.cc:276] 0709 12:25:08.721168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.721292 (+   124us) service_pool.cc:224] Handling call
0709 12:25:10.142863 (+1421571us) spinlock_profiling.cc:243] Waited 97 ms on lock 0x7b5400004e54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:10.142993 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223172352}
I20260709 12:25:10.148638  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:10.119216  9848 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:10.150041  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.155005  5224 raft_consensus.cc:2393] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:10.149706  5220 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:10.153077  5899 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:10.145267  5946 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:10.153685  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.160043  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:10.164192  5222 raft_consensus.cc:2393] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:10.091190  9987 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:10.150252  9951 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
W20260709 12:25:10.153405  9981 log.cc:927] Time spent T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.383s	user 0.005s	sys 0.000s
I20260709 12:25:10.168237  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1585) took 2017 ms. Trace:
I20260709 12:25:10.149737  9850 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.171326  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.168809  5073 rpcz_store.cc:276] 0709 12:25:08.150591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.150839 (+   248us) service_pool.cc:224] Handling call
0709 12:25:08.170329 (+ 19490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.957508 (+787179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.167216 (+1209708us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b5400568bd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:10.168158 (+   942us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1808739712}
I20260709 12:25:10.143858  9873 raft_consensus.cc:476] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Not starting pre-election -- already a leader
I20260709 12:25:10.161387  9745 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:10.182140  5142 raft_consensus.cc:1240] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:25:10.149482  4894 catalog_manager.cc:5697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 9 to 10, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 10 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:10.173002  9898 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:10.174019  9865 consensus_queue.cc:1059] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:10.178094  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.178176  9930 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:10.179284  9904 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:10.180157  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.126266  9932 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:10.180228  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.202760  5141 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.172693  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:10.183284  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.183706  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:10.187841  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.187925  9896 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:10.188217  5145 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:10.189471  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.208177  5940 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:10.211112  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.190429  5146 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:10.189841  4894 catalog_manager.cc:5697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
W20260709 12:25:10.191071  9860 consensus_meta.cc:287] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.160s	user 0.012s	sys 0.002s
I20260709 12:25:10.191540  9942 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:10.149986  9927 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:25:10.229990  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.192381  5142 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:25:10.192524  9745 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:10.199229  5148 raft_consensus.cc:2368] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15.
I20260709 12:25:10.233342  5068 raft_consensus.cc:2410] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:25:10.204368  5147 raft_consensus.cc:2393] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 13: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:10.204799  5143 raft_consensus.cc:2410] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:25:10.205788  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:10.215626  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.216053  5940 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:10.235785  5940 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:10.182561  9850 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:10.218318  9991 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.218938  9860 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:10.222043  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.222682  9938 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:10.223465  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.223971  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 1104) took 1119 ms. Trace:
I20260709 12:25:10.221462  5141 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:10.227259  9658 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:10.228430  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.229808  5943 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:10.225358  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 983) took 1068 ms. Trace:
I20260709 12:25:10.192104  9987 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:10.232616  5146 raft_consensus.cc:2393] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:10.204131  5944 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:10.239499  5150 raft_consensus.cc:2393] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:10.206606  9941 raft_consensus.cc:3055] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:10.258287  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.258539  9941 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:10.258780  9932 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:10.259078  5143 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:10.259984  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.259682  9941 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:10.260547  5143 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:10.261159  9941 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:10.243848  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.250512  5941 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:10.251083  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.251160  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.263010  5142 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:10.263533  5147 raft_consensus.cc:2376] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15. Re-sending same reply.
I20260709 12:25:10.256754  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.253017  9745 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:10.282989  9987 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.283774  9987 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:10.284945  9850 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:10.286330  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.286204  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:25:10.286561  9849 consensus_meta.cc:287] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.000s
I20260709 12:25:10.287230  5146 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 11: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:10.253679  9879 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.288348  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.245214  5145 raft_consensus.cc:2376] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12. Re-sending same reply.
I20260709 12:25:10.289945  5146 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:10.290423  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.290820  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.291734  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.292303  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 1114) took 1067 ms. Trace:
I20260709 12:25:10.292444  5145 raft_consensus.cc:2376] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16. Re-sending same reply.
I20260709 12:25:10.292691  5945 rpcz_store.cc:276] 0709 12:25:09.202522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.278785 (+ 76263us) service_pool.cc:224] Handling call
0709 12:25:09.332358 (+ 53573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.927629 (+595271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.269499 (+341870us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b5400004954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:10.269748 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1331779456}
I20260709 12:25:10.263617  5148 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:10.242156  9865 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.257838  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:25:10.271693  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.252434  5070 rpcz_store.cc:276] 0709 12:25:09.155613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.155742 (+   129us) service_pool.cc:224] Handling call
0709 12:25:10.223972 (+1068230us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b540047e354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:10.224146 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182283136}
I20260709 12:25:10.264746  5141 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 9.
I20260709 12:25:10.278959  9860 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:10.282272  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:10.288228  9879 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:10.279062  9877 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:10.289179 10004 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:25:10.289368  9859 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.291807  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:10.294281  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.279804  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.293301  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.280586  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.294605  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:10.247233  5939 rpcz_store.cc:276] 0709 12:25:09.104355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.104556 (+   201us) service_pool.cc:224] Handling call
0709 12:25:09.142823 (+ 38267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.154821 (+ 11998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.154861 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.158679 (+  3818us) log.cc:844] Serialized 26 byte log entry
0709 12:25:09.322295 (+163616us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:09.322341 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.322429 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.152544 (+830115us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:10.206911 (+ 54367us) raft_consensus.cc:1685] finished
0709 12:25:10.206933 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.223909 (+ 16976us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":945,"raft.queue_time_us":16098,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":8783,"spinlock_wait_cycles":7810560,"thread_start_us":160344,"threads_started":1,"wal-append.queue_time_us":583238}
I20260709 12:25:10.298424  5941 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:25:10.307337  5941 pending_rounds.cc:85] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 9
I20260709 12:25:10.305321  5146 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:10.307639  5941 pending_rounds.cc:107] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:25:10.307900  5941 raft_consensus.cc:2889] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:10.299782  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.300835  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.308655  5147 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:10.308740  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.309576  5149 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:10.305190  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.313803  5142 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:10.303476  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.317978  5148 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:10.261580  9947 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:10.312966  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.312526  5115 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:10.319742  9859 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:10.324925  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.325956  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.326769  5150 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 16: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:10.327963  5150 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:10.329272  5150 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:10.253759  9978 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:10.331830  9978 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:10.332432  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1492) took 1319 ms. Trace:
I20260709 12:25:10.281260  4894 catalog_manager.cc:5697] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.333082  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:10.333875  4894 catalog_manager.cc:5697] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:10.325245  9947 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:10.335520  4894 catalog_manager.cc:5697] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 9 to 11, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 11 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.332868  5215 rpcz_store.cc:276] 0709 12:25:09.013245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.016064 (+  2819us) service_pool.cc:224] Handling call
0709 12:25:09.016731 (+   667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.886699 (+869968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.332176 (+445477us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b5400410354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:10.332363 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1974232064}
I20260709 12:25:10.325999  5146 raft_consensus.cc:2410] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:25:10.334605  5945 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:25:10.334600  5068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 1222) took 1441 ms (client timeout 1875 ms). Trace:
I20260709 12:25:10.339033  5217 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:10.299098  9679 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:10.343569  5216 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:10.342777  5149 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:10.247109  9991 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:10.346031  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:10.347213  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:10.362185  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:10.362550  5142 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:10.362936  5946 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:10.351032  5147 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:10.347601  5041 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:25:10.347681  5068 rpcz_store.cc:269] 0709 12:25:08.893280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.955401 (+ 62121us) service_pool.cc:224] Handling call
0709 12:25:10.334395 (+1378994us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540058a054. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:10.334535 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2936386944}
I20260709 12:25:10.349188  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:10.357779  5145 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:10.366034  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.368070  5150 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:10.367959  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:10.369563  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:10.301777  9865 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:10.371739  5149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0"
dest_uuid: "ceec344dd00f40c18ef58414ce147016"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49182
I20260709 12:25:10.388203  5149 raft_consensus.cc:606] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:10.388602  5149 raft_consensus.cc:614] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:10.345548 10003 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:10.373636  9968 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:10.374302  9863 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:10.374899  9996 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:10.375502  9978 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:10.377049  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:10.412963  5942 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:10.382704  9848 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:10.383248  9843 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:10.342964  9895 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:10.372009  9898 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:10.393399  9896 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:10.394527  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1603) took 1146 ms. Trace:
I20260709 12:25:10.402714  5143 raft_consensus.cc:2410] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:25:10.407034  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.435714  5072 rpcz_store.cc:276] 0709 12:25:09.248354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.248859 (+   505us) service_pool.cc:224] Handling call
0709 12:25:09.254800 (+  5941us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.746431 (+491631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.746459 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.746991 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.007253 (+260262us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:10.007315 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.394069 (+386754us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54006a2554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:10.394123 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.394175 (+    52us) raft_consensus.cc:1685] finished
0709 12:25:10.394188 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.394460 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2658,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":3782,"spinlock_wait_cycles":1124715392,"thread_start_us":210673,"threads_started":1,"wal-append.queue_time_us":321583}
I20260709 12:25:10.348341  9930 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.408030  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:10.306404  9877 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:10.415782  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:10.443807  5947 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:10.419466  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:10.428690  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:10.441391  5150 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:10.448648  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:10.449630  5069 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.363726  9932 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:25:10.453920  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.416788  9848 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:10.460433  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:10.422899  9843 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:10.471351  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:10.474009  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.411293  9938 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:25:10.481920  5189 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:10.484587  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39878 (request call id 1211) took 1763 ms. Trace:
I20260709 12:25:10.439531  9987 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:10.484992  5948 rpcz_store.cc:276] 0709 12:25:08.721467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.721641 (+   174us) service_pool.cc:224] Handling call
0709 12:25:08.944265 (+222624us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b5800003998. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:25:10.484406 (+1540141us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54002422d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:10.484525 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496866048}
I20260709 12:25:10.463523  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:10.494683  5073 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:25:10.442952  9896 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:10.482483  5946 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:10.484912  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:10.488466  5221 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:25:10.497377  5221 pending_rounds.cc:85] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 6
I20260709 12:25:10.497781  5221 pending_rounds.cc:107] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:25:10.498081  5221 raft_consensus.cc:2889] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:10.506693  5189 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:10.512416  5940 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:10.519590  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.466559  9848 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:10.519636  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.520429  5216 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:10.528872  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.478844  9843 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:10.450695  9930 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:10.358053  4894 catalog_manager.cc:5697] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 5 to 8, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 8 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.546500  5899 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:10.547668  9927 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:10.548100  9927 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.548408  9483 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:10.548849  9483 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:10.491516  9932 consensus_queue.cc:1243] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:10.527753  9894 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:10.503515  9942 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:10.553098  5074 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:10.568900  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 178107us
W20260709 12:25:10.558836  9949 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.002s	sys 0.006s
I20260709 12:25:10.566731  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:10.548475  9927 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:10.574733  5141 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:10.577436  5069 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:10.510550  9745 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:25:10.583925  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.531922  9867 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:25:10.604110  5074 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:10.604005  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:10.538021  5942 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:10.536665  5947 raft_consensus.cc:2468] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:10.612813  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.613042  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:10.574573  9949 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.003s	sys 0.006s
I20260709 12:25:10.614223  5072 raft_consensus.cc:2368] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11.
I20260709 12:25:10.624287  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:25:10.604856  9856 consensus_meta.cc:287] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.000s
I20260709 12:25:10.622700  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:10.631237  5074 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:10.549750  9483 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:10.633270  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.561856  9904 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:25:10.634793  5150 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 16: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:10.605209  9991 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:10.651203  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.655794  5143 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:10.611480  9942 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:10.657974  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:10.658676  5072 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:10.659816  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.617523  9988 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:25:10.673413  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.625257  9888 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:25:10.675796  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:10.626029  9894 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:25:10.662585  9978 raft_consensus.cc:697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:10.695463  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.696422  5945 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:10.696894  5067 raft_consensus.cc:2410] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:10.691226  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.698297  5143 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:25:10.713778  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.638933  9856 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:10.725703  5067 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:10.715233  5147 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18. Ops: [14.6-17.8]
I20260709 12:25:10.732645  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.678946  9864 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:10.661648  9904 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:25:10.694299 10015 raft_consensus.cc:3055] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:10.744439  5072 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 9.
I20260709 12:25:10.734035  5143 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:10.735916  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:10.737452  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.739908  5074 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:10.685490  9958 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:10.684859  9806 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.749413  9806 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:10.747382  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.752969  9877 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:10.754355  9894 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:10.754940  9894 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:10.755928  9894 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:10.756950  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.758735  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.692804  9895 raft_consensus.cc:697] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:10.752662  5072 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:10.715737  9865 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:10.773286  9658 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.003s
W20260709 12:25:10.763687  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:10.766623 10007 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
W20260709 12:25:10.767347 10012 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 12:25:10.768244 10009 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 12:25:10.768944 10010 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:25:10.769912  9930 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:10.684019  9892 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:25:10.775509  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:10.813383  5067 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:10.776227  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.814640  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.815784  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:10.777472  5900 leader_election.cc:336] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.912s (SENT)
I20260709 12:25:10.769335  9941 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:25:10.729735  9799 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:10.737555  9867 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.789973  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:10.758596  9864 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:10.832021  5072 raft_consensus.cc:2410] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:10.737861  9996 raft_consensus.cc:3055] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:10.799018  5143 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:10.800348  5071 raft_consensus.cc:2410] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:25:10.794668  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.802868  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.744091 10015 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:10.844058  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 929) took 1984 ms. Trace:
I20260709 12:25:10.844179  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:10.808646  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 59410.3us
I20260709 12:25:10.846235  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:10.810459  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 404822us
W20260709 12:25:10.789155 10006 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.279s	user 0.000s	sys 0.002s
I20260709 12:25:10.816639  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:10.778080  9942 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:10.851610  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.852568  5072 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:10.740852  9978 consensus_queue.cc:237] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:10.856305  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.857329  5142 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:10.860649  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.867053  5147 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 16: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:10.832835  9799 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:25:10.852944  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 177248us
I20260709 12:25:10.876893  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:10.877100  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:10.877660  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:10.826617  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.884857  5073 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:10.851271  9948 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:10.862850  5188 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:10.828079  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.837419  5074 raft_consensus.cc:2368] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11.
I20260709 12:25:10.836076 10002 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.895413  5188 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:10.844568  5218 rpcz_store.cc:276] 0709 12:25:08.859822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.859978 (+   156us) service_pool.cc:224] Handling call
0709 12:25:08.900019 (+ 40041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.049354 (+1149335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.843756 (+794402us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400164c54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:10.843984 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2608383872}
I20260709 12:25:10.834638  9996 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:10.887956  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.889230  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.890602  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:10.881062  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.908689  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.900153  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.926258  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.894922  5075 raft_consensus.cc:2410] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:10.926635  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:10.926978  5142 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:10.927347  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:10.843278  9867 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:10.924983  5147 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:10.934870  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.886861  9799 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:10.937842  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.927963  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.942209  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:10.900878  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:10.944842  5067 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:25:10.945824  5142 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.897264  9991 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:10.931881  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.935484  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.933790 10035 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:10.899016  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.949512  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:10.949831  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 69987.7us
I20260709 12:25:10.950085  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.898559  9904 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:10.949580  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.969825  5073 raft_consensus.cc:2410] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:25:10.929281  9974 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:10.972028  5147 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:10.956277  5219 raft_consensus.cc:2410] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:25:10.898788 10019 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.965821  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.970961  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.978269  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:10.934392  9892 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:25:10.929994  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 152581us
W20260709 12:25:10.953368  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.992223  5067 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
W20260709 12:25:10.967258 10009 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:10.999011  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:10.932268  9848 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:10.880036  9895 consensus_queue.cc:237] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:11.006779  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 50156us
I20260709 12:25:10.957520 10035 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.041742  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:10.997614  9679 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.651s	user 0.009s	sys 0.004s
I20260709 12:25:11.061899  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:10.992033  9856 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:11.086470  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:11.046466  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:10.999807  5147 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:11.087415  9892 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:11.088254  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:11.089767  9877 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:10.918641 10002 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:10.961465 10033 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:11.051427 10020 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.064334  9941 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:11.092816  9941 raft_consensus.cc:697] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:11.052985 10014 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:11.093796  9904 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:25:11.094074  9941 consensus_queue.cc:237] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:11.054584  5115 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:11.115144  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 60649.3us
I20260709 12:25:11.116320  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:11.054807  9867 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:11.117359  9679 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:11.117995 10003 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.118155  9679 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:11.118860 10003 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:11.119729  9679 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:11.121840 10014 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:11.124140  5115 leader_election.cc:400] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16.
I20260709 12:25:11.126111  5115 leader_election.cc:403] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:25:11.127627  9934 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16.
I20260709 12:25:11.126549 10003 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:11.043818  5142 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:25:11.064181  5189 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:11.136307  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 72273.7us
I20260709 12:25:11.062577 10035 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:11.074057  5144 raft_consensus.cc:2410] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:25:10.991339  9991 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:11.078961  9864 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:11.083642 10027 log.cc:927] Time spent T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:25:10.996153 10019 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:11.001776  9996 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:10.958025 10015 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:10.996016 10042 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:11.052454 10043 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:25:11.053419 10038 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:25:11.089092  5147 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 15: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:25:11.140661  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 94292.5us
I20260709 12:25:11.141638  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.142721  5940 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:11.138677  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.148757 10003 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:11.074106  9843 consensus_meta.cc:287] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.004s
I20260709 12:25:11.189568  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.188663  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:11.187727  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:11.191905  5944 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:11.146431 10041 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:11.167200  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.193377  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.211905  5940 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:11.213078  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.199419  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.213927  5944 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:11.088958  9892 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:11.215976  5947 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:11.214298  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.213754  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.221312  5147 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:11.222700  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.224149  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.224936  5147 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:11.226225  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:11.228284  5147 raft_consensus.cc:2393] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:11.230538  9991 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:11.234108  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.264700  5940 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:11.235373  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.236186  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.196318  9930 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:25:11.243299 10019 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:11.212082  9843 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:11.274986  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.275701  5940 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:11.212344 10003 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:11.248652  9927 consensus_meta.cc:287] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.639s	user 0.011s	sys 0.002s
I20260709 12:25:11.286260  9843 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:11.286135  9927 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:11.267215 10049 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:25:11.287648 10003 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:11.288158  5175 heartbeater.cc:646] Failed to heartbeat to 127.1.123.126:38285 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.1.123.126:38285 timed out after 15.000s (SENT)
I20260709 12:25:11.288795  9927 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:11.249233  9848 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:11.092197 10020 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:11.287696 10049 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:11.290241 10049 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:11.291923 10020 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:11.292233  5150 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:11.160295 10034 raft_consensus.cc:2804] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:11.293478 10034 raft_consensus.cc:697] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:11.158658  9875 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:11.257973  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.234831 10025 log.cc:927] Time spent T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.459s	user 0.000s	sys 0.003s
W20260709 12:25:11.260699  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 69323.6us
W20260709 12:25:11.295794  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.296617  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:11.297713  5067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:11.298383  5067 raft_consensus.cc:606] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:11.263357  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:11.270793  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.299325  5944 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:11.301843  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.302502  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:11.260388  9872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.053s	sys 0.062s
W20260709 12:25:11.308696  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.311108  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.311679  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:11.315708  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:11.273571  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.320926  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:11.345723  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.347532  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:11.306303  9875 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:11.322322  5940 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:11.323537  5947 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:11.294903 10034 consensus_queue.cc:237] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:11.325246 10003 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:11.358349  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.341086  9799 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:11.309754 10020 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:11.196097 10042 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:11.291422 10049 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:11.334730  9679 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:11.337093  9863 raft_consensus.cc:3055] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:11.339674  9927 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:11.340060  9806 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:11.341787  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.342047  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:11.265527  9949 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:11.390954  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48987us
I20260709 12:25:11.389075  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:11.397599  5218 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:25:11.400367  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:11.400734  5222 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:11.344002  5900 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:11.350018 10032 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:11.307011  9864 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:11.328256  9896 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.798s	user 0.010s	sys 0.002s
I20260709 12:25:11.359048 10003 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.384903  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:11.345425 10047 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.401759  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:11.409447  5215 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:11.415616  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:11.417666  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.422230  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:25:11.402066  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 58193.1us
I20260709 12:25:11.381623  5150 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:11.407825  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:25:11.383926  9968 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.910s	user 0.014s	sys 0.010s
I20260709 12:25:11.440011  5067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:11.388440  9863 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:11.437045  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.410213  9895 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:11.414853 10003 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:11.440361  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:11.408423  9934 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:11.442750  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:11.440650  5067 raft_consensus.cc:606] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:11.457679  5147 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:11.427204  9679 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:11.438764  9968 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.015s	sys 0.011s
I20260709 12:25:11.416867  9927 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:11.424695  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:11.464103  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.469336  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44774.3us
I20260709 12:25:11.436831  9864 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:11.423781 10049 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:11.414387  9896 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:11.414654  9949 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:11.463060  9991 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:11.457363  9934 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.438426 10020 consensus_queue.cc:1059] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:25:11.451776  9872 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 12:25:11.492266  5943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39870 (request call id 1118) took 1564 ms (client timeout 1783 ms). Trace:
I20260709 12:25:11.476683 10043 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:11.498081  5943 rpcz_store.cc:269] 0709 12:25:09.928300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.928418 (+   118us) service_pool.cc:224] Handling call
0709 12:25:11.492059 (+1563641us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400007b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:11.492198 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651943296}
I20260709 12:25:11.426613 10047 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:11.474877 10039 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.500805  5147 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:11.505679  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:11.506273  5147 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:11.507652  5067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:11.508556  5067 raft_consensus.cc:606] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:11.508980  5067 raft_consensus.cc:614] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:11.493709  9991 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.477705  9988 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 12:25:11.500913  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 54431.3us
I20260709 12:25:11.495571  9934 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:11.516752  5114 leader_election.cc:336] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.665s (SENT)
I20260709 12:25:11.519058  5114 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:11.503571  9872 consensus_queue.cc:1243] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:11.486810 10059 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 12:25:11.529403  5114 leader_election.cc:336] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.691s (SENT)
I20260709 12:25:11.535168  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:11.529853  9745 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:11.518136  9991 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:11.539117  9745 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:11.524775  9988 consensus_queue.cc:1243] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:11.540547 10047 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:11.543179 10042 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:11.543486  9705 raft_consensus.cc:3055] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:11.559784  5147 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:11.561679  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:11.568799  5114 leader_election.cc:336] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.783s (SENT)
I20260709 12:25:11.535734 10068 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:11.562362  9968 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:11.574091  5075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:11.514005 10039 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:11.582572  5075 raft_consensus.cc:606] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:11.582842  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.508747  9863 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:11.587954  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:11.534868 10035 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:11.576633 10053 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.571517  9799 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:11.575318  9705 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:11.527233  9927 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:11.594743  9898 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.059s	user 0.009s	sys 0.014s
W20260709 12:25:11.599869 10060 log.cc:927] Time spent T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:25:11.600647 10056 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 12:25:11.602109 10055 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:25:11.602991  9705 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:11.538185  9864 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:11.605381  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.601333 10062 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:25:11.609875  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.611618  9932 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:25:11.613729  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.594569  9863 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:11.597358 10053 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:11.618954 10035 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:11.619854  5144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1318) took 1448 ms (client timeout 1922 ms). Trace:
W20260709 12:25:11.647687  5144 rpcz_store.cc:269] 0709 12:25:10.170930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.171062 (+   132us) service_pool.cc:224] Handling call
0709 12:25:11.619595 (+1448533us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b5400582854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:11.619770 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2073468672}
I20260709 12:25:11.621564  5223 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:25:11.623188  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.623684  5075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:11.650555  5075 raft_consensus.cc:606] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:11.623960  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.629047  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:11.652176  5940 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:11.652473  5900 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:11.630678  5148 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:25:11.622303  9898 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.091s	user 0.009s	sys 0.014s
I20260709 12:25:11.634323  5943 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:25:11.636765  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:11.605476  9745 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:11.646715  9705 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:11.660821  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:11.662477  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:11.663333  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:11.664243  5144 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:11.647176  9879 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:11.665107  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:11.663369  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:11.669587  5144 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 16: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:11.645628  9932 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:11.704397  5114 leader_election.cc:336] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.875s (SENT)
I20260709 12:25:11.704978  5114 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:11.711964  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.736732  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 167067us
I20260709 12:25:11.681624  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:11.742338  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 60824.8us
I20260709 12:25:11.743472  5900 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:11.654357  9896 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:11.790184  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:11.830232  5217 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:11.840943  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 136605us
W20260709 12:25:11.863196  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 119816us
W20260709 12:25:11.869889  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.821635  9949 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:11.822386 10039 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:11.846517 10059 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:11.881196  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:11.867930  9896 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:11.891618  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:11.865450 10075 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:11.914494  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:11.935861  5190 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:11.937433  5190 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:11.905517 10034 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:11.942042  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:11.942659  9941 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:11.943657 10035 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:11.912405  9799 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:11.944502  9859 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:11.944803 10053 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:11.942834  9894 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:11.914580 10075 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:11.946533  9975 raft_consensus.cc:3055] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:11.967031  5218 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:11.970126  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 413766us
I20260709 12:25:11.969826  9975 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:11.947604 10034 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:11.969621 10032 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:11.959720  9941 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:11.973384 10053 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:25:11.993368  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:11.986285  9975 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:11.962080 10035 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:11.998163 10035 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:11.999825  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1485) took 1721 ms. Trace:
I20260709 12:25:11.981837  9898 consensus_queue.cc:1059] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:25:11.962900  9799 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.988723  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:12.002260  9975 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:11.992720  9941 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:12.004822  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:12.006138 10003 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.000s
I20260709 12:25:12.006736 10003 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:11.970894  9806 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.008098  9806 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:12.008615  5145 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 18: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:12.009910  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:11.993233  5945 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:25:11.995213  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.971174 10020 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.906764  9932 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:11.996440  5943 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 14: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:12.040876  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1601) took 2793 ms. Trace:
I20260709 12:25:12.010231  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:12.000278  5149 rpcz_store.cc:276] 0709 12:25:10.278558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.406295 (+127737us) service_pool.cc:224] Handling call
0709 12:25:10.406697 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.536239 (+1129542us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b18000b9c80. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a80e34bc 00007fd8a543148b 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8a8533060 00007fd8a8533bfb 00007fd8a853e64b
0709 12:25:11.536388 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.536415 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.558194 (+ 21779us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.640467 (+ 82273us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:11.646671 (+  6204us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.646907 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.646991 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.999459 (+352468us) raft_consensus.cc:1685] finished
0709 12:25:11.999570 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.999757 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12423,"prepare.queue_time_us":32860,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":219768,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":9637,"spinlock_wait_cycles":7736576,"thread_start_us":290840,"threads_started":2,"wal-append.queue_time_us":365483}
I20260709 12:25:12.012593  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 961) took 1235 ms. Trace:
I20260709 12:25:12.028090  5143 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:12.026454  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1332) took 1837 ms. Trace:
W20260709 12:25:12.046941  5071 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 980) took 2947 ms (client timeout 1562 ms). Trace:
I20260709 12:25:12.012045  5223 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:25:12.016166  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.094287  5150 rpcz_store.cc:276] 0709 12:25:10.776750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.086141 (+309391us) service_pool.cc:224] Handling call
0709 12:25:12.012396 (+926255us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54004fa954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:12.012531 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471420672}
I20260709 12:25:12.096431  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 957) took 1320 ms. Trace:
W20260709 12:25:12.092586  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.097391  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.096366  5071 rpcz_store.cc:269] 0709 12:25:09.099054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.099201 (+   147us) service_pool.cc:224] Handling call
0709 12:25:12.046502 (+2947301us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400347454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:12.046774 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3870326656}
I20260709 12:25:12.102429  5069 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:11.988231 10034 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:12.118070  5149 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:12.118985  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 958) took 1343 ms. Trace:
I20260709 12:25:12.119371  5149 rpcz_store.cc:276] 0709 12:25:10.775901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.117424 (+1341523us) service_pool.cc:224] Handling call
0709 12:25:12.117821 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.118650 (+   829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.118915 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:12.096015  5141 rpcz_store.cc:276] 0709 12:25:10.188772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.341108 (+152336us) service_pool.cc:224] Handling call
0709 12:25:10.362279 (+ 21171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.575382 (+213103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.026057 (+1450675us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54003ae3d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:12.026348 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466765568}
I20260709 12:25:12.091449  5076 rpcz_store.cc:276] 0709 12:25:09.247690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.249451 (+  1761us) service_pool.cc:224] Handling call
0709 12:25:10.224159 (+974708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.224553 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.224580 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.228997 (+  4417us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.295854 (+ 66857us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:10.295901 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.295963 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.951184 (+655221us) raft_consensus.cc:1685] finished
0709 12:25:10.951210 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.040558 (+1089348us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b540047e350. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:12.040780 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6022,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9283,"spinlock_wait_cycles":2137701632,"thread_start_us":66468,"threads_started":1,"wal-append.queue_time_us":214960,"wal-append.run_cpu_time_us":4326,"wal-append.run_wall_time_us":1525284}
I20260709 12:25:11.971652 10075 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:12.105108  5898 connection.cc:256] RPC call timeout handler was delayed by -1.1832014420001542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
W20260709 12:25:12.109820  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.110387  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 144131us
W20260709 12:25:12.094352  5114 leader_election.cc:336] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.953s (SENT)
I20260709 12:25:12.125239  5114 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:12.042582 10011 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 1.409s	user 0.004s	sys 0.000s
I20260709 12:25:12.100905  5143 rpcz_store.cc:276] 0709 12:25:10.775602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.027504 (+1251902us) service_pool.cc:224] Handling call
0709 12:25:12.027862 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.095999 (+ 68137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.096347 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:11.987896 10032 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:25:12.036152 10029 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 1.262s	user 0.004s	sys 0.000s
I20260709 12:25:12.002267  9799 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:12.135485  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:12.026635 10013 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:12.009912  9898 raft_consensus.cc:3055] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:25:12.115540  9991 consensus_meta.cc:287] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.543s	user 0.016s	sys 0.017s
I20260709 12:25:12.137912  5149 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:12.038956  9875 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:12.017179 10003 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:12.010281  9806 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:12.148195  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1344) took 1011 ms. Trace:
W20260709 12:25:12.143460  5941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.005s	sys 0.015s
W20260709 12:25:12.178082  5941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.005s	sys 0.016s
I20260709 12:25:12.137411 10032 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:12.175105  5141 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15. Ops: [11.8-14.10]
I20260709 12:25:12.041903 10088 consensus_queue.cc:1059] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 12:25:12.178721  5190 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:12.177886  5149 rpcz_store.cc:276] 0709 12:25:11.137177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.136943 (+999766us) service_pool.cc:224] Handling call
0709 12:25:12.137584 (+   641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.147733 (+ 10149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.148034 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:12.116134 10020 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:12.042199  9968 consensus_queue.cc:1059] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 12:25:12.145569  9947 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:12.145313  9898 raft_consensus.cc:740] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:12.190403  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:12.185179  5149 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:12.192485  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1350) took 1050 ms. Trace:
I20260709 12:25:12.180135  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1345) took 1042 ms. Trace:
I20260709 12:25:12.193837  5141 rpcz_store.cc:276] 0709 12:25:11.140009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.141791 (+1001782us) service_pool.cc:224] Handling call
0709 12:25:12.174661 (+ 32870us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:12.179704 (+  5043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.180048 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:12.146929  9934 consensus_meta.cc:287] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.002s
I20260709 12:25:12.145777 10013 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:12.192943  5149 rpcz_store.cc:276] 0709 12:25:11.142509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.184601 (+1042092us) service_pool.cc:224] Handling call
0709 12:25:12.184971 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.192110 (+  7139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.192407 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:12.197647  9284 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:25:12.197273  9863 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.000s
I20260709 12:25:12.185567  9879 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:25:12.203208  9888 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:12.205798 10020 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:12.206274  9898 consensus_queue.cc:260] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:12.207849 10003 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:12.209138  9947 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:12.211360 10049 consensus_meta.cc:287] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.732s	user 0.012s	sys 0.000s
I20260709 12:25:12.255127 10014 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:12.260915  9284 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:12.270924 10014 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:12.271286  9284 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:12.271983 10014 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:12.273175 10014 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:12.177870  9991 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:12.215965  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1238) took 1070 ms. Trace:
I20260709 12:25:12.217862  9968 consensus_queue.cc:1243] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:12.219476  9996 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:12.239598  5147 raft_consensus.cc:2410] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:25:12.239257  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 144931us
I20260709 12:25:12.230707  9958 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:12.280839  5114 leader_election.cc:336] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 2.190s (SENT)
I20260709 12:25:12.241830  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 960) took 1465 ms. Trace:
I20260709 12:25:12.281747  5150 rpcz_store.cc:276] 0709 12:25:10.776485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.114238 (+1337753us) service_pool.cc:224] Handling call
0709 12:25:12.114833 (+   595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.241275 (+126442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.241317 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:12.241368 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:12.241406 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.241508 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.241741 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":126057,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":25171,"thread_start_us":125457,"threads_started":1}
I20260709 12:25:12.283736  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:12.284524  5150 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:12.243453  5141 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 17. Current term is 18. Ops: []
W20260709 12:25:12.285776  5114 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 2.226s (SENT) [suppressed 47 similar messages]
I20260709 12:25:12.195703  9875 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:12.249821  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.254168 10088 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:12.274333  5948 rpcz_store.cc:276] 0709 12:25:11.145420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.145601 (+   181us) service_pool.cc:224] Handling call
0709 12:25:11.145903 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.261318 (+115415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.261367 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.267618 (+  6251us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.418349 (+150731us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:11.418417 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.215446 (+797029us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540000a854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:12.215512 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.215574 (+    62us) raft_consensus.cc:1685] finished
0709 12:25:12.215587 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.215882 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7600,"raft.queue_time_us":8308,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7501,"spinlock_wait_cycles":246561664,"thread_start_us":96721,"threads_started":1,"wal-append.queue_time_us":115496}
I20260709 12:25:12.289054  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1494) took 1145 ms. Trace:
I20260709 12:25:12.241168  9888 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.268115  9679 raft_consensus.cc:3055] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:12.230481  9934 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:12.289835  5141 rpcz_store.cc:276] 0709 12:25:11.143784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.194819 (+1051035us) service_pool.cc:224] Handling call
0709 12:25:12.243196 (+ 48377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.288241 (+ 45045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.288916 (+   675us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:12.256017  9898 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:12.261924 10094 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.255488 10092 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.301425  5190 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:12.296841  9888 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:12.310505  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:25:12.304610 10043 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.011s	sys 0.010s
I20260709 12:25:12.296558  9679 raft_consensus.cc:740] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:12.290522  9875 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:12.302127  9284 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:12.288487 10034 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:12.323729  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:12.292091  9941 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:12.327885 10035 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:12.330438  9875 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:12.374116  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1331) took 2185 ms. Trace:
I20260709 12:25:12.311620  9894 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:12.335135  5141 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 16: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:12.374578  5148 rpcz_store.cc:276] 0709 12:25:10.188476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.318901 (+130425us) service_pool.cc:224] Handling call
0709 12:25:11.620760 (+1301859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.653265 (+ 32505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.373903 (+720638us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400582850. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:12.374054 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2992877440}
I20260709 12:25:12.376040  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:12.376694  5215 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:12.377336  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 950) took 1642 ms. Trace:
I20260709 12:25:12.339988 10002 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:12.341899  9888 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:12.351162  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:12.352056  9799 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:12.356657 10080 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:12.357506 10039 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.398200 10039 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:12.401192 10039 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:12.292016  9947 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:12.361733  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:12.405901  5224 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:12.407147  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:12.363884  5219 raft_consensus.cc:1217] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Deduplicated request from leader. Original: 9.6->[12.7-12.7]   Dedup: 12.7->[]
I20260709 12:25:12.365195  9934 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:12.321996 10046 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
W20260709 12:25:12.411917 10018 log.cc:927] Time spent T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 1.557s	user 0.003s	sys 0.000s
I20260709 12:25:12.412170 10046 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:12.412595 10046 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:12.360996  9745 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:12.413677 10046 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:12.366400  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 376707us
W20260709 12:25:12.368453 10083 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.396s	user 0.002s	sys 0.000s
I20260709 12:25:12.369872 10034 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:12.418808  5219 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:25:12.370314  5217 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:12.370895 10035 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:12.371374 10032 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:12.424175  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:12.424490  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:12.425688  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:12.426759  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:12.318030 10043 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.011s	sys 0.012s
I20260709 12:25:12.380136  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:12.373853 10106 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:12.385589  9679 consensus_queue.cc:260] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:12.387415  9859 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:12.388115  9941 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:12.311293 10092 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:12.388212  9864 consensus_queue.cc:1059] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:25:12.388890  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.440850  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.440878  5223 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:12.358215  9896 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:12.377877  5144 rpcz_store.cc:276] 0709 12:25:10.734874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.867399 (+1132525us) service_pool.cc:224] Handling call
0709 12:25:11.886828 (+ 19429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:12.115128 (+228300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.115175 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:12.115905 (+   730us) log.cc:844] Serialized 52 byte log entry
0709 12:25:12.308523 (+192618us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:12.318863 (+ 10340us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.319218 (+   355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.319288 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.377109 (+ 57821us) raft_consensus.cc:1685] finished
0709 12:25:12.377129 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.377275 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11872,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":59260,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"spinlock_wait_cycles":2334080,"thread_start_us":338277,"threads_started":2,"wal-append.queue_time_us":223370}
I20260709 12:25:12.394450  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:12.442344  5221 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:12.307801 10094 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:12.372493  9991 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:12.379340  9894 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.372676 10013 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:12.463021 10013 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:12.464517 10075 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:12.427822  5942 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:12.428025  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:12.504488  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:12.505288  5150 raft_consensus.cc:2410] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:25:12.430415  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:12.485960  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.506489  5224 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 19: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:12.431442  5222 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:12.432282  5218 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:12.450496  9864 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:12.430648 10043 raft_consensus.cc:697] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:25:12.453586  5114 leader_election.cc:336] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.704s (SENT)
W20260709 12:25:12.513610  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 60097.9us
I20260709 12:25:12.513648  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:12.430594 10070 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.429776  9895 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:12.462030  9894 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:12.468045 10032 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:12.438658 10106 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:12.470225 10080 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:12.471395  9888 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:12.420539  5215 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:12.438395 10034 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:25:12.476397  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1225) took 2132 ms. Trace:
I20260709 12:25:12.477743  9483 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:12.525172  5215 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:12.526979  9745 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.527668  9745 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:12.516785 10031 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 1.743s	user 0.000s	sys 0.005s
I20260709 12:25:12.479377  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 953) took 1704 ms. Trace:
I20260709 12:25:12.529721  5147 rpcz_store.cc:276] 0709 12:25:10.774581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.587662 (+813081us) service_pool.cc:224] Handling call
0709 12:25:12.479105 (+891443us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b540014da54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:12.479296 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495746816}
I20260709 12:25:12.482910  9932 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:12.482996  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:12.530798  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:12.530841  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:12.531890  5940 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:12.483853  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.524668  5941 rpcz_store.cc:276] 0709 12:25:10.343906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.344459 (+   553us) service_pool.cc:224] Handling call
0709 12:25:11.493121 (+1148662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.493333 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:11.573081 (+ 79748us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.371556 (+798475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:12.371635 (+    79us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:12.371691 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.371772 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.476161 (+104389us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400007b50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:12.476295 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":192943,"prepare.queue_time_us":32686,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":193523,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":2640845568,"thread_start_us":675468,"threads_started":2,"wal-append.queue_time_us":717442}
I20260709 12:25:12.509244  9864 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:12.538625  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:12.543504  5947 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:12.485919  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:12.486958  5067 raft_consensus.cc:1240] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:12.487614  9896 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:12.488385  9859 raft_consensus.cc:3055] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:12.490471  5146 raft_consensus.cc:2410] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:25:12.485064  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.495921 10094 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:12.454759  9679 raft_consensus.cc:3060] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:12.454434  9934 consensus_queue.cc:1175] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "90e5220a62ed42fe83c82d0302d18fac" responder_term: 13 status { last_received { term: 13 index: 8 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: [12.7-12.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:12.511130  5221 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
W20260709 12:25:12.512449  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.576805  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.506739  5074 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:12.515661  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:12.514406  5219 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
W20260709 12:25:12.516769  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 146820us
I20260709 12:25:12.519721  9895 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:12.520573  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.585837  5042 connection.cc:256] RPC call timeout handler was delayed by -1.0605653530001291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
I20260709 12:25:12.529174  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:12.586308  5042 connection.cc:256] RPC call timeout handler was delayed by -1.0533653530001175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
I20260709 12:25:12.531265  5071 raft_consensus.cc:3060] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:12.483239  5076 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:12.587719  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:12.588505  5076 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:12.532202  5147 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:12.533659  5075 raft_consensus.cc:2410] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:12.436277  9865 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:12.484316  5145 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:25:12.537987  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:12.595875  5941 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:25:12.538836  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:12.546325  5945 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:12.593709  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:12.545519  5217 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:12.559749  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:12.599858  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.600772  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.548207  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.582461  5144 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:12.561439  9843 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:12.593878  5221 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:12.595289  5145 pending_rounds.cc:85] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 6
I20260709 12:25:12.560792  9875 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.607550  5145 pending_rounds.cc:107] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:25:12.607920  5145 raft_consensus.cc:2889] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:12.608364  5940 raft_consensus.cc:2410] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:25:12.602429  5067 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 11.
I20260709 12:25:12.610481  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.613056  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:12.557803  9859 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:12.600826  5076 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:12.605337  5219 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:12.616074  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:12.621819 10043 consensus_queue.cc:237] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:12.623188 10106 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:12.603390  9895 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:12.624596  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:12.675925  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:12.676622  5941 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:12.625806 10075 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:12.625926  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.606066  9991 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.630849  9745 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:12.632583  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:12.633997  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:12.634822 10032 consensus_queue.cc:1059] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:12.605834  9419 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.636253 10080 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:12.636742  9867 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:12.638695  9843 raft_consensus.cc:697] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:12.639183  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:12.641184 10109 log.cc:927] Time spent T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:25:12.642635  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56170 (request call id 1353) took 1499 ms. Trace:
I20260709 12:25:12.540452  9850 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:25:12.544688  9978 consensus_queue.cc:1059] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 12:25:12.651618  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.652350  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:12.712771  5071 raft_consensus.cc:2468] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:12.713172  5939 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:12.654312  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 1015) took 1881 ms. Trace:
I20260709 12:25:12.656361  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 1017) took 1883 ms. Trace:
W20260709 12:25:12.659722  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.714296  5072 rpcz_store.cc:276] 0709 12:25:10.773252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.950899 (+177647us) service_pool.cc:224] Handling call
0709 12:25:12.656037 (+1705138us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b5400328f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:12.656282 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1068754816}
I20260709 12:25:12.714184  5069 rpcz_store.cc:276] 0709 12:25:10.772684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.776652 (+  3968us) service_pool.cc:224] Handling call
0709 12:25:12.049600 (+1272948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.104118 (+ 54518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.654046 (+549928us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400347450. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:12.654245 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2926590336}
I20260709 12:25:12.716712  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:12.717449  5947 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:12.661334  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:12.718497  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:12.720109  5115 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:12.661813  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:12.520555 10070 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:12.667908  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.670004  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1604) took 2282 ms. Trace:
W20260709 12:25:12.726248  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 65064.5us
I20260709 12:25:12.725198  5068 rpcz_store.cc:276] 0709 12:25:10.387911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.390016 (+  2105us) service_pool.cc:224] Handling call
0709 12:25:10.418224 (+ 28208us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.633569 (+215345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.633612 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.644194 (+ 10582us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.715740 (+ 71546us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:10.715859 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.716009 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.502482 (+1786473us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:12.616844 (+114362us) raft_consensus.cc:1685] finished
0709 12:25:12.616868 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.669918 (+ 53050us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":139715,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":6679,"spinlock_wait_cycles":12525952,"thread_start_us":259098,"threads_started":2,"wal-append.queue_time_us":121188}
I20260709 12:25:12.681165  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:12.684698  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50793.9us
I20260709 12:25:12.729647  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.729842  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:12.730033  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:12.683615  5940 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:12.666278  9875 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:12.583880  9888 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:25:12.698293  5219 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:12.736243  5188 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:12.702248  5146 rpcz_store.cc:276] 0709 12:25:11.143264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.425095 (+281831us) service_pool.cc:224] Handling call
0709 12:25:12.642374 (+1217279us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54004fae54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:12.642556 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2404192000}
I20260709 12:25:12.709250  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:12.720368  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.739722  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.740200  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:12.661558  9894 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:12.726140  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:12.725175  9927 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.006s	sys 0.029s
I20260709 12:25:12.730378  5069 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:12.691021  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:12.750586  5945 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:25:12.750754  5943 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:12.749915  5188 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:12.734459  5940 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:12.735946  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.752192  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:12.705513  5900 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.718s (SENT)
I20260709 12:25:12.698449  9867 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:12.711128  9799 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:12.678889  5147 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:12.746464  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:12.729341 10002 raft_consensus.cc:3055] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:12.696908  9991 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:12.736323  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:12.751550  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:12.693554  5144 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:12.753086  5900 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:12.754354  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:12.755273  5072 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:12.747442  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:12.731695  9879 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:12.702579  9419 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:12.766330  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.770795  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:12.772318  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.772511  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.760792  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:12.774847  5144 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:12.763458  5146 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:12.765014  5940 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:12.765165  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:12.773164  5076 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 9.
I20260709 12:25:12.759831  5143 raft_consensus.cc:2376] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15. Re-sending same reply.
W20260709 12:25:12.761751  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 56287us
I20260709 12:25:12.777715  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.744206 10014 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:12.770249  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.780444  5149 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:25:12.743876 10023 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:12.749394  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:12.779932  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.781005  5149 pending_rounds.cc:85] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 8
I20260709 12:25:12.792487  5149 pending_rounds.cc:107] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:25:12.792878  5149 raft_consensus.cc:2889] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:12.680120  9864 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:12.752113  9927 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.007s	sys 0.029s
I20260709 12:25:12.775665  5941 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:12.754076  9867 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:12.803022  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 53753.5us
I20260709 12:25:12.798044  5945 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:12.778538  5076 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:12.805692  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:12.784451  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.748230  9987 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:12.737746 10034 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 12:25:12.789013  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.749341  9947 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:12.793598  5068 raft_consensus.cc:2393] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 17: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:12.794379  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:12.754030  9284 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:12.807622  5941 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:12.766641  9892 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:12.808674  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.787514  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:12.780064 10014 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:12.829559  5114 leader_election.cc:336] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 2.140s (SENT)
I20260709 12:25:12.873181  5114 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:12.874267  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 44814.2us
I20260709 12:25:12.842180  5143 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:12.811105  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:12.758154 10002 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:12.845402  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.748347  9988 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:25:12.719643  9859 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:12.769626 10086 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:12.847007  5067 raft_consensus.cc:2410] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 12:25:12.851717  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 203447us
W20260709 12:25:12.898285  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.852180  5147 raft_consensus.cc:2410] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:25:12.852671  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.850608  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:12.901731  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.841414  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.905998  5076 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:12.857076  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:12.908303  5073 raft_consensus.cc:2368] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14.
I20260709 12:25:12.871091  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.810994  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.880756  5945 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:12.881968  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.882059  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.911248  5069 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:25:12.890117  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.791690 10023 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:12.892318  5114 leader_election.cc:336] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.930s (SENT)
I20260709 12:25:12.791018 10042 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 12:25:12.884541 10014 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:12.929443  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:12.893760  9864 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:12.896510  5146 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:12.850448 10104 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:12.918848  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:12.911463  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:12.955797  5076 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:12.955619  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.958138  5072 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:12.784086  9843 consensus_queue.cc:237] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:12.869469  9892 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:12.926014  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.915446  5075 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:12.795601  9860 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 12:25:12.808956 10049 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 12:25:12.882680  9898 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:12.930429  5143 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:12.916062  5042 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.922s (SENT)
W20260709 12:25:12.855576  9745 consensus_queue.cc:1175] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" responder_term: 15 status { last_received { term: 13 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: [14.9-14.9]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:12.917987  5114 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:12.944898  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.987104  5071 raft_consensus.cc:2410] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:25:12.949442  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.904918  9996 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.953398  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:12.954813  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:12.958333  5147 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:12.989303  5146 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:12.921413  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:12.903604  9850 consensus_queue.cc:1059] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
W20260709 12:25:12.974697  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 185036us
I20260709 12:25:12.993877  5073 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:12.999033  5076 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:12.982422  5072 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:25:12.955399 10104 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.924280 10023 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.000214  5072 pending_rounds.cc:85] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 5
I20260709 12:25:13.012212  5072 pending_rounds.cc:107] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:25:13.012549  5072 raft_consensus.cc:2889] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:13.002405  5076 raft_consensus.cc:3055] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:13.013063  5076 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:13.015571  5069 raft_consensus.cc:1240] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:13.013929  5076 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:13.023469  5076 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:13.026043 10113 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:25:13.028476  9419 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:13.013105 10104 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:13.016801 10039 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:25:12.959714 10002 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:13.037088 10002 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:13.037917  9894 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:13.007400  9996 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.039490  9419 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.040338  9419 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:13.042816  5071 raft_consensus.cc:1240] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:13.004770  5146 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
W20260709 12:25:13.006734  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 114447us
I20260709 12:25:13.053776  5073 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:13.009867  5898 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:13.054852  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 155023us
W20260709 12:25:13.055545  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45770.6us
I20260709 12:25:12.970896  9892 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:13.058224  5898 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:13.070999  5042 leader_election.cc:336] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.941s (SENT)
I20260709 12:25:13.011086  9988 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:13.061467  5143 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:13.082146  5147 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:13.078797  5898 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:13.083413  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:13.090209  5076 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:25:13.091815  5898 leader_election.cc:400] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11.
W20260709 12:25:13.089663  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.092149  5076 pending_rounds.cc:85] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 8
I20260709 12:25:13.093066  5898 leader_election.cc:403] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:13.093331  5076 pending_rounds.cc:107] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:25:13.092365  5143 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:13.093982  5076 raft_consensus.cc:2889] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:13.089332  5070 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
W20260709 12:25:13.138499  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 46899.7us
I20260709 12:25:13.138969  5147 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:13.139488  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:13.140818  5147 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:13.073586  9860 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:13.149225  5898 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:13.138314  9980 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.178601  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.074923 10070 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:13.204336  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 55239.4us
I20260709 12:25:13.206138  5898 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:13.213891  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 158321us
I20260709 12:25:13.147327 10125 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:13.228014  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.229224  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.230340  5042 connection.cc:256] RPC call timeout handler was delayed by -1.2954155589995935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32610000000000006s for a response
I20260709 12:25:13.154897 10039 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 10. Current term is 11.
W20260709 12:25:13.216248 10115 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:25:13.237907  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:13.236150  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.203925 10042 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:13.238430  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:13.239766 10117 log.cc:927] Time spent T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
W20260709 12:25:13.245123 10127 log.cc:927] Time spent T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:25:13.246441 10118 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
W20260709 12:25:13.248229 10114 log.cc:927] Time spent T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
W20260709 12:25:13.249109 10119 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 12:25:13.219204  9996 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.218173  9860 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:13.218091 10086 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:13.263310  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:13.235052  9978 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.266149  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:13.202081  9980 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:13.265090  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:13.235761  9483 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.301412  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.241080  9991 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:13.213932  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:25:13.309243  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 95416us
I20260709 12:25:13.305045  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:13.306491  5069 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:25:13.280076  5939 consensus_meta.cc:287] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 12:25:13.310070  5073 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:13.310323  5939 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:13.312281  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:13.261826  9898 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.316717  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.281034  9988 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.296407 10049 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:13.302904 10041 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:13.318078  9860 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:13.322918  9996 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:13.339051  9987 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.366466  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 312144us
I20260709 12:25:13.312187  9932 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:25:13.308687  9978 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:13.319042  9483 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:13.363786  9679 consensus_meta.cc:287] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.786s	user 0.014s	sys 0.006s
I20260709 12:25:13.377453  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1501) took 1141 ms. Trace:
I20260709 12:25:13.378365  5150 rpcz_store.cc:276] 0709 12:25:12.236097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.300397 (+ 64300us) service_pool.cc:224] Handling call
0709 12:25:13.375733 (+1075336us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54003421d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:13.377383 (+  1650us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437724928}
I20260709 12:25:13.323314  9898 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.339979  9988 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:13.382798  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.386315  5141 raft_consensus.cc:1275] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:13.367053  9860 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:13.385706  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:13.397114  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 80887us
I20260709 12:25:13.397624  5071 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:25:13.398207  5071 pending_rounds.cc:85] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 7
W20260709 12:25:13.398168  5898 leader_election.cc:336] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 3.048s (SENT)
I20260709 12:25:13.398875  5071 pending_rounds.cc:107] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:25:13.402024  5071 raft_consensus.cc:2889] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:13.384558 10124 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:25:13.403364  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:13.380766  9419 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.387981  9679 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.374285  9987 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.393814  9975 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:25:13.413341  9896 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:13.415339  9991 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:13.423580  9679 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.465459  9898 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.466599  9898 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.467375  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.467258  9898 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:13.468537  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.469937 10128 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:25:13.472426  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1237) took 2328 ms. Trace:
W20260709 12:25:13.454591 10131 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
I20260709 12:25:13.472923  5946 rpcz_store.cc:276] 0709 12:25:11.143930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.144193 (+   263us) service_pool.cc:224] Handling call
0709 12:25:11.144507 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.156067 (+ 11560us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:12.504354 (+1348287us) spinlock_profiling.cc:243] Waited 8.68 ms on lock 0x7b180047d520. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8abf333c7 00007fd8a85459a7 00007fd8a85550bf 00007fd8a8555038
0709 12:25:12.844627 (+340273us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.299129 (+454502us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:13.301201 (+  2072us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.301365 (+   164us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:13.301404 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.301598 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.472032 (+170434us) raft_consensus.cc:1685] finished
0709 12:25:13.472146 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.472346 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11363,"prepare.queue_time_us":330288,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":17844,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":17346,"spinlock_wait_cycles":19967744,"thread_start_us":737834,"threads_started":2,"wal-append.queue_time_us":386220}
I20260709 12:25:13.497126  9892 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:13.499526  9987 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.504993  9892 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:13.499521 10014 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.004s
I20260709 12:25:13.510648  9988 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.518091 10014 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:13.521797 10014 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:13.524516  9483 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.457131  9860 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:13.534320  9978 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:13.539711 10015 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:13.546231  9679 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:13.547995  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.552114  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 166537us
I20260709 12:25:13.552171  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1507) took 1296 ms. Trace:
I20260709 12:25:13.552484  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:13.554072  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:13.553023  5143 rpcz_store.cc:276] 0709 12:25:12.255800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.094359 (+838559us) service_pool.cc:224] Handling call
0709 12:25:13.157696 (+ 63337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:13.202446 (+ 44750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.202488 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:13.218755 (+ 16267us) log.cc:844] Serialized 52 byte log entry
0709 12:25:13.387417 (+168662us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:13.417027 (+ 29610us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.431297 (+ 14270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.431407 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.551818 (+120411us) raft_consensus.cc:1685] finished
0709 12:25:13.551851 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.552081 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20441,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":124544,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":12130,"thread_start_us":141590,"threads_started":2,"wal-append.queue_time_us":219270}
I20260709 12:25:13.552175  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:13.556912  5944 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:13.559231  5042 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:13.559659  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1513) took 1163 ms. Trace:
I20260709 12:25:13.555685  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:13.560441  5143 rpcz_store.cc:276] 0709 12:25:12.396164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.558227 (+1162063us) service_pool.cc:224] Handling call
0709 12:25:13.558701 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.558961 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.559008 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.559064 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:13.559232 (+   168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.559315 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.559474 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.562733  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:13.563616  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1514) took 1109 ms. Trace:
I20260709 12:25:13.564268  5143 rpcz_store.cc:276] 0709 12:25:12.454751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.562354 (+1107603us) service_pool.cc:224] Handling call
0709 12:25:13.562798 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.563064 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.563103 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.563167 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:13.563226 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.563387 (+   161us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.563542 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.566370  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:13.570489  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:13.573138  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:13.567058  5898 leader_election.cc:336] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.506s (SENT)
I20260709 12:25:13.572700  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:13.573874  5946 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:13.567147  5143 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:13.578356  5143 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:13.461987  9996 raft_consensus.cc:697] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:13.566237  9483 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:13.585430  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:13.566403  9978 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:13.579545  9975 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:25:13.578157  9991 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:13.592695  9980 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:13.569823 10002 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:13.594408  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 978) took 1201 ms. Trace:
W20260709 12:25:13.578632  9867 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.721s	user 0.016s	sys 0.007s
I20260709 12:25:13.599335  9896 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:13.600376  5900 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:13.594641  9932 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:13.617256  9896 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:13.620124 10041 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:13.608451 10144 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:25:13.620847  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:13.621917 10134 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:25:13.621790  9867 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:13.622511 10144 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:13.623260  9875 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:13.623970  5947 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:13.623901 10086 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:13.624418  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:13.624562 10086 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.625133 10086 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:13.620849 10041 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:13.625820  9867 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:13.626044  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 1126) took 1041 ms. Trace:
I20260709 12:25:13.626531  9867 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:13.626462  5947 rpcz_store.cc:276] 0709 12:25:12.584834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.733345 (+148511us) service_pool.cc:224] Handling call
0709 12:25:12.733771 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.624523 (+890752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.625785 (+  1262us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b540000a354. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:13.625977 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2046729600}
I20260709 12:25:13.627144  9867 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:13.628852  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:13.614684 10080 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.633879  9875 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.634490  9875 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:13.636633  9864 consensus_meta.cc:287] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.682s	user 0.012s	sys 0.000s
I20260709 12:25:13.636731  9875 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:13.613528  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1498) took 2044 ms. Trace:
I20260709 12:25:13.634248 10080 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:13.586651  5143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4935fc8f35664324a253df9804c8074d"
dest_uuid: "ceec344dd00f40c18ef58414ce147016"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:49182
I20260709 12:25:13.619999  9896 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:13.638346  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:13.639568  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:13.640430  9938 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:13.640645  9850 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:13.605764  5150 rpcz_store.cc:276] 0709 12:25:12.392610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.379317 (+986707us) service_pool.cc:224] Handling call
0709 12:25:13.388031 (+  8714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:13.388337 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.388376 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:13.397613 (+  9237us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.571662 (+174049us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:13.571716 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.571814 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.594146 (+ 22332us) raft_consensus.cc:1685] finished
0709 12:25:13.594171 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.594338 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":140426,"threads_started":1,"wal-append.queue_time_us":174678}
I20260709 12:25:13.642409  9865 raft_consensus.cc:3055] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:13.642683  5150 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:13.642855  9865 raft_consensus.cc:740] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:13.643116  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:13.643796  9865 consensus_queue.cc:260] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.650301  9865 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:13.650787 10092 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:13.639624 10086 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.652740  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1509) took 1289 ms. Trace:
I20260709 12:25:13.647826  5067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58036 (request call id 1240) took 1042 ms. Trace:
W20260709 12:25:13.659066 10023 consensus_meta.cc:287] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.616s	user 0.013s	sys 0.000s
I20260709 12:25:13.660454 10023 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:13.661768  5144 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 10: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:13.662669  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:13.663615  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 1632) took 1089 ms. Trace:
I20260709 12:25:13.655035  5067 rpcz_store.cc:276] 0709 12:25:12.605280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.610051 (+  4771us) service_pool.cc:224] Handling call
0709 12:25:13.647572 (+1037521us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540047d454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:13.647743 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":540417920}
I20260709 12:25:13.664109  5074 rpcz_store.cc:276] 0709 12:25:12.574560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.594851 (+ 20291us) service_pool.cc:224] Handling call
0709 12:25:13.662486 (+1067635us) spinlock_profiling.cc:243] Waited 5.94 ms on lock 0x7b5800560198. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:25:13.663545 (+  1059us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13662592}
I20260709 12:25:13.655035  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:13.666337  5070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 1019) took 2892 ms (client timeout 2159 ms). Trace:
I20260709 12:25:13.666847  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:13.666815  5070 rpcz_store.cc:269] 0709 12:25:10.773796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.877221 (+103425us) service_pool.cc:224] Handling call
0709 12:25:13.666164 (+2788943us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b540047ca54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:13.666266 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4883982080}
I20260709 12:25:13.655368  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:13.611337 10116 log.cc:927] Time spent T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.670s	user 0.006s	sys 0.003s
I20260709 12:25:13.655681  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:13.670238  5941 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:13.671617  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:13.672266  5947 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:13.673844  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 962) took 2896 ms. Trace:
I20260709 12:25:13.674345  5145 rpcz_store.cc:276] 0709 12:25:10.777001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.116129 (+1339128us) service_pool.cc:224] Handling call
0709 12:25:12.116462 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.608528 (+492066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.673522 (+1064994us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b5400837554. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:13.673774 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":845121280}
I20260709 12:25:13.655997  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:13.676985  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:13.669250  5939 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:13.677812  5217 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:13.678454  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:13.678654  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:13.679181  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:13.679981  5223 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:13.680998  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:13.682154  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 871092us
W20260709 12:25:13.657348 10104 consensus_meta.cc:287] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.624s	user 0.011s	sys 0.004s
I20260709 12:25:13.657686  9948 raft_consensus.cc:2804] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:13.683578  9948 raft_consensus.cc:697] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:13.684249  5190 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:13.684623  9948 consensus_queue.cc:237] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:13.685495  9894 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.648861  9938 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:13.686364  9938 consensus_queue.cc:1243] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer ceec344dd00f40c18ef58414ce147016 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:13.686008  9894 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:13.642901  9895 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.960s	user 0.014s	sys 0.005s
I20260709 12:25:13.688279  9894 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:13.688150  9895 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.688819 10053 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:13.689185  9894 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:13.648372  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.639935  9843 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:13.685791  9872 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:13.679152  5215 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:13.678752  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:13.706691  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.691012  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 979) took 1260 ms. Trace:
W20260709 12:25:13.657464  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 907563us
I20260709 12:25:13.658452  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 958) took 1225 ms. Trace:
I20260709 12:25:13.720463  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:13.720832  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:13.654053  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:13.683367 10104 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:13.726577  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 83551us
W20260709 12:25:13.727221  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 73242.7us
I20260709 12:25:13.722115  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:13.728039  5042 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:13.728122  5219 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:13.718657  5144 rpcz_store.cc:276] 0709 12:25:12.430828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.787178 (+356350us) service_pool.cc:224] Handling call
0709 12:25:13.690788 (+903610us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b540014e6d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:13.690943 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1853832064}
I20260709 12:25:13.730815  5144 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:13.732275  5144 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:13.722950  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:13.708510  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:13.736996  5041 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:13.738557  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:13.723083  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:13.719239  5143 raft_consensus.cc:606] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:13.723806  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.638309  5149 rpcz_store.cc:276] 0709 12:25:11.569108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.343303 (+774195us) service_pool.cc:224] Handling call
0709 12:25:12.666612 (+323309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.793563 (+126951us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.447421 (+653858us) spinlock_profiling.cc:243] Waited 99.4 ms on lock 0x7b54004fae54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:13.613192 (+165771us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54004fae50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:13.613439 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":971278848}
I20260709 12:25:13.690303  5190 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:13.741452  5143 raft_consensus.cc:614] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:13.742213  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52002.9us
I20260709 12:25:13.742956  5190 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:13.743772  5190 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:13.745086  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:13.745565  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:13.653177  5147 rpcz_store.cc:276] 0709 12:25:12.363543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.141918 (+778375us) service_pool.cc:224] Handling call
0709 12:25:13.143567 (+  1649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:13.188591 (+ 45024us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:13.188635 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:13.213826 (+ 25191us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.401443 (+187617us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:13.401590 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.652307 (+250717us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b54006e2654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:13.652355 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.652406 (+    51us) raft_consensus.cc:1685] finished
0709 12:25:13.652419 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.652671 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6250,"raft.queue_time_us":6376,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":44383,"spinlock_wait_cycles":10257152,"thread_start_us":153859,"threads_started":1,"wal-append.queue_time_us":221841}
W20260709 12:25:13.723379  5114 leader_election.cc:336] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.661s (SENT)
I20260709 12:25:13.747047  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:13.747273  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:13.747684  5147 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:25:13.748093 10021 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:13.748478 10021 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.748812 10021 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:13.719774  5216 rpcz_store.cc:276] 0709 12:25:12.432534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.545331 (+112797us) service_pool.cc:224] Handling call
0709 12:25:12.574583 (+ 29252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.574833 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:12.605114 (+ 30281us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.658060 (+1052946us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.658126 (+    66us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:13.658174 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.658252 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.658378 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":849232,"prepare.queue_time_us":39861,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":849894,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":203210,"threads_started":1,"wal-append.queue_time_us":387021}
I20260709 12:25:13.746685  5114 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:13.751072 10145 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:13.751765  9932 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:13.752410  9932 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:13.752688  5114 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.708s (SENT)
I20260709 12:25:13.752852  9932 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:13.753229  5114 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:13.754120  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:13.740844  5947 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.745855  5146 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: [12.6-13.7]
I20260709 12:25:13.739804  9938 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:13.699718  9859 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:13.653728  9483 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 12:25:13.758992  9888 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:13.689414 10053 raft_consensus.cc:697] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:13.734124 10144 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:13.724591  5941 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:13.768805  9284 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:13.769356 10053 consensus_queue.cc:237] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:13.653505 10086 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:13.728935  9879 consensus_queue.cc:1059] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:25:13.740127  9894 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:13.784251 10135 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.334s	user 0.003s	sys 0.000s
I20260709 12:25:13.715663  9872 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:13.732493  9941 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:25:13.761651  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:13.817191  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 55665.6us
I20260709 12:25:13.818830  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:13.690457  9895 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.776047  5147 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:13.764595  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:13.776594  5219 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:13.739609  9991 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:25:13.729478  9980 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:25:13.744787  9975 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:13.739832 10002 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:25:13.760777  9745 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:25:13.771157  9947 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:25:13.784369  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.746024  9898 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.785346  9843 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:13.785884  9938 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:13.838713 10104 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:13.778040 10088 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:13.740980  9896 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:13.689738  9892 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:25:13.790831  5898 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:13.825296  9934 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:13.794401  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:13.797089  5947 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:25:13.746671 10023 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:13.852001  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:13.764878  5942 raft_consensus.cc:1240] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:13.799916  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 54903.6us
I20260709 12:25:13.852977  5941 raft_consensus.cc:2376] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13. Re-sending same reply.
I20260709 12:25:13.825311 10144 consensus_queue.cc:1059] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:25:13.853704  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:13.804898  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:13.854734  5215 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:13.809139  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:13.854967  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:13.809648  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:13.740767 10041 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.856623  5223 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:13.851953  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:13.858613 10041 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:13.823724  5114 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:13.825984  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 61443.8us
I20260709 12:25:13.866989  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:13.830099  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 102564us
I20260709 12:25:13.831959  5216 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:13.832314 10092 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:13.869175 10092 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.811236  9850 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:25:13.869607 10092 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:13.841745  9419 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:25:13.838891  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1506) took 1601 ms. Trace:
I20260709 12:25:13.870692  5141 rpcz_store.cc:276] 0709 12:25:12.237611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.142784 (+905173us) service_pool.cc:224] Handling call
0709 12:25:13.202886 (+ 60102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.386903 (+184017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.838593 (+451690us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540045a954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:13.838812 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416456576}
I20260709 12:25:13.772230 10145 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:13.799163 10049 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:25:13.780746 10021 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.874423 10021 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.874514  9991 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.875109  9991 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:13.877142  9991 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.764634  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1248) took 1342 ms. Trace:
I20260709 12:25:13.877854  5943 rpcz_store.cc:276] 0709 12:25:12.422032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.422163 (+   131us) service_pool.cc:224] Handling call
0709 12:25:12.433275 (+ 11112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.751310 (+318035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.764387 (+1013077us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b5400009454. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:13.764579 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729270656}
I20260709 12:25:13.822793 10095 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:13.881413  9947 raft_consensus.cc:697] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:13.846319  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 55625us
I20260709 12:25:13.827842  9867 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:13.847338  9894 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:13.831697  9975 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.897207  9975 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:13.896786 10092 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.899377 10092 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.901893  9865 consensus_queue.cc:1059] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:13.811990  9996 consensus_queue.cc:237] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:13.904237  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 985) took 1379 ms. Trace:
W20260709 12:25:13.906215  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 82558.3us
I20260709 12:25:13.907913  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:13.850523  5142 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:13.858994  5947 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:25:13.840799 10094 raft_consensus.cc:3055] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:13.912201 10145 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:13.912361  9898 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.913218 10023 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:13.913607  9898 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.911098 10097 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.916217 10097 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:13.916652  9898 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:13.917595 10094 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:13.918926 10094 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:13.920961  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 54050us
I20260709 12:25:13.922011  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:13.904860  5942 rpcz_store.cc:276] 0709 12:25:12.524455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.525635 (+  1180us) service_pool.cc:224] Handling call
0709 12:25:13.764608 (+1238973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.852944 (+ 88336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.903909 (+ 50965us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400009450. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:13.904143 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2848485120}
I20260709 12:25:13.923712  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:13.928913  5114 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:13.828016 10032 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:13.896044 10046 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:13.843235 10088 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.912317  9877 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:13.944432  5148 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:13.942524  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:13.931727  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:13.889446  9896 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:13.937376  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:13.937935  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:13.938412 10091 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:13.883314  9978 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 12:25:13.887810  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:13.936699 10015 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:13.941764  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:13.948801  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:13.968017  5216 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:13.968163  5147 raft_consensus.cc:2368] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13.
I20260709 12:25:13.949591  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:13.937968  9898 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:13.968927  5947 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:13.952003  5223 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:13.952023  5143 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:13.970867  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:13.952768  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:13.972224  5942 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:13.953832  5142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 968) took 2794 ms (client timeout 1718 ms). Trace:
W20260709 12:25:13.973040  5142 rpcz_store.cc:269] 0709 12:25:11.149900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.214035 (+ 64135us) service_pool.cc:224] Handling call
0709 12:25:13.943631 (+2729596us) spinlock_profiling.cc:243] Waited 2.63 s on lock 0x7b540036ea54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:13.943803 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6057312000}
I20260709 12:25:13.982743  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:13.983705  5142 raft_consensus.cc:2368] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12.
I20260709 12:25:13.933089  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:13.930881 10021 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:13.938699  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:13.936908 10023 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:13.889818 10086 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:13.929194 10049 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:25:13.960549  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:13.999794  5141 raft_consensus.cc:2368] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12.
I20260709 12:25:13.964469  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:14.001147  5144 raft_consensus.cc:2368] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10.
I20260709 12:25:14.002225  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:14.003281  5144 raft_consensus.cc:2410] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:25:13.963092  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:14.004498  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:13.882565  9947 consensus_queue.cc:237] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:13.969432  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 81746.3us
I20260709 12:25:14.009961  5898 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:13.942205 10088 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:13.992147  5224 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:14.023386  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:14.024339  5147 raft_consensus.cc:2410] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:25:14.024909  5149 raft_consensus.cc:2410] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:25:13.971531  5143 raft_consensus.cc:2376] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12. Re-sending same reply.
I20260709 12:25:14.025805  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:14.026551  5150 raft_consensus.cc:2368] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17.
I20260709 12:25:14.025338  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:14.028467  5146 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 15: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:25:13.993664  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 60797.6us
I20260709 12:25:13.923285  9941 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:25:13.942685  9877 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.011655  5942 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:14.029006  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:14.048373  5142 raft_consensus.cc:2368] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15.
I20260709 12:25:14.029407  5114 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:14.031039  5223 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:14.036105  5148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1500) took 1800 ms. Trace:
I20260709 12:25:14.050449  5148 rpcz_store.cc:276] 0709 12:25:12.235733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.375676 (+139943us) service_pool.cc:224] Handling call
0709 12:25:13.944003 (+1568327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.945026 (+  1023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.035879 (+ 90853us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b540036ea50. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:14.036039 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3583840512}
I20260709 12:25:14.046489  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:14.073158  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:14.074436  5141 raft_consensus.cc:2410] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:25:13.996769  9888 raft_consensus.cc:697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:14.075778  5143 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 12:25:14.077104  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:14.085486  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:14.086509  5940 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:14.086203  9888 consensus_queue.cc:237] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:13.997305 10023 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.994590  9745 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:14.175253  5146 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:25:14.024912  9898 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 17 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:14.188421 10015 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:13.941004  9872 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:14.072149  9877 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:14.101238 10097 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:14.209126 10143 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.209299  9872 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:14.176218  9927 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:14.186967 10023 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:14.186475 10041 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.049821  9892 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.159s
I20260709 12:25:14.049243  9864 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.159s
W20260709 12:25:14.206825 10131 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 12:25:14.230139  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.176390 10032 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
W20260709 12:25:14.254251  5945 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.013s	sys 0.030s
W20260709 12:25:14.254437  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:14.254832  5945 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.014s	sys 0.031s
I20260709 12:25:14.257324  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:14.258149  5070 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:14.261044  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:14.261212  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:14.261487  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:14.262148  5067 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:14.263162  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:14.263161  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:14.264837  5074 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:14.265730  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:14.266502  5072 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:14.254531  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 225201us
I20260709 12:25:14.282467  5143 pending_rounds.cc:85] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 8
I20260709 12:25:14.282969  5143 pending_rounds.cc:107] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:25:14.283281  5143 raft_consensus.cc:2889] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:14.287395  5144 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:14.279245  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:14.294307  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:14.294499  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:14.295439  5070 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
W20260709 12:25:14.297060  5041 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.007s	sys 0.035s
W20260709 12:25:14.297585  5041 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.008s	sys 0.035s
W20260709 12:25:14.297839  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 508145us
W20260709 12:25:14.327679  5948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.004s	sys 0.038s
I20260709 12:25:14.328715  5216 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:14.338114  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:25:14.311712  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 442917us
I20260709 12:25:14.338749  5141 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:14.336930  5948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.005s	sys 0.039s
W20260709 12:25:14.311462  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 428785us
I20260709 12:25:14.331202  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:14.257905 10080 consensus_meta.cc:287] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.619s	user 0.009s	sys 0.011s
I20260709 12:25:14.346799  5071 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:14.353112  5947 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:14.352716  5115 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:14.230388  9948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.032s	sys 0.048s
I20260709 12:25:14.203953  9896 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:14.305960  9938 raft_consensus.cc:3055] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:14.223798 10143 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:14.309036  9284 raft_consensus.cc:697] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:14.391055 10143 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:14.360934  9948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.033s	sys 0.057s
I20260709 12:25:14.393220 10094 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:14.248806 10041 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:14.330281 10015 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:14.372220  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:14.376504  5072 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 10.
I20260709 12:25:14.387686  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:14.337867 10097 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:25:14.370721 10080 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:14.378479  9938 raft_consensus.cc:740] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:14.420573  5071 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 10: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:14.268604  9927 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:14.364493  9932 consensus_meta.cc:287] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.611s	user 0.017s	sys 0.017s
I20260709 12:25:14.302469  9483 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:25:14.362557 10145 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:14.363118  9988 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.437350  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:14.395226 10163 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.349851 10034 consensus_queue.cc:1059] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 12:25:14.406065  5141 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:14.455638  5071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "552d065b39ba44699c339e80bcc02744"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:14.483181  5071 raft_consensus.cc:606] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:14.483664  5071 raft_consensus.cc:614] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:14.471323  5146 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
W20260709 12:25:14.418892  5116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.413084 10021 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:14.477732 10097 consensus_queue.cc:1243] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:14.480883  5939 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 17: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:14.355963 10148 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 12:25:14.415000  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:14.453595 10147 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 12:25:14.531584  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 191307us
W20260709 12:25:14.540316  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 162476us
I20260709 12:25:14.542047  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:14.532152  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:14.391234  9284 consensus_queue.cc:237] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:14.372483  9898 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:14.548507  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 203643us
I20260709 12:25:14.566069  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:14.599114  5074 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:14.535585  9975 consensus_meta.cc:287] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.000s
I20260709 12:25:14.566502  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:14.605454  5075 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:14.476331  9932 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:14.480963 10158 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:25:14.488515 10163 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:14.486961  9988 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:14.590374  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 237760us
I20260709 12:25:14.566888  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:14.606184  5072 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:14.532740 10021 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:14.586778 10166 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:14.624886  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:14.574427  5076 raft_consensus.cc:1275] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:14.633340  5069 raft_consensus.cc:2410] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:25:14.644383  5067 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:14.633392  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:14.652818  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:14.654412  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.598291 10159 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:14.626375  9860 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:14.659711  5945 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:14.625303 10162 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:14.630430 10166 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:14.646330  9996 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:14.674569  9996 raft_consensus.cc:697] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:14.675665  9996 consensus_queue.cc:237] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:14.555801 10165 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.117s
I20260709 12:25:14.681680  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:14.611303  9975 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:14.685962  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:14.655690 10159 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:14.660871 10148 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:14.699540  5067 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:14.651490 10157 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:14.707633  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:14.695410  9859 consensus_meta.cc:287] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.928s	user 0.006s	sys 0.007s
I20260709 12:25:14.690307 10097 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:25:14.711005 10167 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:25:14.684443  9934 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:14.722148  5944 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:14.713711  5067 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:25:14.723306  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:14.723861  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 119370us
W20260709 12:25:14.726817  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 178939us
I20260709 12:25:14.749145  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:14.762943  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:14.763936  5067 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:25:14.738528  5190 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:14.736954  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:14.861047  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 122595us
W20260709 12:25:14.862851  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 122587us
I20260709 12:25:14.876572  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:14.692726  9894 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:25:14.874786  5900 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:14.891124  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:14.914726  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:14.902318  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 1004) took 1178 ms. Trace:
I20260709 12:25:14.915861  5939 rpcz_store.cc:276] 0709 12:25:13.723914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.745278 (+ 21364us) service_pool.cc:224] Handling call
0709 12:25:14.902007 (+1156729us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b540000ad54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:14.902225 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651959168}
I20260709 12:25:14.919572  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:14.891569  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:14.725137 10015 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:14.712302 10181 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:25:14.933835  5898 leader_election.cc:336] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.622s (SENT)
I20260709 12:25:14.908566  5041 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:14.861312  9745 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:14.906427 10164 log.cc:927] Time spent T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.374s	user 0.001s	sys 0.001s
I20260709 12:25:14.873857 10097 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:25:14.995237  5143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.005s	sys 0.022s
W20260709 12:25:14.996341  5143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.005s	sys 0.022s
I20260709 12:25:14.910982 10095 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:14.891191 10002 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:25:15.032207  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 123742us
I20260709 12:25:14.889389  9859 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 12:25:14.874079  9934 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:14.946787  9987 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:25:15.047472  5040 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.025s	sys 0.058s
W20260709 12:25:15.083282  5040 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.025s	sys 0.059s
I20260709 12:25:14.909767 10092 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:14.720096 10021 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:15.067947  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:14.945583 10181 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.999789 10043 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:15.055035  9879 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:14.924413 10157 consensus_queue.cc:1059] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.147s
I20260709 12:25:15.034111 10002 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:15.058320  9843 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:15.033948  9877 consensus_meta.cc:287] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.813s	user 0.008s	sys 0.012s
I20260709 12:25:15.057492 10095 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:15.071136 10165 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:14.997399  9745 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:14.880745 10166 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:15.052994 10032 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:25:15.118945  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:15.130295  5075 consensus_meta.cc:287] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.521s	user 0.012s	sys 0.001s
I20260709 12:25:15.146842  5075 raft_consensus.cc:2468] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
W20260709 12:25:15.119828  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 245129us
I20260709 12:25:15.090298  9932 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:15.094342 10148 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:15.147702  5075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:15.186990  5075 raft_consensus.cc:606] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:15.187327  5075 raft_consensus.cc:614] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:15.148602  5072 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 12:25:15.176055  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 108235us
I20260709 12:25:15.188302  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:15.115350 10169 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.457s	user 0.000s	sys 0.001s
I20260709 12:25:15.108112  9843 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:15.099471 10181 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:15.111244 10043 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:15.189767  9987 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:15.181257  5074 consensus_meta.cc:287] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.573s	user 0.014s	sys 0.000s
I20260709 12:25:15.208751  5074 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:15.126560 10195 raft_consensus.cc:3055] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:15.099215 10092 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:15.123031 10163 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:15.231405  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:15.175230  9988 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:15.111105  9877 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:15.195997  9843 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:15.112586  9879 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:15.200825 10189 raft_consensus.cc:3055] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:25:15.229054 10152 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.538s	user 0.001s	sys 0.000s
I20260709 12:25:15.246444  9978 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.014s
W20260709 12:25:15.253783  5898 leader_election.cc:336] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.564s (SENT)
I20260709 12:25:15.212247 10188 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:15.220872 10195 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:15.125957 10097 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:15.033026 10178 consensus_queue.cc:1048] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.252s
I20260709 12:25:15.228826  9987 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:15.229646 10002 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:15.282457  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 51182.9us
W20260709 12:25:15.249092  5114 leader_election.cc:336] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.691s (SENT)
I20260709 12:25:15.323906  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1261) took 1439 ms. Trace:
W20260709 12:25:15.532637  5947 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.004s	sys 0.005s
I20260709 12:25:15.776659  9980 raft_consensus.cc:3055] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:15.757097 10094 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.363s	user 0.017s	sys 0.005s
I20260709 12:25:15.481468 10200 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:15.326530  9865 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:15.436530  5897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.008s	sys 0.004s
I20260709 12:25:15.953413  5146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1522) took 1039 ms. Trace:
W20260709 12:25:15.668365 10023 consensus_meta.cc:287] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.406s	user 0.014s	sys 0.000s
I20260709 12:25:15.972890 10023 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:15.975438 10159 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:15.957234  5943 rpcz_store.cc:276] 0709 12:25:13.884403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.884556 (+   153us) service_pool.cc:224] Handling call
0709 12:25:13.895214 (+ 10658us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:13.929349 (+ 34135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.929380 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:13.942087 (+ 12707us) log.cc:844] Serialized 52 byte log entry
0709 12:25:14.343910 (+401823us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:14.388741 (+ 44831us) log.cc:844] Serialized 14 byte log entry
0709 12:25:14.396101 (+  7360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.396221 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.323643 (+927422us) raft_consensus.cc:1685] finished
0709 12:25:15.323666 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.323838 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54969,"prepare.queue_time_us":75020,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":47266,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":8672,"thread_start_us":385489,"threads_started":1,"wal-append.queue_time_us":644997}
W20260709 12:25:15.957966  5947 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.004s	sys 0.010s
I20260709 12:25:15.327292 10092 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:15.979249  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 1012) took 1227 ms. Trace:
I20260709 12:25:15.959399  9980 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:15.334609 10095 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:15.983371  9980 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:15.968511  5897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.064s	user 0.008s	sys 0.008s
I20260709 12:25:15.976018  5150 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:15.952514  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 996) took 1600 ms. Trace:
I20260709 12:25:15.971335  5146 rpcz_store.cc:276] 0709 12:25:14.916110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.922072 (+  5962us) service_pool.cc:224] Handling call
0709 12:25:14.939735 (+ 17663us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.940307 (+   572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.940349 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.984311 (+ 43962us) log.cc:844] Serialized 26 byte log entry
0709 12:25:15.334255 (+349944us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:15.334322 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.334431 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.952773 (+618342us) raft_consensus.cc:1685] finished
0709 12:25:15.952812 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.953157 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55620,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":13386,"thread_start_us":240797,"threads_started":1,"wal-append.queue_time_us":640608}
W20260709 12:25:15.478181 10185 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.274s	user 0.000s	sys 0.000s
I20260709 12:25:15.677114  9895 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:16.032037  9895 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:16.032397  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:16.034735  9898 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:15.479379  9745 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:15.325351 10086 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:16.036936 10086 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:16.034108  9895 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:15.967072  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 1045) took 1334 ms. Trace:
I20260709 12:25:16.045441 10086 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:15.570951  9872 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 1.348s	user 0.015s	sys 0.002s
I20260709 12:25:16.033967  9996 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:15.461619 10207 log.cc:927] Time spent T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 12:25:15.332681 10174 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
W20260709 12:25:15.718567  9860 consensus_meta.cc:287] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 1.052s	user 0.012s	sys 0.006s
I20260709 12:25:15.962284 10002 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:15.967672  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:15.243439 10161 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:15.979550  9879 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:15.432112 10175 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.474s	user 0.000s	sys 0.001s
I20260709 12:25:15.979550  9877 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:15.283131 10165 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:15.282378 10163 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:15.953325  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:15.809489 10080 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:15.511439 10046 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.569s	user 0.018s	sys 0.007s
I20260709 12:25:15.967667  9865 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:15.979733  5947 rpcz_store.cc:276] 0709 12:25:14.751905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.752094 (+   189us) service_pool.cc:224] Handling call
0709 12:25:14.905698 (+153604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:15.978523 (+1072825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:15.978583 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:15.978662 (+    79us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:15.978703 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.978848 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.979071 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42387,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":45118,"thread_start_us":1072011,"threads_started":1}
I20260709 12:25:16.047647  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1525) took 1129 ms. Trace:
W20260709 12:25:15.321974 10182 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
W20260709 12:25:15.484555 10088 consensus_meta.cc:287] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.414s	user 0.010s	sys 0.005s
I20260709 12:25:15.475602  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 1001) took 1755 ms. Trace:
W20260709 12:25:16.063329  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:15.322028 10199 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:25:16.098320  5944 rpcz_store.cc:276] 0709 12:25:13.720008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.720149 (+   141us) service_pool.cc:224] Handling call
0709 12:25:15.475372 (+1755223us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54002d7dd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:15.475529 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2301844736}
I20260709 12:25:16.100859  5941 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:15.671564  9419 raft_consensus.cc:3055] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:16.101253  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1637) took 1539 ms. Trace:
I20260709 12:25:16.101935  9419 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:16.103082 10209 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:16.102018  5070 rpcz_store.cc:276] 0709 12:25:14.562198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.565364 (+  3166us) service_pool.cc:224] Handling call
0709 12:25:14.569977 (+  4613us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.615147 (+ 45170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.615179 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.629190 (+ 14011us) log.cc:844] Serialized 26 byte log entry
0709 12:25:15.047346 (+418156us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:15.047398 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.047489 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.100693 (+1053204us) raft_consensus.cc:1685] finished
0709 12:25:16.100741 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.101035 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49716,"raft.queue_time_us":87097,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":10559,"thread_start_us":316195,"threads_started":1,"wal-append.queue_time_us":485560}
I20260709 12:25:16.103977  9419 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:15.276280  9975 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.105141  9927 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.066027  5076 raft_consensus.cc:2376] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15. Re-sending same reply.
W20260709 12:25:15.806602 10155 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.757s	user 0.006s	sys 0.005s
W20260709 12:25:16.087039  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:15.282953 10053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.005s	sys 0.003s
W20260709 12:25:15.322727 10186 log.cc:927] Time spent T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:25:16.062796  9895 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:15.779726 10166 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:16.050017 10086 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:15.784842 10222 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.297s	user 0.005s	sys 0.000s
W20260709 12:25:15.798344 10173 log.cc:927] Time spent T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.748s	user 0.001s	sys 0.007s
W20260709 12:25:15.485983 10201 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 12:25:15.474514 10196 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:25:15.963660 10200 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:15.313283 10015 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:16.051319  9996 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.048991  9987 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:16.026525  5144 rpcz_store.cc:276] 0709 12:25:14.352291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.353188 (+   897us) service_pool.cc:224] Handling call
0709 12:25:14.353570 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.547980 (+194410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.548012 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.576599 (+ 28587us) log.cc:844] Serialized 26 byte log entry
0709 12:25:14.951958 (+375359us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:14.952019 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.952128 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.951660 (+999532us) raft_consensus.cc:1685] finished
0709 12:25:15.951736 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.952161 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4554,"raft.queue_time_us":17946,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":26077,"thread_start_us":532490,"threads_started":2,"wal-append.queue_time_us":984075}
I20260709 12:25:16.105727 10161 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.094365  5150 rpcz_store.cc:276] 0709 12:25:14.918360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.918504 (+   144us) service_pool.cc:224] Handling call
0709 12:25:15.975475 (+1056971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.038994 (+ 63519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.044932 (+  5938us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.054229  9860 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:16.196555  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 947437us
I20260709 12:25:16.202310  5220 raft_consensus.cc:2410] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:25:15.224424 10148 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:15.385344 10179 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.412s	user 0.002s	sys 0.000s
I20260709 12:25:16.048390 10021 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:16.205870 10204 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.206411 10021 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.206799 10204 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.208043  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:15.479797  9991 raft_consensus.cc:3055] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:16.061960 10104 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.118259  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49640 (request call id 1046) took 1455 ms. Trace:
W20260709 12:25:16.127770 10053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.542s	user 0.009s	sys 0.008s
I20260709 12:25:16.130867  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:16.131016 10088 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.132032  5071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 1042) took 1584 ms. Trace:
I20260709 12:25:15.497813  9934 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:15.492105  9938 consensus_queue.cc:260] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.210510  5076 rpcz_store.cc:276] 0709 12:25:14.656775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.664784 (+  8009us) service_pool.cc:224] Handling call
0709 12:25:16.032014 (+1367230us) spinlock_profiling.cc:243] Waited 8.17 ms on lock 0x7b5800560198. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad5010a0 00007fd8ad500984 00007fd8a6a7b6d4 00007fd8a6a7b662
0709 12:25:16.111962 (+ 79948us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18798464}
I20260709 12:25:15.432430 10043 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:15.322701 10183 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.252s	user 0.000s	sys 0.000s
I20260709 12:25:15.323489 10041 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:16.063584  9898 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.224267  5149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 995) took 1878 ms. Trace:
I20260709 12:25:16.224593 10148 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.106227 10159 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.101854 10144 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.202289  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:16.111052 10189 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:16.230023 10159 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:16.230710  9938 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:16.231567 10206 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.232033  9996 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.234066 10144 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:16.237506  5216 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:16.214761  5071 rpcz_store.cc:276] 0709 12:25:14.535386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.536071 (+   685us) service_pool.cc:224] Handling call
0709 12:25:14.536641 (+   570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.728006 (+191365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.728133 (+   127us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.861607 (+133474us) log.cc:844] Serialized 26 byte log entry
0709 12:25:15.179474 (+317867us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:15.240821 (+ 61347us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.241114 (+   293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.241202 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.089055 (+847853us) raft_consensus.cc:1685] finished
0709 12:25:16.093783 (+  4728us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.119390 (+ 25607us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13701,"prepare.queue_time_us":73067,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":94628,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":9069,"thread_start_us":443154,"threads_started":3,"wal-append.queue_time_us":344915}
I20260709 12:25:16.209779 10161 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:16.239966  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.242169 10214 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:16.141292 10209 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.243263 10214 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.218542  5067 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:16.244366  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.245390  5224 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:16.234146  5142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 994) took 1888 ms. Trace:
I20260709 12:25:16.217808 10104 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.246814  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:15.277900 10197 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.955s
I20260709 12:25:16.246212  5142 rpcz_store.cc:276] 0709 12:25:14.345745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.345880 (+   135us) service_pool.cc:224] Handling call
0709 12:25:14.346260 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.349429 (+  3169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.349469 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.350033 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 12:25:14.556259 (+206226us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:14.556452 (+   193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.556616 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.233461 (+1676845us) raft_consensus.cc:1685] finished
0709 12:25:16.233522 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.233852 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8646,"raft.queue_time_us":3684,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":24734,"thread_start_us":205451,"threads_started":1,"wal-append.queue_time_us":361130}
I20260709 12:25:16.217911 10188 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.245635  5216 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:16.250128 10188 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:16.252978  9865 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.210004  9991 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:16.222165  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 1044) took 1621 ms. Trace:
I20260709 12:25:16.210582  9419 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:16.248746 10197 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:16.243376 10209 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.247110 10088 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.229194 10189 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:16.233110 10159 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:16.122601  9975 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:16.266678  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.255800  5218 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:16.232419 10206 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.261094  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:16.285006  9980 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:16.044478  5069 rpcz_store.cc:276] 0709 12:25:14.632079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.634403 (+  2324us) service_pool.cc:224] Handling call
0709 12:25:14.634871 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.647076 (+ 12205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.647106 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.647689 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 12:25:15.121263 (+473574us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:15.283426 (+162163us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.283744 (+   318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.283849 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.966186 (+682337us) raft_consensus.cc:1685] finished
0709 12:25:15.966238 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.966703 (+   465us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11712,"prepare.queue_time_us":129949,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":50709,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":40122,"thread_start_us":493767,"threads_started":2,"wal-append.queue_time_us":888485}
I20260709 12:25:16.287405 10189 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:16.254259  9872 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:16.228590 10015 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:16.225744  9898 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.284157 10171 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:16.253368 10188 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:16.249401 10104 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.278806  5072 rpcz_store.cc:276] 0709 12:25:14.598511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.644158 (+ 45647us) service_pool.cc:224] Handling call
0709 12:25:15.148289 (+504131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:15.187826 (+ 39537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.218386 (+1030560us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b5400162950. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:16.220225 (+  1839us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071189888}
W20260709 12:25:16.310269  5143 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.007s	sys 0.026s
I20260709 12:25:16.253963  9745 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:16.325484  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.322557  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 1.06872e+06us
I20260709 12:25:16.278178 10197 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:16.254675  9978 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:16.351788  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.352491  9978 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.356494  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.287552 10214 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.357343  5221 raft_consensus.cc:2410] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:25:16.288209 10144 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.285363 10209 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.334050  9927 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:16.224843  5149 rpcz_store.cc:276] 0709 12:25:14.346087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.347341 (+  1254us) service_pool.cc:224] Handling call
0709 12:25:14.347637 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.349793 (+  2156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.349822 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.368059 (+ 18237us) log.cc:844] Serialized 26 byte log entry
0709 12:25:14.675962 (+307903us) spinlock_profiling.cc:243] Waited 36.9 ms on lock 0x7b480004c758. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ad464aa1 00007fd8ad45bb37 00007fd8ad45b9ca 00007fd8ad55326f 00007fd8ad552f37 00007fd8ad56755c 00007fd8ad567508 00007fd8ad567488 00007fd8ad567443 00007fd8ad5666ec 00007fd8ac490a96
0709 12:25:14.706142 (+ 30180us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:14.706191 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.706270 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.223645 (+1517375us) raft_consensus.cc:1685] finished
0709 12:25:16.223685 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.224010 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1742,"raft.queue_time_us":8648,"raft.run_cpu_time_us":3566,"raft.run_wall_time_us":313783,"spinlock_wait_cycles":84837248,"thread_start_us":231672,"threads_started":1,"wal-append.queue_time_us":659356}
I20260709 12:25:16.339776  9872 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.358626  9927 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.359705 10023 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.360642  9872 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.340948 10171 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.286134 10195 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.343623 10163 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:16.309306  9860 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:25:16.346357  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.362313 10041 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:16.363255 10041 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:16.365690 10163 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.366389 10144 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.366981 10041 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:16.316370  9975 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:16.253742 10208 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:16.370111 10208 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.350400  9865 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:16.348150  5219 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:25:16.350497 10165 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.373703  9865 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:16.375599  5072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:16.376324  9865 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:16.351516  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.367170  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1523) took 1450 ms. Trace:
W20260709 12:25:16.377370  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.377985 10171 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:16.380858  5143 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.322s	user 0.007s	sys 0.027s
I20260709 12:25:16.385727 10159 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:16.386359  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.386680 10159 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:16.389196 10206 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.385848  9860 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:16.390216 10206 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.391070 10206 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.362440 10195 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:16.411123  5072 raft_consensus.cc:606] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:16.389348 10159 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:16.418052  5072 raft_consensus.cc:614] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:16.419435  9927 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.420024  9975 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.421415  5218 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:16.427904  9860 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:16.435410  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:16.445305  5224 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:16.436334  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:16.395323 10021 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.367257 10144 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:16.392868 10163 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.402268 10208 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.452105  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:16.453186  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:16.459507  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.460083  5944 raft_consensus.cc:2410] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:16.461159  9978 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.377228  5141 rpcz_store.cc:276] 0709 12:25:14.916666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.916824 (+   158us) service_pool.cc:224] Handling call
0709 12:25:14.917217 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.922518 (+  5301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.922558 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:15.956996 (+1034438us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b2c00193f58. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:25:16.054187 (+ 97191us) log.cc:844] Serialized 26 byte log entry
0709 12:25:16.118367 (+ 64180us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:16.273555 (+155188us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.300622 (+ 27067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.307038 (+  6416us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.366626 (+ 59588us) raft_consensus.cc:1685] finished
0709 12:25:16.366752 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.366945 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4853,"prepare.queue_time_us":88213,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":76,"raft.queue_time_us":28204,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":15957,"spinlock_wait_cycles":7126912,"thread_start_us":59387,"threads_started":1,"wal-append.queue_time_us":243445}
I20260709 12:25:16.461228  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.479826  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:16.486644  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:16.493068  5940 raft_consensus.cc:1240] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 18. Current term is 19. Ops: []
W20260709 12:25:16.512734 10229 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:25:16.516185  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.461190 10041 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:16.518151  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:16.520287  5224 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:16.521016  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:16.522276  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.522982  5218 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:16.523875  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:25:16.525056  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.525674 10095 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:16.518496 10236 log.cc:927] Time spent T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:25:16.550338 10095 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.543226  5947 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 18: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:16.551424 10161 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:16.552220  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:16.553038 10095 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.553805  9988 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:16.543723  9980 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.554337  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.554610  9988 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:16.554977  9980 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:16.556092  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:16.557345  9980 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:16.545809  5114 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:16.546579 10021 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.546084 10197 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.534759  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39870 (request call id 1154) took 2188 ms. Trace:
I20260709 12:25:16.547384  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56170 (request call id 1376) took 2270 ms. Trace:
I20260709 12:25:16.547693 10209 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.557437  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.518065 10041 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:16.544240 10163 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:16.527349  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:16.529148  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.525153  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.540617  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:16.483598 10208 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:16.529286  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:16.490661  9938 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:16.521065  5898 connection.cc:256] RPC call timeout handler was delayed by -1.0749095349997333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21589999999999998s for a response
I20260709 12:25:16.526516 10086 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.588967  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:16.600566  5939 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:16.570930  5942 rpcz_store.cc:276] 0709 12:25:14.346514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.346659 (+   145us) service_pool.cc:224] Handling call
0709 12:25:14.346971 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.347382 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:14.641911 (+294529us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.069110 (+427199us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:15.074562 (+  5452us) log.cc:844] Serialized 26 byte log entry
0709 12:25:15.074766 (+   204us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:15.074802 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.074891 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.534068 (+1459177us) raft_consensus.cc:1685] finished
0709 12:25:16.534128 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.534412 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":82607,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":2337,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":19769,"thread_start_us":720895,"threads_started":2,"wal-append.queue_time_us":613618}
W20260709 12:25:16.589589  5899 leader_election.cc:336] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449): Timed out: RequestConsensusVote RPC to 127.1.123.68:43449 timed out after 1.825s (SENT)
I20260709 12:25:16.590241 10209 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.591514  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.594051 10041 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:16.594655 10163 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:16.596238  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50698us
I20260709 12:25:16.596922 10015 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:16.535465 10144 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:16.615088  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.537819  9978 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.611110  5219 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 17: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:16.611282  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:16.611797  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.621703  5943 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:25:16.622339  5943 pending_rounds.cc:85] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 8
I20260709 12:25:16.612293  5114 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:16.614249  5940 raft_consensus.cc:2393] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 16: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:16.621412  5223 raft_consensus.cc:2376] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16. Re-sending same reply.
I20260709 12:25:16.626044  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:25:16.628172  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.600327 10161 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:16.630831  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.631407  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:16.632478 10200 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:16.632820  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:16.633198 10023 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:16.635687  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.572521  5143 rpcz_store.cc:276] 0709 12:25:14.280995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.281704 (+   709us) service_pool.cc:224] Handling call
0709 12:25:14.282189 (+   485us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:14.331396 (+ 49207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.331453 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:14.332195 (+   742us) log.cc:844] Serialized 52 byte log entry
0709 12:25:14.996788 (+664593us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:16.407293 (+1410505us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.546535 (+139242us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.546918 (+   383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.547061 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.547115 (+    54us) raft_consensus.cc:1685] finished
0709 12:25:16.547128 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.547297 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74413,"prepare.queue_time_us":858846,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":108,"raft.queue_time_us":35926,"raft.run_cpu_time_us":557,"raft.run_wall_time_us":48365,"thread_start_us":1924032,"threads_started":2,"wal-append.queue_time_us":876408}
I20260709 12:25:16.622768  5943 pending_rounds.cc:107] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:25:16.657740  5943 raft_consensus.cc:2889] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.660861  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.629812 10209 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.663605  5221 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
W20260709 12:25:16.669656  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 45863us
I20260709 12:25:16.538480  9991 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.670876 10144 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:16.671221  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 83817.1us
I20260709 12:25:16.672199  5219 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:16.673107 10163 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.629041 10021 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:16.661192  9978 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:16.659152 10002 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.005s
I20260709 12:25:16.621922  9938 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.658856 10023 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.622187 10208 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:16.678452 10041 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.689827  9975 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:16.712118  9975 consensus_queue.cc:1243] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:16.690302  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.712996  5221 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:16.691243  9978 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:16.680145 10144 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.714500 10144 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:16.717758 10181 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.514s	user 0.009s	sys 0.009s
I20260709 12:25:16.718533 10181 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.722879 10208 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:16.692276  5223 raft_consensus.cc:1240] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:16.692646  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.693048 10092 raft_consensus.cc:2764] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:25:16.693390  5042 leader_election.cc:400] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13.
I20260709 12:25:16.651754 10086 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:16.694058 10197 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:16.682744 10171 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.689970  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:16.691255  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.748528  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.704281  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.702807 10023 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:16.706944  9938 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:16.718544  9843 consensus_meta.cc:287] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.006s
I20260709 12:25:16.711283 10209 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.713374 10021 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:16.714560  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.694098  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:16.761590 10209 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.709950 10041 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.762728  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:16.763266  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 69413.8us
I20260709 12:25:16.764160 10209 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:16.755326 10246 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:25:16.704416 10043 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
W20260709 12:25:16.768194  5220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 948) took 4388 ms (client timeout 2574 ms). Trace:
W20260709 12:25:16.754351  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 216779us
I20260709 12:25:16.760793  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1524) took 1842 ms. Trace:
W20260709 12:25:16.755944  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 123270us
I20260709 12:25:16.759408  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1260) took 2940 ms. Trace:
I20260709 12:25:16.764042  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:16.769944 10208 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:16.770433  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 80604.2us
I20260709 12:25:16.772367  5944 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:16.769363  9860 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:16.809031  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.768394 10181 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.808969  5072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "90e5220a62ed42fe83c82d0302d18fac"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:16.778098  5941 rpcz_store.cc:276] 0709 12:25:13.818884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.879590 (+ 60706us) service_pool.cc:224] Handling call
0709 12:25:13.879943 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.120956 (+2241013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.759079 (+638123us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b5400006c54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:16.759335 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4876460928}
W20260709 12:25:16.816083  9419 consensus_meta.cc:287] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.518s	user 0.015s	sys 0.000s
I20260709 12:25:16.817416  5042 leader_election.cc:400] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12.
I20260709 12:25:16.817692  5072 raft_consensus.cc:606] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Received request to transfer leadership to TS 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:16.818459  5072 raft_consensus.cc:614] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:16.818292 10171 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:16.818915  5042 leader_election.cc:400] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12.
I20260709 12:25:16.819790  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:16.776592  5145 rpcz_store.cc:276] 0709 12:25:14.917853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.918013 (+   160us) service_pool.cc:224] Handling call
0709 12:25:14.918357 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:15.954108 (+1035751us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:15.955043 (+   935us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.079975 (+124932us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.080060 (+    85us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:16.092609 (+ 12549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.760362 (+667753us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b5400238f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:16.760424 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.760595 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3402,"prepare.queue_time_us":30615,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":88,"raft.queue_time_us":7497,"raft.run_cpu_time_us":584,"raft.run_wall_time_us":46687,"spinlock_wait_cycles":787359616,"thread_start_us":114991,"threads_started":1,"wal-append.queue_time_us":260302}
I20260709 12:25:16.776336  9843 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.826388  9745 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:16.774098  5220 rpcz_store.cc:269] 0709 12:25:12.379765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.379893 (+   128us) service_pool.cc:224] Handling call
0709 12:25:16.767813 (+4387920us) spinlock_profiling.cc:243] Waited 3.76 s on lock 0x7b5400114254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:16.768060 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8655999744}
I20260709 12:25:16.826498  5042 leader_election.cc:400] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10.
I20260709 12:25:16.833240  5042 leader_election.cc:403] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:16.844729 10216 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.845461 10216 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.846746  5072 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:25:16.848912  5189 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:16.849078 10046 consensus_queue.cc:1059] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:16.849983  9879 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:16.850436  9879 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.850865  9879 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:16.852629  5114 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.508s (SENT)
I20260709 12:25:16.853703 10046 consensus_queue.cc:1243] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 19.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:16.855322 10021 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.856189  9888 raft_consensus.cc:3055] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:16.856011 10021 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:16.856899  9888 raft_consensus.cc:740] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:16.858466  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.859359 10021 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.858536  9888 consensus_queue.cc:260] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:16.860536 10086 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:16.860841  9888 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:16.901073  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.836267 10104 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
W20260709 12:25:16.944850  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.908517  5145 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:25:16.904340  9419 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 9. Current term is 10.
W20260709 12:25:16.910424  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 84122.1us
I20260709 12:25:16.910979  5143 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: [11.7-12.8]
W20260709 12:25:16.913693 10241 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.234s	user 0.001s	sys 0.003s
W20260709 12:25:16.914793 10243 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.235s	user 0.002s	sys 0.000s
I20260709 12:25:16.916016 10041 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:16.919267  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:16.941686 10181 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.921033  5070 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:16.920936  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:16.921990  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:16.922207  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:16.922571  9865 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.923223  9799 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:16.925822  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.930974  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1256) took 3231 ms. Trace:
I20260709 12:25:16.933570  5188 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:16.934768 10206 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.933182  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 1043) took 2352 ms. Trace:
I20260709 12:25:16.940809  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:16.895839 10204 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:16.920181  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:16.971710  5068 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 16: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:16.974050 10171 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:16.974973  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:16.982565  5150 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: [10.6-11.7]
I20260709 12:25:16.984350  5070 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:16.984586  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:16.907588 10043 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:25:16.906718 10086 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:25:16.993253  5141 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:16.972085  5948 rpcz_store.cc:276] 0709 12:25:13.699579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.703450 (+  3871us) service_pool.cc:224] Handling call
0709 12:25:13.703898 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:13.705160 (+  1262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.705194 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:13.705829 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 12:25:14.339698 (+633869us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:14.339747 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.339826 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.805698 (+1465872us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:16.929752 (+1124054us) spinlock_profiling.cc:243] Waited 95.8 ms on lock 0x7b2400037448. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a52d49c4 00007fd8a52d2acb 00007fd8a548ddc9 00007fd8a548da55 00007fd8a548cfb0 00007fd8a8103488 00007fd8a8103be8 00007fd8a80db398 00007fd8a80d676a 00007fd8a80d5bb6
0709 12:25:16.930552 (+   800us) raft_consensus.cc:1685] finished
0709 12:25:16.930572 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.930773 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169,"raft.queue_time_us":2146,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":9125,"spinlock_wait_cycles":250377856,"thread_start_us":633383,"threads_started":1,"wal-append.queue_time_us":1154450}
I20260709 12:25:16.998018  5071 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:17.008813  5074 raft_consensus.cc:1240] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:25:17.009250 10204 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:17.009485  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:16.976640  5073 rpcz_store.cc:276] 0709 12:25:14.581047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.616784 (+ 35737us) service_pool.cc:224] Handling call
0709 12:25:15.942579 (+1325795us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b5800560198. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:25:15.943393 (+   814us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.194306 (+250913us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:16.467069 (+272763us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.601246 (+134177us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:16.601818 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 12:25:16.601967 (+   149us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:16.602158 (+   191us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.602356 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.932685 (+330329us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54006effd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:16.932742 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.932795 (+    53us) raft_consensus.cc:1685] finished
0709 12:25:16.932809 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.932967 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":467974,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":3524,"raft.queue_time_us":93694,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":31816,"spinlock_wait_cycles":455082368,"thread_start_us":211440,"threads_started":2,"wal-append.queue_time_us":185399}
W20260709 12:25:17.008630  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 75378.4us
I20260709 12:25:17.012218  5188 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:17.013841 10214 consensus_queue.cc:1048] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:17.015584 10214 raft_consensus.cc:2804] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:17.016215 10214 raft_consensus.cc:697] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:17.013924  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:17.009629 10166 consensus_meta.cc:287] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.714s	user 0.015s	sys 0.000s
I20260709 12:25:16.985504 10041 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.010963  5073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5751113d37b44af87d3d912da388078"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:17.024967  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 1053) took 1043 ms. Trace:
I20260709 12:25:17.030031  5215 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:17.027899  5042 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:17.029633  5074 rpcz_store.cc:276] 0709 12:25:15.982055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.055087 (+ 73032us) service_pool.cc:224] Handling call
0709 12:25:16.844117 (+789030us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b5800560198. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5416b71 00007fd8a5416a22 00007fd8ae32f8d9 00007fd8ad489ee0 00007fd8ad5530ab 00007fd8ad553131 00007fd8ad514fc0 00007fd8ad4ff929 00007fd8ad4ff3d2 00007fd8a6a76bd4
0709 12:25:16.897169 (+ 53052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:17.009340 (+112171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.024892 (+ 15552us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5715200}
I20260709 12:25:17.035004  5073 raft_consensus.cc:606] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:17.036038  5073 raft_consensus.cc:614] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:16.994403 10171 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.980708  9865 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:17.042665 10041 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:17.049779 10189 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:17.050568  5150 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:17.053304 10216 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:17.055061  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 54500.1us
I20260709 12:25:17.056049  5042 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:16.990341 10181 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:17.049294 10041 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:17.065274  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:17.070575  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:17.071815  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.072687  5143 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:17.071343  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.081840  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:17.017339 10214 consensus_queue.cc:237] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:17.060827  9843 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:17.092715  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.002923  9799 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:17.097604  5141 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:25:17.102263  5188 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:17.104746  5145 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:17.105232 10204 consensus_queue.cc:1059] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:17.107467  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:17.107256 10206 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:17.109004  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:17.111167  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:17.082016 10171 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:17.110682 10189 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:17.109647  9978 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:17.115736 10197 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:17.130858  5143 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 12:25:17.123910  9975 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.134459  9975 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:17.136937  9975 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:17.138227  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.139243  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.139972  5141 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:17.140967  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.141806  5146 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 17: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:17.145023  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:17.147197  9799 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:17.138105  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:17.141686  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.130853 10002 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.149910  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:17.151113  9991 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:17.151633  5220 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 19.
I20260709 12:25:17.150206 10002 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:17.141660 10189 raft_consensus.cc:697] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:17.150786  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 85603.5us
I20260709 12:25:17.138278  5145 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:25:17.148619  5222 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:17.154820  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:17.149398  5144 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:17.126341 10216 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:17.165994  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.128679 10204 raft_consensus.cc:3055] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:17.167654  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 319307us
I20260709 12:25:17.168419  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:17.172423  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 54394.1us
I20260709 12:25:17.173448  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:17.173503  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:17.153095 10144 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:17.178223  5219 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:17.183645  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:17.178963  9975 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:17.185345  9975 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.185837  9975 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:17.186322  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:17.186996  5224 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:17.149330  9879 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:17.189007  9879 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:17.167960 10204 raft_consensus.cc:740] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:17.190002  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.184859  5215 raft_consensus.cc:2410] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:25:17.183979  5143 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:17.192363  5042 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:17.133678 10023 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:17.211599  5075 raft_consensus.cc:2368] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16.
I20260709 12:25:17.213065  5188 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:17.215180  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:17.216050  5215 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:17.218564  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:17.199810 10166 raft_consensus.cc:2764] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:25:17.239212 10023 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:17.242318  9745 raft_consensus.cc:3055] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:17.247360  5142 raft_consensus.cc:2393] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 13: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:17.236977  9860 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:25:17.250020  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.254331  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:17.257879  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:17.259128  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:17.269970  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 453023us
I20260709 12:25:17.270828  5218 raft_consensus.cc:2393] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 12: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:17.228858  9927 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:17.246070 10197 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:17.272060 10191 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.005s	sys 0.028s
I20260709 12:25:17.235412  9938 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:17.273097 10206 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:17.271417  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:17.307353  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:17.272364  5215 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:17.253904  9745 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:17.234148 10171 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:17.326242 10144 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:17.311079  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:17.327239  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 145164us
I20260709 12:25:17.293462  5224 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
W20260709 12:25:17.306502  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.308984  5073 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:17.311218  5941 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:17.312127  5948 raft_consensus.cc:2410] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:25:17.304430 10189 consensus_queue.cc:237] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:17.315918  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:17.320771 10002 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:17.321925  5069 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:25:17.300894 10191 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.977s	user 0.006s	sys 0.028s
I20260709 12:25:17.323734  9745 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:17.322887  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:17.339290  5939 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:17.335054  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:17.301095 10181 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:17.344395  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45846.1us
I20260709 12:25:17.336398  5217 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:17.294097  9978 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:17.347996  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 103745us
I20260709 12:25:17.349449  5068 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:25:17.349767 10261 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:17.363703 10261 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:17.364107  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:17.366024  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.367096 10023 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:17.367895  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:17.367760 10023 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:17.369894  5042 leader_election.cc:400] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17.
I20260709 12:25:17.370337  5042 leader_election.cc:403] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:17.370427 10023 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:17.371737  5042 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:17.375495  5042 leader_election.cc:400] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15.
I20260709 12:25:17.375820  5042 leader_election.cc:403] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:17.372597  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:17.372793 10257 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:17.377285  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:17.373355  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:17.373941  9843 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17.
I20260709 12:25:17.361976  5939 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:17.335175 10041 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:17.361703  5068 pending_rounds.cc:85] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 7
I20260709 12:25:17.382192  5899 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:17.364207 10261 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:17.396797  5068 pending_rounds.cc:107] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:25:17.393496  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:17.399138  5074 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:17.400197  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:17.402997  5944 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:17.404927  5068 raft_consensus.cc:2889] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:17.332422  9927 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:17.407660  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:17.407928  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:17.413342  5939 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:17.394145 10144 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15.
I20260709 12:25:17.392568 10181 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:17.410071  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:17.415166  5948 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 15: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:17.418744  5145 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: [12.6-13.7]
I20260709 12:25:17.420365  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:17.424829  5143 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:17.397261 10002 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:17.425266  5144 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:17.431504  5942 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:25:17.449575  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:17.461925  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 86600.9us
I20260709 12:25:17.471774  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:17.402271  9978 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:17.472685  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:17.469894  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.464504  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:17.479499  5939 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:17.483497  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:17.426483  9975 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:17.482628  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:17.490294  5148 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:17.491143 10256 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 12:25:17.493043  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.494082  5943 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:25:17.498459 10086 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:17.498945  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:17.506582  9888 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.507225  9975 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:17.486579 10209 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.508323 10209 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:17.510571 10209 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:17.511327 10209 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:17.511791 10209 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:17.513918  5945 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:17.494589  5943 pending_rounds.cc:85] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 7
I20260709 12:25:17.518395  5943 pending_rounds.cc:107] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:25:17.519026  5943 raft_consensus.cc:2889] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:17.519760  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:17.522591  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 140505us
I20260709 12:25:17.524587  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:17.525434  5215 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:17.528246  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.528458  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:25:17.477521 10123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.008s	sys 0.020s
I20260709 12:25:17.529608  5220 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:25:17.540208  5943 pending_rounds.cc:107] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:25:17.540792  5943 raft_consensus.cc:2889] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:17.546041  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.571813  9975 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.572715  9975 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:17.576328  9975 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:17.584131  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.585309  5145 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 16: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:17.580129  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:17.578462 10123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.009s	sys 0.020s
I20260709 12:25:17.590826  5944 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:17.498569  9927 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:17.586612  5143 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:17.602034  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:17.608161  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:17.608280  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.507228  9888 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:17.610994  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:17.638772  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.606431 10209 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:17.653265  5143 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:17.643285  5941 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:17.660149  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.661127  5143 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:17.658104  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:17.662770  5944 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:17.663365  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 174733us
W20260709 12:25:17.664399  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 67293us
I20260709 12:25:17.673189  5215 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 20.
I20260709 12:25:17.723963  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:17.724448  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 59183us
I20260709 12:25:17.724944  5941 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 20
W20260709 12:25:17.728478  5041 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.65:37253 timed out after 1.719s (SENT) [suppressed 42 similar messages]
I20260709 12:25:17.757566  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.763118  5142 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 18: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:17.770804 10268 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:17.771847 10268 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:17.775105 10268 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:17.795241  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 112738us
I20260709 12:25:17.807134  5148 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:17.804491  5899 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:17.809741  9978 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:17.821084  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 299908us
I20260709 12:25:17.824544  5142 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:17.840852  5941 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 20.
I20260709 12:25:17.847781  9988 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:17.850468  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 122050us
I20260709 12:25:17.855795  5142 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:17.870676  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:17.835471 10273 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:17.874836 10268 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.878976  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:17.891776  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 51903.6us
I20260709 12:25:17.891757  5074 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
W20260709 12:25:17.896308  5114 leader_election.cc:336] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.540s (SENT)
I20260709 12:25:17.900089  5074 raft_consensus.cc:1240] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 12:25:17.898731  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 163055us
W20260709 12:25:17.914759  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 115322us
I20260709 12:25:17.913475  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.872484  9888 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:17.906847 10264 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 12:25:17.920907  5142 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 17: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:17.898110 10263 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:17.923955  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:17.898416  9978 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:17.930915  5898 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 2.159s (SENT)
I20260709 12:25:17.936374  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.898331 10268 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:17.931985 10181 consensus_meta.cc:287] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.001s
W20260709 12:25:17.945158  5942 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.000s
W20260709 12:25:17.946687 10270 log.cc:927] Time spent T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.127s	user 0.001s	sys 0.001s
W20260709 12:25:17.949651 10275 log.cc:927] Time spent T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:25:17.926681 10273 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:25:17.955714  5147 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:17.957882  5042 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:17.960363  5942 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:17.961344  5220 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:17.933553 10263 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:17.950544 10271 log.cc:927] Time spent T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 12:25:17.971910  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 48686.4us
I20260709 12:25:17.973400  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:17.976985  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:17.938819 10144 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:18.026685  5075 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:25:18.027758  5900 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:18.028499  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 102544us
I20260709 12:25:18.030421  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:18.000730  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.011759  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.928186 10002 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:25:18.027300  5075 pending_rounds.cc:85] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 6
W20260709 12:25:18.042173  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 108478us
I20260709 12:25:18.044273  5075 pending_rounds.cc:107] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:25:18.046234  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:18.046964  5075 raft_consensus.cc:2889] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:18.048926  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:17.970403 10181 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:18.051972  5215 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 19.
W20260709 12:25:18.054092  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.982403 10263 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:18.058636 10272 log.cc:927] Time spent T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 12:25:18.058423  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:18.068243  5943 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:18.059725  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 84798.2us
I20260709 12:25:18.060353 10257 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:18.065207  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:18.066468  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 142749us
W20260709 12:25:18.066720  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 216643us
I20260709 12:25:18.070544  5941 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 19.
I20260709 12:25:17.981914  9978 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:18.062095 10265 log.cc:927] Time spent T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 12:25:18.080050  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:18.080248  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:18.083937  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.084040  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:18.087899  5944 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:18.062816  9975 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:25:18.090308  5217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 972) took 1639 ms (client timeout 2018 ms). Trace:
I20260709 12:25:18.102305  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.061828 10277 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.092514  5217 rpcz_store.cc:269] 0709 12:25:16.436234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.452948 (+ 16714us) service_pool.cc:224] Handling call
0709 12:25:18.075386 (+1622438us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b540031d8d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:18.075570 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1292710272}
I20260709 12:25:18.072050 10257 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.105317  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:18.112246  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 83446.8us
I20260709 12:25:18.073673  9978 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:18.115572  5042 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:18.117385  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:18.118961 10288 log.cc:927] Time spent T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 12:25:18.120429 10254 log.cc:927] Time spent T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.832s	user 0.001s	sys 0.008s
I20260709 12:25:18.125878  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:18.125757 10279 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:18.127508 10279 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:18.127761  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:18.111976  9419 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:18.131253  9419 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:18.131624  9419 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:18.131942  9419 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:18.132344 10280 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:18.133355 10280 raft_consensus.cc:697] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 17 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:18.134645 10280 consensus_queue.cc:237] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:18.132681 10041 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.730s	user 0.005s	sys 0.005s
I20260709 12:25:18.137693 10041 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:18.139151  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1269) took 1724 ms. Trace:
I20260709 12:25:18.139631  5946 rpcz_store.cc:276] 0709 12:25:16.414434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.467402 (+ 52968us) service_pool.cc:224] Handling call
0709 12:25:16.521630 (+ 54228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.597749 (+ 76119us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:16.730660 (+132911us) log.cc:844] Serialized 14 byte log entry
0709 12:25:17.161743 (+431083us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:17.161796 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:17.175065 (+ 13269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.138195 (+963130us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b180026d280. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a541c2a6 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419
0709 12:25:18.138772 (+   577us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b54002422d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:18.138821 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.138958 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":126110,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":64,"raft.queue_time_us":2305,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":8368,"spinlock_wait_cycles":212180608,"thread_start_us":448828,"threads_started":2,"wal-append.queue_time_us":562508}
I20260709 12:25:18.146344  5900 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:18.148890  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:18.091320 10191 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:25:18.110212 10261 consensus_meta.cc:287] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.689s	user 0.016s	sys 0.006s
I20260709 12:25:18.159220  5943 raft_consensus.cc:2468] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
W20260709 12:25:18.162779 10285 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:25:18.113005 10257 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:18.148078  9978 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:18.155663 10273 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:18.157598 10191 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:18.182991  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 99017.8us
I20260709 12:25:18.172799  9927 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:18.190807  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:18.195737  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:18.150171 10277 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:18.199872  5944 raft_consensus.cc:1240] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:18.200654  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 93922.1us
W20260709 12:25:18.204553  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 55734.2us
I20260709 12:25:18.205862  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.208098  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:18.179522 10273 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:18.172540  9988 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:25:18.183295 10191 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:18.167866 10261 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.209214 10278 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
W20260709 12:25:18.222642 10282 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:25:18.193244  9927 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:18.185881 10279 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:25:18.223603 10303 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:18.214226 10189 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:18.215250 10273 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:18.223569 10292 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:25:18.180750 10165 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:25:18.212100 10002 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:18.222688 10297 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:18.230973 10171 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.014s	sys 0.010s
I20260709 12:25:18.236070  9927 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:18.262809  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 147292us
I20260709 12:25:18.264048  5042 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.186108 10204 consensus_queue.cc:260] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.237501 10043 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:18.268246 10189 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:18.268877 10297 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.271487 10299 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:18.274003 10277 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:18.290645  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.275269 10002 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:18.276613  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:18.277942  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:18.284942  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 77090.8us
I20260709 12:25:18.234580  9975 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:18.286926  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 113960us
W20260709 12:25:18.260506 10286 log.cc:927] Time spent T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 12:25:18.238857 10268 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:18.265985 10171 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.015s	sys 0.011s
I20260709 12:25:18.296525  5067 raft_consensus.cc:2410] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:25:18.290023  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 87517.9us
I20260709 12:25:18.299535  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:18.244846 10191 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.334106  5115 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:18.335336 10277 raft_consensus.cc:2804] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:18.336860 10295 log.cc:927] Time spent T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:25:18.338047  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:18.338744 10267 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.605s	user 0.000s	sys 0.001s
I20260709 12:25:18.338371  9975 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.339648  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.341254  5899 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:18.289572 10302 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:18.294225 10204 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:18.294505 10043 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:18.348269  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:18.350760  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.351270  5075 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:25:18.370797  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 224548us
I20260709 12:25:18.335326  9988 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:18.377594  5067 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:18.346257 10302 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:18.382201 10206 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.383136  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.382215  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:18.384021  5067 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:18.386656  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.391933  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:18.399024  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 69243.6us
I20260709 12:25:18.373921 10304 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:18.385259 10171 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.384878  9988 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 18 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:18.413331  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.415164  5067 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:18.383692 10302 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:18.415562  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.409049  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.419626  5074 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:25:18.400592 10277 raft_consensus.cc:697] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:18.425741  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.383249 10206 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:18.429039  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.409622 10304 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:18.435993  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:18.435187  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:18.441756  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:18.444192  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:18.444870  5220 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:18.443140  5114 leader_election.cc:336] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.619s (SENT)
I20260709 12:25:18.448185  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:18.449255  5215 raft_consensus.cc:2410] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:25:18.427556 10261 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:18.453848  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.456331  5217 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:25:18.458170  5067 raft_consensus.cc:2410] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:18.434475 10304 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:18.459491  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:18.407829 10002 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.463441  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:18.482235  5073 raft_consensus.cc:2410] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:25:18.425990 10043 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.483290  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:18.484512  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.445812 10279 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:18.485500 10279 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:18.486413  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.488590 10279 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:18.446033 10305 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:18.465685 10311 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:18.465920  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:18.490082 10305 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:18.490684 10305 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:18.491511 10189 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.492133 10189 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:18.489429 10281 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.526s	user 0.002s	sys 0.000s
I20260709 12:25:18.466924  9988 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:18.468679  5114 leader_election.cc:336] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.644s (SENT)
I20260709 12:25:18.469528  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:18.505295  5217 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:18.469682 10301 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:25:18.470228 10191 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:18.541740  5114 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:18.494879 10189 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:18.507982 10273 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:18.426914 10171 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:18.470110 10260 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:25:18.446043 10299 consensus_queue.cc:1059] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:25:18.471654  5215 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:18.473901 10297 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:18.476583 10263 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:18.477603 10303 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:18.479069 10206 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:18.480441 10257 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:18.481004  9978 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:18.489351  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.496481  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.509188  5220 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
W20260709 12:25:18.525574  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.530915  9419 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:18.532387  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:18.615195  5946 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:18.615453  5942 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:18.533844  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:18.573319  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 128811us
I20260709 12:25:18.581403  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 1031) took 1205 ms. Trace:
I20260709 12:25:18.585877  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:18.593689  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 125065us
I20260709 12:25:18.596145 10299 raft_consensus.cc:3055] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:18.597012  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 224626us
I20260709 12:25:18.621510  5217 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 20.
I20260709 12:25:18.618202  5939 rpcz_store.cc:276] 0709 12:25:17.376065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.480433 (+104368us) service_pool.cc:224] Handling call
0709 12:25:17.488892 (+  8459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:17.496195 (+  7303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:17.496237 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:17.496830 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 12:25:17.764760 (+267930us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:17.764809 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.764920 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.443354 (+678434us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:18.546050 (+102696us) raft_consensus.cc:1685] finished
0709 12:25:18.546075 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.581279 (+ 35204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11637504,"thread_start_us":267436,"threads_started":1,"wal-append.queue_time_us":418465}
I20260709 12:25:18.579447 10312 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:18.633899  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:18.635097  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 194244us
I20260709 12:25:18.636070  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:18.639003  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.641207  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.642293  5144 raft_consensus.cc:2410] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:25:18.642577  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.626210 10299 raft_consensus.cc:740] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:25:18.650753  5114 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Timed out: RequestConsensusVote RPC to 127.1.123.66:33955 timed out after 1.813s (SENT)
I20260709 12:25:18.651449  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.654541  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.541787 10277 consensus_queue.cc:237] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:18.629249 10041 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:25:18.660122  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.660739  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.661293  5939 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:18.663332  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.663614  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.661509  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.666095  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.604179 10304 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:18.671931  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.655398  5076 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
W20260709 12:25:18.667330  5042 connection.cc:256] RPC call timeout handler was delayed by -2.3290452589999404s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:25:18.660482  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.653887 10206 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:18.670502  5149 raft_consensus.cc:2410] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:18.672614  5073 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:18.675171  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:18.673692  5147 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:18.680378  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:18.656195 10297 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:25:18.683326  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:18.693930  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.682426  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:18.670018 10158 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:25:18.678359 10206 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.697223  5143 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:18.695605  5073 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:18.696002 10189 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.704510 10312 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:18.707617 10304 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:18.670038 10310 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:25:18.714576  9419 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:18.718616  9978 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:18.691506  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.680474 10273 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:18.722527  5071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1663) took 1432 ms. Trace:
W20260709 12:25:18.729106  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 78412.9us
I20260709 12:25:18.730476  5071 rpcz_store.cc:276] 0709 12:25:17.283754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.283967 (+   213us) service_pool.cc:224] Handling call
0709 12:25:17.297773 (+ 13806us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:17.325290 (+ 27517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:17.325327 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:17.325984 (+   657us) log.cc:844] Serialized 52 byte log entry
0709 12:25:17.641846 (+315862us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:17.641891 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.716082 (+1074191us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b54003570d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:18.716156 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.716220 (+    64us) raft_consensus.cc:1685] finished
0709 12:25:18.716236 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.716509 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2190,"raft.queue_time_us":10066,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":3845,"spinlock_wait_cycles":62161280,"thread_start_us":315395,"threads_started":1,"wal-append.queue_time_us":436229}
I20260709 12:25:18.733289  5042 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.734161  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:18.731271  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 48989.4us
I20260709 12:25:18.738256  5073 raft_consensus.cc:2410] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:25:18.754408  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.756023  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:18.724386 10206 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:18.763584  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.761490  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.762408  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.765756  5942 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:18.765848  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.765239  5939 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 18: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:18.766422  5073 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:18.683462 10043 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:25:18.769449  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.728688 10318 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.774032  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:18.777669  5939 raft_consensus.cc:3060] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:18.779865  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.780525  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.781850  5942 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 17: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:18.686821  9927 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:18.780611  5114 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:18.783808  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.737134 10280 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:18.723783 10189 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:18.758836 10321 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:18.760150  9978 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:18.790241  5071 raft_consensus.cc:2393] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:18.767781  9419 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:18.791685  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 112085us
I20260709 12:25:18.797643  5942 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:18.802083  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.767532  9988 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:25:18.791607 10002 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:18.697080 10299 consensus_queue.cc:260] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.809505  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.753490 10305 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:18.788338 10257 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:18.795944  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.812347  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:18.812911  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 51564.7us
W20260709 12:25:18.815860  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 59995.2us
I20260709 12:25:18.816504  5076 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:18.817890  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.818188  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.812443  9975 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.819270  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.819586  9975 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.820545  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.808522 10171 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:18.821736  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.822634  5073 raft_consensus.cc:2393] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:18.823791  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.824474  5076 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:18.825775  5076 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:25:18.826140  5076 pending_rounds.cc:85] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 7
I20260709 12:25:18.826393  5076 pending_rounds.cc:107] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:25:18.826618  5076 raft_consensus.cc:2889] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:18.827456 10319 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:25:18.828066 10189 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:18.828881 10332 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:18.804764 10321 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.829579 10332 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.831498 10321 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:18.831993 10332 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:18.834635  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.835340  5143 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:18.836218  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.837064  5945 raft_consensus.cc:2368] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16.
I20260709 12:25:18.837350  9988 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:18.838115  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.838557 10298 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:18.838855  5143 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:18.839049  9988 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:18.804232 10263 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:18.808917 10297 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:18.840325 10297 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:18.842869 10297 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:18.813342 10315 consensus_queue.cc:1059] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:25:18.844367  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.845249  5143 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
W20260709 12:25:18.847204 10262 log.cc:927] Time spent T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 1.384s	user 0.003s	sys 0.000s
I20260709 12:25:18.839365 10302 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.851797  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.849782  9978 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:18.852670  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:18.853901  9978 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:18.854719  9978 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:18.855154  5073 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:18.854725  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:18.857565  5143 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:18.859354  5946 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 20.
I20260709 12:25:18.860163  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:18.862234  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:18.862313  5042 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:18.863938  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:18.864711  5146 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:18.866398  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.869823  5217 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:18.870460  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.875610 10189 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:18.876192  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.878031  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:18.877125  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 96665.1us
I20260709 12:25:18.879251  5220 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:25:18.879608  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:18.880795  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.881651 10191 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:18.879262 10189 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:18.883395  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.885206  5042 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.885226 10189 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:18.886475 10191 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:18.888424  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:18.822634 10322 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:25:18.796671 10280 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:18.895983  5042 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:18.856822 10302 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:18.893553  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:18.894299  5939 raft_consensus.cc:2468] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:18.894758  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:18.815518 10312 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:18.864660  9988 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:18.855890  9978 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:18.784675 10318 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:18.910386  5215 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:25:18.912441  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.922235  5224 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:18.892556  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:18.914454  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 239024us
I20260709 12:25:18.916018  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:18.917186  5071 raft_consensus.cc:2368] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14.
I20260709 12:25:18.891340  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:18.925201  5147 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:18.918169  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1649) took 2000 ms. Trace:
I20260709 12:25:18.873924  9419 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:18.926575  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1661) took 1650 ms. Trace:
I20260709 12:25:18.912503 10002 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:18.927125  5068 rpcz_store.cc:276] 0709 12:25:17.276446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.285661 (+  9215us) service_pool.cc:224] Handling call
0709 12:25:17.302719 (+ 17058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.410880 (+108161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.926252 (+1515372us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b540058a054. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:18.926488 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106423552}
I20260709 12:25:18.923146  5073 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:18.928390  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.942220  5943 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:18.925815  5072 rpcz_store.cc:276] 0709 12:25:16.917818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.917992 (+   174us) service_pool.cc:224] Handling call
0709 12:25:16.918501 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.918979 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.919009 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:16.919553 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 12:25:17.285017 (+365464us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:17.285078 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.285155 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.846378 (+1561223us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:18.911867 (+ 65489us) raft_consensus.cc:1685] finished
0709 12:25:18.911889 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.918070 (+  6181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2511,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":113,"spinlock_wait_cycles":68526336,"thread_start_us":351190,"threads_started":1,"wal-append.queue_time_us":463894}
I20260709 12:25:18.920935  9988 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:18.839241 10298 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:18.960640  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.954964  5946 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:18.938711 10002 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.923528 10312 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:18.919934 10189 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:18.925107  9975 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:18.961282  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:18.942346 10191 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:18.959638  9927 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:18.927410  9419 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:18.969501  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 87899us
I20260709 12:25:18.923369 10268 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:25:18.973717 10158 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:18.983062  5189 leader_election.cc:400] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16.
I20260709 12:25:18.983485  5189 leader_election.cc:403] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:18.978773  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 62904.2us
I20260709 12:25:18.973407 10297 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.969264 10002 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:18.972704 10189 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.947198 10334 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:18.973896 10206 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:18.986497  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:18.980661  5114 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:18.981483 10191 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.984906  9865 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:18.992962 10298 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:18.993641 10307 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:25:19.008199 10197 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:18.994091 10335 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16.
I20260709 12:25:19.009090 10158 raft_consensus.cc:3055] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:19.009616 10158 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:19.009830 10305 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:19.008235 10268 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:19.010692 10268 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.011150 10336 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.136s
I20260709 12:25:19.013394 10302 consensus_queue.cc:1059] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:18.988783 10297 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:19.002951 10269 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 1.158s	user 0.000s	sys 0.004s
I20260709 12:25:19.014413 10302 raft_consensus.cc:3055] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:18.983703  9419 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:19.010653 10158 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.007807  5189 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:18.992591  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1565) took 1474 ms. Trace:
I20260709 12:25:19.013432  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:19.016789  5114 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:19.019747  5219 rpcz_store.cc:276] 0709 12:25:17.517978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.528156 (+ 10178us) service_pool.cc:224] Handling call
0709 12:25:17.584528 (+ 56372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.327521 (+742993us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.327562 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:18.327620 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:18.327652 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.992317 (+664665us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b54004f2ed4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:18.992373 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.992508 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1708095744}
I20260709 12:25:19.001680 10189 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:19.025575  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:18.972132 10312 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:19.006093 10191 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:19.000599  9975 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:25:19.040546  5898 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:19.030500  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:19.042943  5898 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:19.009254 10280 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:19.029986  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:19.032806  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1666) took 1741 ms. Trace:
I20260709 12:25:19.018224 10336 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:19.047626  5898 leader_election.cc:400] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14.
I20260709 12:25:19.046744  5070 rpcz_store.cc:276] 0709 12:25:17.291075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.291211 (+   136us) service_pool.cc:224] Handling call
0709 12:25:17.303528 (+ 12317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:17.321792 (+ 18264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:17.321822 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:17.322358 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 12:25:17.659236 (+336878us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:17.659494 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.659898 (+   404us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.998093 (+1338195us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:19.032445 (+ 34352us) raft_consensus.cc:1685] finished
0709 12:25:19.032476 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.032726 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10372,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":4546,"spinlock_wait_cycles":14988160,"thread_start_us":335712,"threads_started":1,"wal-append.queue_time_us":477146}
I20260709 12:25:19.052322  5114 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:19.052436  5143 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:19.053268 10342 consensus_queue.cc:1059] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:19.055030 10342 raft_consensus.cc:3055] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:19.055574 10342 raft_consensus.cc:740] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:19.014828 10302 raft_consensus.cc:740] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:19.058365  5114 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:19.059078 10278 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:19.059863 10278 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:19.061708 10158 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.062222  5114 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:19.029975 10334 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:19.050625 10257 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.063074 10278 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:19.064738  5114 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:19.065580 10277 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:19.065876 10268 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:19.066061 10277 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:19.067396  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.069319  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.070932 10318 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:19.072149 10339 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:19.073288 10277 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:19.074682 10206 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:19.075378 10297 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:19.077292 10312 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:19.067723 10280 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:19.067328  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:19.025987  9927 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:19.025991  9865 raft_consensus.cc:2804] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:19.068418  5041 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:19.081580  5114 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:19.084014  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:19.084249  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:19.085582  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:19.082894 10280 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:19.087184 10280 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.087559 10280 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:19.088244  5946 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:25:19.092330  5114 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:19.056808 10342 consensus_queue.cc:260] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:19.097092  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.028116 10197 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:25:19.098371  5114 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:19.099691  5147 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:19.100500  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:19.104977  5071 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:19.080539 10263 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:19.105600  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:19.106303  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:19.106880  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:19.102988  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:19.108755  5068 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:19.109993  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:19.108956  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:19.095755 10268 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:19.096356  9865 raft_consensus.cc:697] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:19.113188  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:19.096805 10312 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:19.122200  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:19.126547  5073 raft_consensus.cc:2368] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14.
I20260709 12:25:19.126962  5067 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:25:19.114677  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:19.103235 10336 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:19.110340 10278 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:19.138813  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:19.121999 10334 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:19.128613 10312 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.128111 10342 raft_consensus.cc:3060] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:19.168741  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.140111  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:19.187757  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 47776.6us
I20260709 12:25:19.122494  9865 consensus_queue.cc:237] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:19.178632 10298 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:19.131196 10104 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.653s	user 0.012s	sys 0.012s
W20260709 12:25:19.190322 10104 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.716s	user 0.013s	sys 0.012s
I20260709 12:25:19.190044 10298 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.191572  9745 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.191505 10104 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:19.192628 10298 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:19.111593  9978 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:25:19.138249 10278 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.210484  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:19.211371  5224 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:19.122079 10340 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:19.181767 10191 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:19.195490  9419 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.213651 10278 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:19.215512  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:19.214969 10340 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:19.193455 10104 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:19.245759  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:19.246282 10104 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:19.246877 10104 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:19.247298  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:19.248649  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:19.249289  5948 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:19.264405 10189 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:19.266749 10189 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:19.240763  9419 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:19.241374  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:19.242834  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.197451 10312 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:19.255878 10348 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.061s	user 0.004s	sys 0.000s
I20260709 12:25:19.257097  5076 raft_consensus.cc:2410] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:25:19.271270 10158 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.271987 10158 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:19.200076 10336 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:25:19.274582 10158 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:19.132179 10302 consensus_queue.cc:260] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.276450  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:19.276576 10302 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:19.277266  5217 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 17: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:19.277410  5220 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 12:25:19.277653 10340 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:19.277850  5220 pending_rounds.cc:85] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 8
I20260709 12:25:19.278369  5220 pending_rounds.cc:107] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 12:25:19.278617  5220 raft_consensus.cc:2889] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:19.277985  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:19.280229  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:19.280774  5224 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 16: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:19.282822  9988 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:19.285470  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:19.286391  5220 raft_consensus.cc:2368] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18.
I20260709 12:25:19.287261  5224 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:25:18.787597  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:19.297616  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:19.298544  5074 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:19.301625 10350 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.261617 10277 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.302587 10350 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:19.262538  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:19.246675  5939 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 18: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:19.270324 10332 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:19.315969  5189 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:19.323573  5224 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:19.324823  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:19.328032  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:19.324337  5071 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:19.326450 10332 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.336768 10280 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:19.338128 10332 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:19.325721  5898 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:19.341027  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:19.343462  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:19.345000  5939 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:19.346722  5939 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:19.347209  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:19.344863  5898 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:19.352315  5074 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:19.344007  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:19.354988  5073 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:19.357234  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:19.358279  5898 leader_election.cc:400] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14.
I20260709 12:25:19.359102  5898 leader_election.cc:403] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:25:19.325133 10303 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.324018 10277 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:19.359576  5947 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:19.360772  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:19.366389  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:19.368685  9419 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:19.369305  9419 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.369743  9419 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:19.356932  9975 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:19.373294 10191 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:19.357683 10158 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:19.374768 10257 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:19.371022 10197 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:25:19.377497 10197 raft_consensus.cc:3055] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:19.377941 10197 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:19.379025 10280 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:19.378902 10197 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:19.360858 10273 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:19.367944 10320 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.370697 10297 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:25:19.380916 10320 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:19.370536 10303 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.384684 10303 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:19.366894  9927 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14.
I20260709 12:25:19.359853  9988 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:19.384855 10158 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:19.354351 10337 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:19.357270 10261 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:19.387451  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:19.391511  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:19.396418  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:19.367219 10332 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:19.393285 10261 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:19.403299  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:19.404237  5073 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:19.381682 10197 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:19.397244  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:19.410658  9419 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.418682  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:19.419320  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:19.419867  5939 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:19.422101  5224 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:25:19.390408 10337 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:19.429062  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.430081  5189 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:25:19.410105  9975 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:19.413955 10261 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.436483  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:19.394665 10158 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.427876 10337 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:19.437966  5074 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:19.438678  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:19.439036  5943 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:19.441061  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:19.450083 10349 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:19.465431  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:19.459715  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:19.487422  9419 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:19.429451 10350 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:19.492301  5070 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:19.444540 10284 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 1.389s	user 0.001s	sys 0.003s
I20260709 12:25:19.496253  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:19.490877 10349 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:19.436211 10332 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:25:19.487239 10352 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:25:19.505404  5067 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:25:19.503923  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 1033) took 2127 ms. Trace:
I20260709 12:25:19.526286  5223 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:19.508910  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:19.445187 10278 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:19.489862 10353 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:25:19.508452  5940 rpcz_store.cc:276] 0709 12:25:17.376617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.500247 (+123630us) service_pool.cc:224] Handling call
0709 12:25:17.500680 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:17.500984 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:17.501014 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:17.508216 (+  7202us) log.cc:844] Serialized 26 byte log entry
0709 12:25:17.750367 (+242151us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:17.750410 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.750508 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:19.422287 (+1671779us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:19.497688 (+ 75401us) raft_consensus.cc:1685] finished
0709 12:25:19.497720 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.503824 (+  6104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15643520,"thread_start_us":240968,"threads_started":1,"wal-append.queue_time_us":452257}
W20260709 12:25:19.557367  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.507445 10342 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:25:19.572372  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:19.500707 10298 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:19.592315  5219 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:19.593020  5948 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 19.
I20260709 12:25:19.439381  9975 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.501488 10332 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:19.601748 10332 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:19.529872 10274 raft_consensus.cc:3055] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:19.605782 10274 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:19.598979  9975 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:19.606858 10274 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:19.625339  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.626552 10354 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:19.628232  5219 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:19.589982 10325 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:25:19.615286 10274 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:19.630038  5224 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
W20260709 12:25:19.555554 10351 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 12:25:19.630937 10339 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:25:19.635566  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.549325 10002 consensus_meta.cc:287] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.005s
I20260709 12:25:19.651005  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:19.661032  5943 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:19.670274  5899 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:19.664527 10353 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:19.678633  5899 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:19.680897  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:19.590380 10166 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:25:19.658921  9860 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:19.666339  9975 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:19.687180 10104 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:19.688014 10302 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:19.711119  5073 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:19.714884 10261 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.685439  9927 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:19.690745 10257 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:25:19.719307 10332 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:19.719771 10321 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.723768  5899 leader_election.cc:400] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18.
I20260709 12:25:19.685465 10041 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:19.733348  5943 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:19.677430 10325 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:25:19.737450  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:19.780694  5141 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 19.
I20260709 12:25:19.738788 10299 raft_consensus.cc:3060] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:19.746338 10343 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.562s	user 0.004s	sys 0.000s
I20260709 12:25:19.726370  9860 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.749634 10206 raft_consensus.cc:3055] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:19.751240  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:19.751720  5150 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: [13.8-14.9]
I20260709 12:25:19.758903 10278 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:19.760023  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:19.762228  5142 pending_rounds.cc:85] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 9
I20260709 12:25:19.798815  5142 pending_rounds.cc:107] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:25:19.799201  5142 raft_consensus.cc:2889] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:19.800410  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:19.765085  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:19.801143  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:19.802522  5223 raft_consensus.cc:2393] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 12: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:19.766585 10043 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:19.697708 10158 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:19.813572  9860 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:19.807291 10043 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:19.785202 10321 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:19.823585  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:19.823911 10321 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:19.826050 10261 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:19.709072 10312 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:19.828145 10312 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:19.675789 10002 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:19.787026  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:19.790598  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:19.786412  9419 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:19.801146  5141 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:19.830579  5145 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:25:19.805965  5072 raft_consensus.cc:1275] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:25:19.832383  5072 pending_rounds.cc:85] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 7
I20260709 12:25:19.832626  5072 pending_rounds.cc:107] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:25:19.832836  5072 raft_consensus.cc:2889] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:19.825232  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:19.728663  9975 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:19.830339  5149 raft_consensus.cc:2376] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16. Re-sending same reply.
I20260709 12:25:19.831584  5942 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
W20260709 12:25:19.841060  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 54168.3us
I20260709 12:25:19.793835 10206 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:19.841131  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:19.841966  5143 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:19.696298 10353 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:19.844969  9419 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:19.847162  9419 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:19.850008  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:19.845938  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:19.854081  9419 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.773262  5899 leader_election.cc:403] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:19.850704  5149 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:19.817416 10043 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:19.799021 10337 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:19.862735  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:19.866227  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 135191us
W20260709 12:25:19.882332  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.882412  5069 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.887285  5143 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 19.
I20260709 12:25:19.846417 10191 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:19.852614 10342 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.895942  5143 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:19.896363  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:19.898113  9419 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:19.901860  5899 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:19.904749  5072 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:19.884003 10257 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18.
I20260709 12:25:19.909596 10274 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:19.908185 10043 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:19.910624 10274 raft_consensus.cc:697] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:19.910821 10043 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:19.911783 10274 consensus_queue.cc:237] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.916149 10104 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:19.925071 10002 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:19.925387 10043 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:19.927052  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:19.940346  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:19.942996  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:19.943809  5072 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:19.905309 10191 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:19.949091 10043 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:19.964564  9975 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:19.964807  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:19.965878  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:19.908373 10206 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:19.978250  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.978572 10206 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:19.984174 10191 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:19.992094  5141 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:20.001048  5948 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:20.002658 10332 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:19.988420 10340 consensus_meta.cc:287] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.710s	user 0.018s	sys 0.003s
I20260709 12:25:20.008416  5069 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
W20260709 12:25:20.022154 10360 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:25:19.993415 10257 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:25:20.025121  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:20.035322  5939 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:20.036577 10337 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:20.037750 10332 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:20.038230 10332 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.038574 10332 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:20.040283 10344 log.cc:927] Time spent T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.839s	user 0.000s	sys 0.005s
I20260709 12:25:20.041507  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 1044) took 1697 ms. Trace:
I20260709 12:25:20.041960  5941 rpcz_store.cc:276] 0709 12:25:18.344360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.438135 (+ 93775us) service_pool.cc:224] Handling call
0709 12:25:18.438500 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:18.438869 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.438900 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:18.448271 (+  9371us) log.cc:844] Serialized 26 byte log entry
0709 12:25:19.004642 (+556371us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b1800332da0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db0d1 00007fd8a84f6a0f
0709 12:25:19.079567 (+ 74925us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:19.079606 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.079774 (+   168us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:20.039579 (+959805us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:20.041266 (+  1687us) raft_consensus.cc:1685] finished
0709 12:25:20.041293 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.041442 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23841,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":35571,"spinlock_wait_cycles":80590464,"thread_start_us":74724,"threads_started":1,"wal-append.queue_time_us":663973}
I20260709 12:25:20.045233  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:20.030557  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:20.050292  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:20.051388  5222 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:20.007428 10340 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:20.047079  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:20.035449  5189 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:20.059034 10261 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:20.059304 10206 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:20.059629 10261 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:20.060130 10206 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 19 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:20.057284  5217 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:20.061358 10303 consensus_queue.cc:1059] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:20.063295 10303 raft_consensus.cc:3055] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:20.063724 10303 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:20.060922 10261 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:20.065709  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:20.066591  5073 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 15: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:25:20.067646  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.064750 10303 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:20.066195 10374 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:20.068854 10353 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:20.069293  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.061348 10206 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:20.071058 10261 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:20.079635 10349 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:20.080396  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:20.081058 10349 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:20.082093 10303 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.084055 10299 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:20.084764 10299 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:20.074016  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:20.037436 10320 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:20.057260 10340 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:20.090073  5072 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:20.093395 10158 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:20.093978  5945 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:20.096405  5068 raft_consensus.cc:1275] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:20.098026  5071 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:20.098897  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:20.099689  5070 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:20.103183  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:20.103595  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:20.105711  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.106859  5898 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:20.102319  5114 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:20.117477  5070 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:25:20.102319  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:20.124346  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:20.130193  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:20.130806  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:20.131275  5222 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:25:20.129218  5071 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:25:20.124720  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:20.152289  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:20.138041  5899 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:20.069226 10342 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:20.159142  5189 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:20.159560  5942 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:20.158475  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:20.162457  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:20.163348  5073 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 15: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:25:20.167937  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 61137.5us
I20260709 12:25:20.168850  5898 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:20.170625  5071 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:20.120694 10166 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.167459 10041 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:20.190030 10041 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.190490 10041 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:20.190743 10166 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:20.176242  9927 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:20.167784 10320 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:20.120291 10299 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:20.181828 10263 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:20.173699  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:20.182256  5068 raft_consensus.cc:1275] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:20.174404  5114 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:20.172362 10209 consensus_queue.cc:1048] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:25:20.232254  5068 raft_consensus.cc:1240] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:25:20.188566  9865 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:20.132587 10104 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:20.180887 10332 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:20.155220  9860 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:20.174805 10181 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:20.175243  9888 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:20.184957 10171 consensus_queue.cc:1059] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:20.161968 10260 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:20.241869 10263 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:20.276669  9888 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.216538  5188 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:20.294641  9888 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:20.218228  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:20.210263 10320 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:20.295948 10320 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:20.298110 10197 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:20.299216 10332 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:20.236023 10335 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:20.196242  5067 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:20.233453  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 59149us
W20260709 12:25:20.310190  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.285987 10260 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:20.334035 10171 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:20.334103  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 115961us
I20260709 12:25:20.364099  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:20.368276  5067 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:20.338745 10320 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:20.308367  9745 raft_consensus.cc:3055] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:20.240134 10297 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
W20260709 12:25:20.302785 10359 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.614s	user 0.008s	sys 0.001s
I20260709 12:25:20.263109 10362 consensus_queue.cc:1059] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 12:25:20.368008 10086 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:20.376976  9860 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.418375  9860 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:20.419906 10379 log.cc:927] Time spent T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:20.420233 10320 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:20.421617 10158 raft_consensus.cc:3055] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:20.422240 10158 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:20.423380 10158 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:20.424718 10158 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:20.425695  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.426863 10166 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:20.387087 10337 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:25:20.444927 10086 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:25:20.460927  5947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 1022) took 3136 ms (client timeout 2049 ms). Trace:
W20260709 12:25:20.461584  5947 rpcz_store.cc:269] 0709 12:25:17.310007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.315592 (+  5585us) service_pool.cc:224] Handling call
0709 12:25:20.446107 (+3130515us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b5400240c54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:20.446286 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4660607360}
I20260709 12:25:20.385252  9745 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:25:20.477085  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.450223 10181 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:25:20.492919  5190 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.723s (SENT)
I20260709 12:25:20.494407  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:20.468935 10104 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:25:20.498337  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 281880us
I20260709 12:25:20.488174 10337 consensus_queue.cc:1243] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:20.502741  5188 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:20.497812 10268 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.511353  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:20.504442 10362 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:20.507688  9865 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:25:20.474118 10263 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:20.520462 10197 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:20.528282  9745 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:20.543229  9745 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:20.531056 10388 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:25:20.545930  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:20.545223  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:20.514465 10335 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:20.515125 10332 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:20.517148 10268 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:20.557930 10362 raft_consensus.cc:3055] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:20.558450 10362 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:20.559620 10268 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:20.559352 10362 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:20.560938 10362 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:20.564096  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 1045) took 1099 ms. Trace:
I20260709 12:25:20.539330  9865 consensus_queue.cc:1059] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:20.539564 10181 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:20.541196  9927 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.543629  5148 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 12:25:20.531117 10384 log.cc:927] Time spent T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.005s
I20260709 12:25:20.548548  5146 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:20.541577  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1286) took 1752 ms. Trace:
I20260709 12:25:20.508488 10334 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:20.582588  5945 rpcz_store.cc:276] 0709 12:25:18.789656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.844171 (+ 54515us) service_pool.cc:224] Handling call
0709 12:25:18.844540 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.094507 (+1249967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.541193 (+446686us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400009954. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:20.541464 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2866954496}
I20260709 12:25:20.552713  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:20.552747  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:20.554008 10104 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:20.517899 10297 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:25:20.515041 10086 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:20.536667 10321 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:20.564646  5946 rpcz_store.cc:276] 0709 12:25:19.464399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.489752 (+ 25353us) service_pool.cc:224] Handling call
0709 12:25:19.504332 (+ 14580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:19.546168 (+ 41836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.546225 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:20.190976 (+644751us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b2c00152dc8. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:25:20.191392 (+   416us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.498388 (+306996us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:20.498428 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.498497 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:20.563681 (+ 65184us) raft_consensus.cc:1685] finished
0709 12:25:20.563710 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.564021 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24194,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":20225,"spinlock_wait_cycles":3278720,"thread_start_us":306604,"threads_started":1,"wal-append.queue_time_us":370226}
I20260709 12:25:20.591575 10334 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:20.593267 10335 raft_consensus.cc:3055] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:20.604173 10334 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:20.604787 10335 raft_consensus.cc:740] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:20.605585 10260 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:20.606336 10335 consensus_queue.cc:260] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:20.612402  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 92266.3us
I20260709 12:25:20.618433  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:20.621165  5945 raft_consensus.cc:2410] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:25:20.628242  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:20.629051  5141 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:20.629679  5142 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:25:20.631145  5150 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:20.594350  9927 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:20.633265  5148 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:25:20.602914 10321 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:20.636201  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:20.640846  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:20.643704  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:20.647225  5946 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:20.651326  5149 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:20.652531  5150 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:20.624576 10158 consensus_queue.cc:1048] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:25:20.678623 10321 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:20.680078 10321 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:20.631853 10394 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:20.686688 10394 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:20.695279  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.697001  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:20.646876 10332 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:20.720312  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 180453us
I20260709 12:25:20.718212  5150 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:20.724543  5149 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:20.731086  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:20.629822 10297 consensus_queue.cc:1048] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
W20260709 12:25:20.735507  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 166464us
W20260709 12:25:20.736349  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.740048  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:20.752478  5946 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:20.757239  5141 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:20.762264  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:20.763732  5148 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [11.8-15.11]
I20260709 12:25:20.687742 10394 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:20.774127 10041 consensus_meta.cc:287] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.002s
W20260709 12:25:20.784077  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 134614us
I20260709 12:25:20.785604  5899 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:20.787240  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:20.788208  5142 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:20.788640  5145 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:20.792922  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.793819  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.794226  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.794737  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.794656  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:20.795506  5069 raft_consensus.cc:3060] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:20.795779  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:20.796870  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:20.797163  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:20.797675  5220 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:20.799006  5223 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:20.798856  5072 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:20.799696  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:20.800341  5224 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:20.800856  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:20.801304  5899 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:20.790690 10260 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:20.791836 10394 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:20.812944 10261 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:25:20.815193 10320 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:20.815335  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:20.816004 10320 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:20.812671 10260 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.819348 10260 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:20.803920  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:20.822722  5142 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 11.
I20260709 12:25:20.808837  5222 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:25:20.785192  9888 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:20.834416 10400 consensus_queue.cc:1059] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:20.804126 10263 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:20.836138 10263 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:20.836658 10263 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:20.837266  5072 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:20.827339  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:20.809578  5217 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:20.831959 10320 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:20.786144 10041 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:20.843331 10404 log.cc:927] Time spent T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 12:25:20.836333 10399 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:25:20.846909  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.851202  5219 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:20.853489  5223 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:20.855000  5069 raft_consensus.cc:2468] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:20.857746  5215 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:20.860839  5223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:25:20.861527  5223 raft_consensus.cc:606] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:20.861649  5189 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:20.861955  5223 raft_consensus.cc:614] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:20.867027 10400 raft_consensus.cc:2804] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:20.867616 10400 raft_consensus.cc:697] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:20.868544 10400 consensus_queue.cc:237] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:20.882018  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:20.883639  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:20.884135  5220 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:20.885162  5150 raft_consensus.cc:2376] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16. Re-sending same reply.
I20260709 12:25:20.885492  5224 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:25:20.889276  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:20.888880  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:20.895309  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:20.896160  5219 raft_consensus.cc:2410] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:25:20.880470  9927 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:20.902632 10274 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:20.905704 10337 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:20.906448 10337 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:20.907507 10337 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:20.923353 10399 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:20.896917 10320 consensus_queue.cc:1059] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 12:25:20.973415  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 72908.6us
I20260709 12:25:20.979223  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:25:20.987061  5115 connection.cc:256] RPC call timeout handler was delayed by -1.8646050530001048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2049s for a response
I20260709 12:25:20.976135 10273 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:20.991688  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:20.995484  5219 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 19: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:20.977227  9927 consensus_queue.cc:1059] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:25:20.996104  5900 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:21.002488 10257 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:21.005962  5900 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:20.994498 10041 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:20.995573 10334 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.999357 10320 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:21.027679  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:21.028072  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:21.028690  5141 raft_consensus.cc:2393] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 15: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:21.029408 10334 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:21.030059  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:21.029047  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:21.033068  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:20.997813 10273 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.027349 10086 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:21.038520  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:21.040586  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:21.041252  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:21.043340  5144 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:21.037890 10320 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:21.049047  5147 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 16: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:21.012043 10274 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:25:21.044999 10257 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:21.036990 10308 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:21.129882  5072 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:21.134959  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 92794us
W20260709 12:25:21.108767 10414 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.004s
I20260709 12:25:21.140060  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:21.131381 10278 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.013s	sys 0.037s
I20260709 12:25:21.153329  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:21.038864 10390 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 12:25:21.160060  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:21.161052  5144 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:21.140534 10393 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:21.130496 10086 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:25:21.152465 10278 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.014s	sys 0.038s
I20260709 12:25:21.181761 10304 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:25:21.182175 10412 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:25:21.182897  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:21.168919 10308 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:25:21.190136  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:21.103981 10263 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:21.210520  5215 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
W20260709 12:25:21.202001 10420 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:25:21.233075  5190 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:21.194440 10311 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:21.197958 10304 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:21.227474 10321 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:21.199676 10086 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.231304 10416 consensus_queue.cc:1059] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:25:21.228420 10260 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:21.290904  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 95083.7us
I20260709 12:25:21.268198  9865 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:21.266443 10274 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.280707 10273 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:21.297652  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:21.253289 10406 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:25:21.300151  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:21.301182  5941 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:21.252584 10422 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:25:21.302798  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:21.259171 10311 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:21.306283  5939 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:25:21.307215  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:21.308125  5941 raft_consensus.cc:2393] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:21.308668  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:21.281687 10321 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:21.260386 10320 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.298623 10274 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:21.304231 10433 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.318097 10433 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:21.321420 10433 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:21.322096 10418 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:25:21.323447 10311 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:21.325361  5070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1688) took 1197 ms. Trace:
I20260709 12:25:21.330766 10263 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:21.313438 10419 log.cc:927] Time spent T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:25:21.350987  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:21.306589 10308 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:21.334081  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:21.338096  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.352447 10308 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:21.353359 10312 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:21.353991 10312 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:21.354817 10308 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:21.355531 10308 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:21.357121 10308 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:21.359860 10308 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:21.361894 10308 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:21.362459 10308 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:21.348884  5070 rpcz_store.cc:276] 0709 12:25:20.127672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.153959 (+ 26287us) service_pool.cc:224] Handling call
0709 12:25:20.154405 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.154704 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:20.246841 (+ 92137us) log.cc:844] Serialized 14 byte log entry
0709 12:25:20.494645 (+247804us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.324823 (+830178us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b18003e98c0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a53c8c0a 00007fd8aeb08219 00007fd8a5450a40 00007fd8a544b4e4 00007fd8a544b455 00007fd8a8533060 00007fd8a8545716 00007fd8a8556287 00007fd8a8556208
0709 12:25:21.324985 (+   162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:21.325036 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:21.325080 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.325150 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.325279 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43854,"prepare.queue_time_us":35545,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":841847,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":7071,"spinlock_wait_cycles":4875008,"thread_start_us":304013,"threads_started":2,"wal-append.queue_time_us":264770}
I20260709 12:25:21.357167  5939 raft_consensus.cc:1275] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:21.356313  5943 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:21.351850  5940 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:21.352923  5219 raft_consensus.cc:2410] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:21.343307  5942 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:21.356359 10298 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.346095  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:21.340586  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.374145  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:21.374591  5941 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 18: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:21.314313 10429 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:21.379192 10260 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:21.315166 10424 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:21.380364  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:21.383312  5940 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:25:21.377029  9978 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:21.385210 10426 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.344956 10268 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:21.295171 10304 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.356272 10312 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:21.407672  5074 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:21.417263  5074 raft_consensus.cc:606] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:21.417934  5074 raft_consensus.cc:614] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:21.404372  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:21.376691 10191 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:21.419114  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.421190  5942 raft_consensus.cc:2410] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:21.399951  9865 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:21.429658  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.443804  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.445911  5945 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:21.385456  9978 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.379745 10298 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:21.449110  5144 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
W20260709 12:25:21.450654  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.466751 10353 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:21.424880 10304 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:21.390523 10424 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:21.472880  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.473627  5941 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:21.398087 10268 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:21.469135 10353 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.479419  9865 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:21.481364 10353 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:21.398291 10426 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:21.493342 10298 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:21.493847  9978 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:21.495332 10438 log.cc:927] Time spent T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:21.470032 10273 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:21.427666 10191 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.388024 10429 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:21.475965 10349 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:21.511456  5224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:25:21.513792  5224 raft_consensus.cc:606] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:25:21.483844 10334 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:21.505318 10325 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:21.530069 10332 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:21.537310  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:21.540824  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:21.541584  5219 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:21.544238  5945 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:25:21.544729  5945 pending_rounds.cc:85] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 10
I20260709 12:25:21.544947  5945 pending_rounds.cc:107] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:25:21.545178  5945 raft_consensus.cc:2889] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:21.512820 10274 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:21.545836  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:21.548166  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:21.549988  5068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:21.551098  5068 raft_consensus.cc:606] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:21.551469  5068 raft_consensus.cc:614] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:21.549342  5939 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 10.
I20260709 12:25:21.552754  5942 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 19: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:21.545974  9888 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:21.507948 10273 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.526779 10325 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:21.547170 10308 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:21.542219  9745 raft_consensus.cc:3055] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:21.517494 10353 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:25:21.575311 10353 consensus_queue.cc:1243] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:21.574913 10325 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:21.576534 10325 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:21.517002 10349 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:21.580835 10349 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:21.561625  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:21.586467  5074 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:21.577809 10443 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:21.588630 10273 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:21.590176  5068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:21.591472  5068 raft_consensus.cc:606] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:25:21.592094  5068 raft_consensus.cc:614] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:21.594075  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:21.541908 10298 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:25:21.634380 10426 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:21.634819 10429 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:21.635054 10441 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:21.635643 10191 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:21.640230  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:21.647643  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:21.570789  9745 raft_consensus.cc:740] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:21.637363 10166 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:21.658390 10441 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:21.682974  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:21.685783  5070 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:21.661321  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 366004us
I20260709 12:25:21.634419 10268 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:21.691803  5041 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:21.698518  5042 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:21.699637  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:21.710728  5074 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 10.
I20260709 12:25:21.714427  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:21.725507  5041 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:21.727919  5069 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 12:25:21.779682  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 81287.7us
I20260709 12:25:21.656999 10191 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:21.717898  5219 pending_rounds.cc:85] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 8
I20260709 12:25:21.741638  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:21.774103 10447 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.727785 10448 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:25:21.705878 10353 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.750200  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:21.706295 10349 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:21.777361 10342 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:21.788230 10349 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:21.788429 10342 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:21.751925 10442 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:21.789865 10442 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:21.790799 10421 log.cc:927] Time spent T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.554s	user 0.003s	sys 0.003s
I20260709 12:25:21.792344 10442 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:21.794167 10353 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.752255  9745 consensus_queue.cc:260] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.796168  9745 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:21.752678  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:21.753309  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:21.751937 10424 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:21.752002 10450 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:25:21.687520 10166 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:21.761652  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:21.736267 10268 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:21.824772 10268 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:21.825325 10268 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.766232 10443 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:21.829967 10166 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:21.830211 10443 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:21.831274 10443 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:21.770825  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:21.833771  5074 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:21.835345  5074 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:21.706509 10374 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:21.836177 10382 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 1.411s	user 0.004s	sys 0.012s
I20260709 12:25:21.736649  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:21.732107  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:21.784591  5219 pending_rounds.cc:107] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:25:21.713183 10441 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.799671  5042 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:21.797348  5076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1317) took 2521 ms. Trace:
I20260709 12:25:21.837652  5219 raft_consensus.cc:2889] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:21.839987  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 103492us
I20260709 12:25:21.840731  5190 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:21.814779  5069 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:21.802973  5216 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:21.780059 10304 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.801842 10206 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.026s	sys 0.071s
I20260709 12:25:21.835410 10353 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:21.783258  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 57875.9us
I20260709 12:25:21.837342  5068 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
W20260709 12:25:21.844518  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 56741.3us
W20260709 12:25:21.845093 10206 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.026s	sys 0.072s
I20260709 12:25:21.843636  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:21.899396  5042 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:21.838055  5076 rpcz_store.cc:276] 0709 12:25:19.275749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.294028 (+ 18279us) service_pool.cc:224] Handling call
0709 12:25:19.294462 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:20.090472 (+796010us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:20.090818 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 12:25:20.288051 (+197233us) log.cc:844] Serialized 14 byte log entry
0709 12:25:20.307837 (+ 19786us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:20.321422 (+ 13585us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.340595 (+ 19173us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:20.340649 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.355966 (+1015317us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b18003bee40. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:25:21.780358 (+424392us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b18003bee40. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e522 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:25:21.797011 (+ 16653us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b5400286ed4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:21.797065 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:21.797111 (+    46us) raft_consensus.cc:1685] finished
0709 12:25:21.797123 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.797283 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51784,"prepare.queue_time_us":9463,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":105,"raft.queue_time_us":37111,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":19472,"spinlock_wait_cycles":1853371776,"thread_start_us":188232,"threads_started":1,"wal-append.queue_time_us":276381,"wal-append.run_cpu_time_us":8693,"wal-append.run_wall_time_us":1229031}
I20260709 12:25:21.900291  5074 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:21.852113  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58050 (request call id 1698) took 1061 ms. Trace:
W20260709 12:25:21.907227  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 63706.4us
I20260709 12:25:21.802556  9860 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:21.910983  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.904372  5073 rpcz_store.cc:276] 0709 12:25:20.790876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.791050 (+   174us) service_pool.cc:224] Handling call
0709 12:25:20.802656 (+ 11606us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.468494 (+665838us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:21.558571 (+ 90077us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.776253 (+217682us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:21.776934 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 12:25:21.777113 (+   179us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:21.777159 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.777729 (+   570us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:21.851899 (+ 74170us) raft_consensus.cc:1685] finished
0709 12:25:21.851918 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.852048 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":573138,"prepare.queue_time_us":113854,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":623268,"raft.run_cpu_time_us":460,"raft.run_wall_time_us":20235,"thread_start_us":369256,"threads_started":3,"wal-append.queue_time_us":283933}
I20260709 12:25:21.798797 10191 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:21.806006 10447 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:21.949141  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.990384  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.990927  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.893824 10450 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.806897  9978 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:21.895473  9888 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:21.973925  9865 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.932260 10382 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:25:21.851109 10349 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:21.901778 10394 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.916234  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:21.894251 10374 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.846207 10325 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:21.916071 10432 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:21.848105  9975 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:21.850277 10304 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:21.984597 10434 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.848098 10441 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:22.012928 10191 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:22.029322  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.006794  9978 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.009430  9888 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:22.024158 10374 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:22.015398 10349 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:22.013346  9865 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:22.017343 10442 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:22.023187 10460 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.049161  9978 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:22.015297 10394 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:22.050021  9888 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:22.014735 10450 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.099557 10450 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:22.103642 10450 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:22.103839 10321 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:22.104357 10450 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:22.106482 10394 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:22.108675 10450 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:22.114065  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:22.115338 10374 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:22.117385 10297 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:22.117988 10297 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.118342 10297 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:22.062460  5074 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:22.065949 10460 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:22.064776 10342 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:22.130746  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:22.133512  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 234206us
I20260709 12:25:22.134879  5042 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:22.142431  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:22.042032 10434 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.134346  5223 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:22.144838  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:22.147717  5221 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:22.145587  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.061194 10442 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:22.146938  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:22.150378  5042 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:22.153240  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:22.135334 10395 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.007s	sys 0.058s
I20260709 12:25:22.154775  5220 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:22.159045  5218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 1013) took 2123 ms. Trace:
I20260709 12:25:22.163341  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:22.163723  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:22.172403 10447 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:22.151744 10166 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:22.174772 10261 raft_consensus.cc:3055] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:22.175275 10261 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:22.175413 10304 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:22.176311 10166 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:22.178449  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:22.180270  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:22.181838 10268 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:22.184980 10268 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:22.156420 10395 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.007s	sys 0.059s
I20260709 12:25:22.176313 10261 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:22.188270 10261 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:22.191434  5143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 1042) took 1052 ms. Trace:
I20260709 12:25:22.193642 10023 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:22.194191 10429 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:22.194134 10424 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:22.194790 10023 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.195060 10424 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.195024 10429 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.197444 10424 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:22.200825 10460 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:22.151394 10342 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:22.195765 10442 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:22.193878 10434 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:22.207676  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.156948 10469 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:22.212407  5216 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:22.212838  5143 rpcz_store.cc:276] 0709 12:25:21.139358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.141047 (+  1689us) service_pool.cc:224] Handling call
0709 12:25:21.156257 (+ 15210us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:25:21.192880 (+ 36623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:21.192930 (+    50us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:25:21.227278 (+ 34348us) log.cc:844] Serialized 104 byte log entry
0709 12:25:21.292010 (+ 64732us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:21.300294 (+  8284us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.314176 (+ 13882us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.366142 (+ 51966us) log.cc:844] Serialized 14 byte log entry
0709 12:25:22.187155 (+821013us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b18001a0ec0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a53c8c0a 00007fd8aeb08219 00007fd8ac40f58f 00007fd8a544e571 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd
0709 12:25:22.190871 (+  3716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.191079 (+   208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.191128 (+    49us) raft_consensus.cc:1685] finished
0709 12:25:22.191143 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.191351 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":973970,"prepare.queue_time_us":27474,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":117,"raft.queue_time_us":986388,"raft.run_cpu_time_us":438,"raft.run_wall_time_us":16399,"spinlock_wait_cycles":12589952,"thread_start_us":22413,"threads_started":1,"wal-append.queue_time_us":104491}
I20260709 12:25:22.197351  9865 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:22.228925 10434 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.230156 10434 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:22.232524 10434 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:22.234076 10335 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:22.239326  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 76504us
I20260709 12:25:22.242450  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.243381  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.244786  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:22.245666  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.246518  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.247262  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.162567  5218 rpcz_store.cc:276] 0709 12:25:20.030794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.031704 (+   910us) service_pool.cc:224] Handling call
0709 12:25:20.032151 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:20.055557 (+ 23406us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:20.056085 (+   528us) log.cc:844] Serialized 14 byte log entry
0709 12:25:20.277132 (+221047us) log.cc:844] Serialized 14 byte log entry
0709 12:25:20.838613 (+561481us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b18001c9480. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544b455 00007fd8a8533060 00007fd8a8545716 00007fd8a8556287 00007fd8a8556208
0709 12:25:20.865724 (+ 27111us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:20.866592 (+   868us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.866781 (+   189us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:20.866835 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.866948 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:21.797861 (+930913us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:22.154279 (+356418us) raft_consensus.cc:1685] finished
0709 12:25:22.154298 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.154453 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4304,"prepare.queue_time_us":2808,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":96,"raft.queue_time_us":28471,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":3128,"spinlock_wait_cycles":31558784,"thread_start_us":242727,"threads_started":2,"wal-append.queue_time_us":330220}
I20260709 12:25:22.159682 10374 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:22.255525  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.258764  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.260591  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.261399  5069 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:25:22.262466  5075 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:22.264163  5073 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:22.221400 10023 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:22.173207  9888 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:22.270004  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.271260  5069 raft_consensus.cc:2393] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:22.272956  5221 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
W20260709 12:25:22.265833 10464 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:25:22.257737 10424 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:22.269783 10417 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 1.079s	user 0.004s	sys 0.006s
I20260709 12:25:22.292265  5217 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:22.286804 10350 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.050s	user 0.008s	sys 0.017s
I20260709 12:25:22.275239 10278 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
W20260709 12:25:22.308496  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.301555 10350 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.072s	user 0.008s	sys 0.017s
I20260709 12:25:22.285730 10023 consensus_queue.cc:1059] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:25:22.309748 10278 consensus_queue.cc:1243] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:22.294575 10424 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:22.319880  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.314405  9927 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.321974  9888 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:22.323812 10303 raft_consensus.cc:3055] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:22.346609 10086 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:22.340356 10303 raft_consensus.cc:740] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:22.351154  5073 pending_rounds.cc:85] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting all ops after (but not including) 9
I20260709 12:25:22.355576  5073 pending_rounds.cc:107] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:25:22.355947  5073 raft_consensus.cc:2889] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:22.357975 10429 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:22.340061  9927 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:22.371985  5940 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:22.372664  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:22.374306  5947 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:22.381181  9927 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:22.381942 10395 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.383100 10424 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:22.382115  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:22.388900  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:22.349298  9978 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.390441  5947 raft_consensus.cc:2393] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 18: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:22.391170  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:22.396261  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:22.397395  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.398473  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:22.396817  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:22.399174  5939 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
W20260709 12:25:22.403666  5215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 1334) took 1532 ms (client timeout 2038 ms). Trace:
W20260709 12:25:22.404099  5215 rpcz_store.cc:269] 0709 12:25:20.870769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.870898 (+   129us) service_pool.cc:224] Handling call
0709 12:25:22.403497 (+1532599us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54002d8f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:22.403609 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544829312}
I20260709 12:25:22.407162 10297 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:22.410990  9978 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:22.411959  5071 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 17: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:22.395087 10303 consensus_queue.cc:260] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.413682 10303 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:22.383113 10395 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:22.417119  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 1056) took 1039 ms. Trace:
I20260709 12:25:22.417709  5941 rpcz_store.cc:276] 0709 12:25:21.377738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.377957 (+   219us) service_pool.cc:224] Handling call
0709 12:25:21.378366 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:21.474319 (+ 95953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.416777 (+942458us) spinlock_profiling.cc:243] Waited 95 ms on lock 0x7b540000ad54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:22.417030 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218490624}
I20260709 12:25:22.423576  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:22.401301 10400 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:22.431944  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:22.433148  5222 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:22.434325 10470 log.cc:927] Time spent T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:25:22.429979 10353 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:22.436942  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:22.435635 10353 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.440259  5941 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:22.432660 10297 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:22.441728 10425 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 1.194s	user 0.006s	sys 0.008s
I20260709 12:25:22.441965 10353 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:22.450176  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:22.430087 10416 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:22.402899 10206 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:25:22.432910 10400 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.453572 10297 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.461872 10400 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:22.462056 10297 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:22.465204 10297 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:22.457029 10416 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.466030 10416 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:22.467531  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:22.468448 10469 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:22.471248  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.471578  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.472025  5072 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:22.472409  5074 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:22.475448  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.478792  5070 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:22.483150 10473 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.141s	user 0.002s	sys 0.000s
I20260709 12:25:22.485200 10334 raft_consensus.cc:3055] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:22.485754 10334 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:22.486653 10334 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:22.487043 10400 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:22.487759 10334 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.491660  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1314) took 1638 ms. Trace:
I20260709 12:25:22.492098  5946 rpcz_store.cc:276] 0709 12:25:20.852897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.853022 (+   125us) service_pool.cc:224] Handling call
0709 12:25:20.879953 (+ 26931us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:20.897361 (+ 17408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:20.897388 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:20.939350 (+ 41962us) log.cc:844] Serialized 26 byte log entry
0709 12:25:21.211142 (+271792us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:21.211199 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.211292 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.441028 (+1229736us) spinlock_profiling.cc:243] Waited 57.5 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:22.491365 (+ 50337us) raft_consensus.cc:1685] finished
0709 12:25:22.491388 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.491595 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10272,"raft.queue_time_us":10469,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":12434,"spinlock_wait_cycles":132342144,"thread_start_us":208099,"threads_started":1,"wal-append.queue_time_us":265522}
I20260709 12:25:22.493395  5220 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:22.506356  5219 raft_consensus.cc:1240] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:25:22.507966  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.508728  5074 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:22.508808  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:22.511250  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:22.511528  9978 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:22.512272  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:22.511147 10298 consensus_queue.cc:1059] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:25:22.515877 10400 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:22.516637 10298 raft_consensus.cc:3055] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:22.517094 10298 raft_consensus.cc:740] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:22.518115  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.518013 10298 consensus_queue.cc:260] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:22.520249 10298 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:22.521847  5900 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.712s (SENT)
I20260709 12:25:22.522555  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:22.526433 10477 log.cc:927] Time spent T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.161s	user 0.002s	sys 0.000s
I20260709 12:25:22.529968  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.531031  5068 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:22.517921  5941 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
W20260709 12:25:22.519269  4894 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43234 (request call id 127) took 26381 ms (client timeout 15000 ms). Trace:
W20260709 12:25:22.538298  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.535318  4894 rpcz_store.cc:269] 0709 12:24:56.150939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.178609 (+ 27670us) service_pool.cc:224] Handling call
0709 12:25:22.519002 (+26340393us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:11.567981 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:11.606090 (+ 38109us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:11.606149 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:25:11.606174 (+    25us) tablet.cc:662] Decoding operations
0709 12:25:11.611356 (+  5182us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:11.645150 (+ 33794us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:11.646335 (+  1185us) tablet.cc:685] Acquiring locks for 28 operations
0709 12:25:11.945676 (+299341us) tablet.cc:701] Row locks acquired
0709 12:25:11.945732 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:25:11.945941 (+   209us) write_op.cc:270] Start()
0709 12:25:11.946107 (+   166us) write_op.cc:276] Timestamp: P: 1783599911945849 usec, L: 0
0709 12:25:11.946144 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:25:12.018001 (+ 71857us) log.cc:844] Serialized 10925 byte log entry
0709 12:25:12.682284 (+664283us) op_driver.cc:464] REPLICATION: finished
0709 12:25:13.011679 (+329395us) write_op.cc:301] APPLY: starting
0709 12:25:13.057843 (+ 46164us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:14.338308 (+1280465us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:14.362108 (+ 23800us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:16.709875 (+2347767us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:16.710608 (+   733us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=56,mrs_lookups=0
0709 12:25:16.710718 (+   110us) write_op.cc:312] APPLY: finished
0709 12:25:16.754145 (+ 43427us) log.cc:844] Serialized 243 byte log entry
0709 12:25:17.140959 (+386814us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a54310ac 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b
0709 12:25:17.647047 (+506088us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:17.839724 (+192677us) write_op.cc:454] Released schema lock
0709 12:25:17.870097 (+ 30373us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:18.066934 (+196837us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:18.069849 (+  2915us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2373364,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":280693,"apply.run_cpu_time_us":110148,"apply.run_wall_time_us":4983698,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121111,"num_ops":28,"prepare.queue_time_us":229172,"prepare.run_cpu_time_us":53200,"prepare.run_wall_time_us":772503,"raft.queue_time_us":283422,"raft.run_cpu_time_us":10764,"raft.run_wall_time_us":126163,"replication_time_us":735952,"spinlock_wait_cycles":52934912,"thread_start_us":1501425,"threads_started":5,"wal-append.queue_time_us":1229113,"wal-append.run_cpu_time_us":27091,"wal-append.run_wall_time_us":2602397}]]}
I20260709 12:25:22.541142  5899 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:22.546277 10260 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:22.546736 10260 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.547186 10260 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:22.550446 10261 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:22.553789  5070 raft_consensus.cc:1275] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:22.560271 10395 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:22.565747  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:22.574270 10261 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:22.574872 10261 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:22.575356 10261 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.568861  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:22.576303  5068 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
W20260709 12:25:22.581704  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 103641us
I20260709 12:25:22.584600  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:22.569648 10434 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.586894  9927 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:22.599375  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:22.608181  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.622072  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.624320  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:22.625277  5220 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:22.628077  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:22.610309  5222 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:22.631667  5041 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:22.617584  5067 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:22.619367  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:22.593055 10261 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:22.647020  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.639920  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:22.620180 10335 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:22.666862 10478 log.cc:927] Time spent T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:25:22.622102 10171 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:22.638404 10428 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:22.641249 10298 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:22.638955 10400 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:22.678612  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:22.722683 10400 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:22.723958 10407 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:22.724584 10407 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.725037 10407 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:22.622159 10086 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:25:22.679571 10171 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:25:22.723094  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 45045.9us
I20260709 12:25:22.635078 10315 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:25:22.619674 10304 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:25:22.707177 10416 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:22.729997  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 303019us
I20260709 12:25:22.678328 10297 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:22.671464 10335 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.636339 10434 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.743021  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:22.683565 10428 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.750921  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:22.751919  5143 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 19: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:22.731451 10400 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:22.754825  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:22.757798  5147 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:22.739686 10171 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:22.770460 10086 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:22.771469 10086 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:22.771970 10086 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:22.758234  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:22.774314  5146 raft_consensus.cc:2410] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 12:25:22.776722  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:22.776923  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:22.777465  5142 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:22.778506  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:22.779251  5146 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:22.736682 10298 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:22.780441 10298 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.781411  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:22.782130  5143 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:22.747361 10353 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.783700 10353 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:22.784068 10086 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:22.786015 10353 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:22.787741 10416 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:22.789944 10335 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:22.750396 10428 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:22.792688  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:22.793452  5141 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:25:22.798659 10260 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:22.800383  5142 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:22.801064 10260 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:22.807807  5147 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:25:22.809829  5147 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:22.811506  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:22.812772  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:22.813462  5146 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:22.744618 10206 consensus_queue.cc:1048] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:25:22.759430  5148 raft_consensus.cc:1240] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: [13.8-14.9]
I20260709 12:25:22.761752 10434 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:22.745198 10297 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.818632  5115 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:22.822345 10297 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:22.822234  5141 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:22.823812  5149 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:22.820946  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:22.815671 10171 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:22.833196  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:22.829175 10424 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:22.841114 10429 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:25:22.852492  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 244322us
W20260709 12:25:22.854666  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.855362  5115 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:22.874892  5190 leader_election.cc:336] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.798s (SENT)
I20260709 12:25:22.875694  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:22.838781 10209 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:22.860158  5115 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:22.836298 10334 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:25:22.857311 10023 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:22.859615 10315 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:22.895220 10206 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.893311 10304 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.894088 10400 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:22.947463  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 328155us
I20260709 12:25:22.939816 10315 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:22.918900 10209 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:22.967010 10315 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:22.967840 10315 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:22.969292  5898 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:22.981844 10261 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:22.983412 10261 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:22.983826 10261 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.984143 10261 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:22.973098  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:22.990478  5898 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:22.926331 10171 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:22.994503  5898 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:22.966764  4893 catalog_manager.cc:5697] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 14 to 16, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 16 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:22.995431 10260 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:23.006946  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:22.957551 10206 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:23.008962  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.009562  5945 raft_consensus.cc:2393] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 16: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:22.990069 10209 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:22.961475 10304 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:22.999958 10260 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.000911 10171 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:23.023034  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.026157  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.026495  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.027115  5945 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:23.024879 10407 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:23.031895  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.032612  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.025234 10209 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:23.036712 10481 log.cc:927] Time spent T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.185s	user 0.005s	sys 0.000s
I20260709 12:25:23.036971 10407 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:23.021256 10315 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:23.023336 10260 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:23.039685  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.042140  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.043352  5939 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:23.047003  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:23.048460  5945 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 11: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:23.050616  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.051518  5946 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
W20260709 12:25:23.054760  5042 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.528s (SENT) [suppressed 21 similar messages]
I20260709 12:25:23.022512 10171 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:23.064134  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:23.069308  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 331255us
I20260709 12:25:23.070484  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:23.074308  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:23.056789 10304 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:23.077750  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:23.077939  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.079928  5068 raft_consensus.cc:2410] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:25:23.068593 10171 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:23.087558  9978 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:23.086709 10407 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:23.099948  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:23.180888  5068 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:23.105020  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:23.183027  5067 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:23.126819  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:23.184729  5071 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:23.172623  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 312547us
I20260709 12:25:23.188094  5115 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:23.096359  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:23.208710  5115 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:23.191494  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:23.224509  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 128281us
I20260709 12:25:23.223237  5072 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:23.134773 10304 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:23.189476  5899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.017s	sys 0.002s
I20260709 12:25:23.068185  4893 catalog_manager.cc:5697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:23.225484  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:23.236486  5899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.017s	sys 0.003s
W20260709 12:25:23.266588  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 657447us
I20260709 12:25:23.265731  5041 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:23.276856  5071 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:23.261240 10424 consensus_queue.cc:1048] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:25:23.228632 10304 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:23.267572  5899 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:23.279568  5041 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:23.279973  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 103905us
I20260709 12:25:23.261430 10429 consensus_queue.cc:1059] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:25:23.114832 10315 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:23.225302 10023 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:23.166319 10206 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:23.101195 10268 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:23.304345  5188 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:23.269670  9978 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:23.147086 10407 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:23.315840  5188 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:23.224997 10334 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:23.328255 10334 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:23.329188 10334 consensus_queue.cc:1243] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:23.322084 10261 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:23.331358 10261 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:23.323282  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.331588  9865 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.332195  5222 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:23.332147  9865 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:23.322649 10407 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:23.335014 10335 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:23.288344 10442 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:23.335510 10335 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.335598 10442 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:23.335990 10335 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:23.336081 10442 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:23.339471  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:23.326438 10260 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:23.327316 10297 consensus_meta.cc:287] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.004s
I20260709 12:25:23.286123 10353 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:23.340674  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:25:23.339996  5150 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.000s	sys 0.097s
I20260709 12:25:23.157591 10428 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:23.342906 10303 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:23.348407  5150 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.000s	sys 0.097s
I20260709 12:25:23.345841 10434 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:23.320751 10298 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:23.317306 10315 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:23.334724  9865 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:23.367439 10206 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:23.320006 10268 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:23.366902 10428 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:23.375396  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:23.341444 10489 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:23.379156  5067 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
W20260709 12:25:23.377792  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 323050us
I20260709 12:25:23.354635 10303 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.371379 10434 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.352555 10297 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:23.370460 10298 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:23.378566 10268 raft_consensus.cc:3060] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:23.246888  4893 catalog_manager.cc:5697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:23.400101 10428 raft_consensus.cc:3055] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:23.404769 10434 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:23.398869 10303 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:23.399083 10489 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:23.404486 10206 raft_consensus.cc:697] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:23.432449  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 269206us
I20260709 12:25:23.414706 10298 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:23.424949 10428 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:23.435940  5188 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:25:23.447404  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.448179  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.462802  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.464100  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 172359us
W20260709 12:25:23.465644  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 241390us
W20260709 12:25:23.472735  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 192956us
I20260709 12:25:23.451121  9865 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:23.479187  5942 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:23.481587  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.485311  5219 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:23.485832  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.486914  5944 raft_consensus.cc:1240] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:23.482533  5945 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:23.494580  5942 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:25:23.496335  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.499123  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:23.502878  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.503600  5222 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 12:25:23.505750  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.509637  5947 raft_consensus.cc:1240] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:23.511341 10297 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:23.512604  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:23.513319 10495 log.cc:927] Time spent T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:25:23.513763 10260 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:23.508647  4893 catalog_manager.cc:5697] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 3 to 10. New cstate: current_term: 10 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:23.471994 10488 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:25:23.518035  4893 catalog_manager.cc:5697] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 5 to 11, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 11 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:23.522838  4893 catalog_manager.cc:5697] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 12 to 13, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 13 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:23.523438 10335 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:23.504736 10206 consensus_queue.cc:237] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:23.476148  9865 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.528127  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.528940  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:23.530061  5115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.530562  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:23.555922  5067 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.557670  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.558662  5215 raft_consensus.cc:2393] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 17: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:25:23.560534  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 61736.1us
I20260709 12:25:23.564965  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:23.567086  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:23.574533  5115 connection.cc:256] RPC call timeout handler was delayed by -1.1972592229999464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
I20260709 12:25:23.575476  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:23.579936 10023 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.580463 10023 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:23.582876 10023 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:23.583621 10424 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:23.584357 10209 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:25:23.587888  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 110745us
W20260709 12:25:23.619169  5041 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.590462  5067 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:23.592319 10315 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:23.595458  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:23.555362  9865 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:23.591427  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.620947  5944 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 17: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:23.602896  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.483876 10428 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:23.557951 10304 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:23.623308  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:23.625172  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:23.632912  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.633656  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.633073  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:23.634246  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:23.635425  5944 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 16: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:23.641340  5945 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:23.643926  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:23.640502  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:23.642959  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:23.651845  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:23.655738  5190 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:23.656138  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.628618 10407 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:23.660593  5215 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:25:23.502051 10489 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:23.665089  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.672441  5224 raft_consensus.cc:2393] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 17: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:23.665936  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.666218  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:23.673995  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 96567.6us
I20260709 12:25:23.677184  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.638177 10304 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.682334  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:23.646795 10488 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.642146 10315 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.679351  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:23.683789  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.687973  5143 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:23.685068  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.688380  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:23.686056  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.624377 10424 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:23.656648 10491 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:23.686847  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:23.639134 10503 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:25:23.661950 10490 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:23.696548  5150 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 16: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:23.696614  5218 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:23.699196  5141 raft_consensus.cc:2376] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16. Re-sending same reply.
I20260709 12:25:23.700381  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.701758  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.702514  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.707365 10407 raft_consensus.cc:2804] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:23.694790 10488 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:23.689283 10304 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:23.736963  5144 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:23.688961  9888 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:23.741756  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.744838  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:23.751645  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 96034.2us
I20260709 12:25:23.737428  5149 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
W20260709 12:25:23.730284 10501 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 12:25:23.729650 10497 log.cc:927] Time spent T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.283s	user 0.000s	sys 0.002s
I20260709 12:25:23.762933  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.766973  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:23.771633  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 145708us
I20260709 12:25:23.733645 10518 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.776618  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.734776 10490 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:23.782927  5144 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 19: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
W20260709 12:25:23.782984  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 131257us
I20260709 12:25:23.734937 10491 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:23.784744  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.829500  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:23.829792  5142 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
I20260709 12:25:23.798926 10260 raft_consensus.cc:3055] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:23.800019  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.735261 10513 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:23.698254 10315 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:23.834415  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.755432 10407 raft_consensus.cc:697] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:23.844029  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.725412 10268 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:23.844583  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 56073.9us
I20260709 12:25:23.859709  5142 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:23.844645  5149 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 17: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:23.777804  9888 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:23.864066  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.846052  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.852946  5141 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11.
W20260709 12:25:23.855043  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 57658.4us
I20260709 12:25:23.869529  5143 raft_consensus.cc:2410] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:25:23.873932  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:23.874946  5141 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.898128  5141 raft_consensus.cc:2410] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:25:23.825863 10490 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:23.910995  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.920595  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:23.913147  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 49535.2us
W20260709 12:25:23.858513 10171 consensus_meta.cc:287] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.719s	user 0.012s	sys 0.003s
W20260709 12:25:23.927254  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 122966us
I20260709 12:25:23.843827 10260 raft_consensus.cc:740] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:23.974125  5143 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:25:23.972004  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:23.974793  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:23.975533  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:23.975909  5144 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:23.779304 10416 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.134s
I20260709 12:25:23.987059  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:23.997092  5149 raft_consensus.cc:1240] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:24.003878  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.009367  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.010092  5143 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:25:24.021620  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:23.987421 10209 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.040073  5149 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:24.028112  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 158458us
I20260709 12:25:24.028982  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:23.926131 10514 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 12:25:23.826890 10518 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:23.926054 10303 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:23.925374 10524 consensus_queue.cc:1059] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.163s
I20260709 12:25:24.058457 10304 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:23.986732 10171 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:24.063402 10171 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:24.064157  5940 raft_consensus.cc:1240] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:24.066485 10303 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:24.067004 10442 consensus_meta.cc:287] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.730s	user 0.014s	sys 0.000s
I20260709 12:25:24.067317 10303 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:24.067761 10527 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:24.067500 10442 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:24.068753  5144 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:24.068522 10527 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:24.069732  5221 raft_consensus.cc:2368] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15.
I20260709 12:25:24.047910 10400 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:24.072068 10400 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.074576 10400 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:24.076206 10434 consensus_meta.cc:287] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.002s
I20260709 12:25:24.076821 10434 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.077458 10315 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:24.078382  5224 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:24.079242 10434 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:24.079736 10535 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.080189 10442 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:24.080322 10535 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.080112  9888 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.082945 10535 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:24.083535 10524 raft_consensus.cc:3055] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:24.084012 10524 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:24.087795  5143 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:24.098511 10424 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:24.099713 10522 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
W20260709 12:25:24.101013 10517 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:25:24.037256 10407 consensus_queue.cc:237] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:24.079739  4893 catalog_manager.cc:5697] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 10 to 12, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 12 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.102967 10527 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:24.104328  4893 catalog_manager.cc:5697] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:24.105703 10509 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.404s	user 0.001s	sys 0.000s
I20260709 12:25:24.107190  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.107883  5141 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:24.108901  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.110189  5142 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:24.111932  5146 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:24.109120  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.114853  5145 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 18. Current term is 19. Ops: []
W20260709 12:25:24.116381  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 144490us
W20260709 12:25:24.122834  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.084942 10524 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.156395  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.157284  5148 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:24.161267  5946 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:24.161473  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.047797 10209 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:24.164263  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.167932  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:24.169909  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:24.174103  5942 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 16: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:24.025249 10491 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:24.177997  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.160260 10297 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:24.177273 10524 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:24.106596  4893 catalog_manager.cc:5697] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 12 to 14, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 14 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:24.176011 10525 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:24.181788 10298 consensus_meta.cc:287] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.730s	user 0.010s	sys 0.005s
I20260709 12:25:24.186944 10298 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:24.186018 10525 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:24.187907 10514 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:24.191641  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.192587  5145 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:24.192637  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.199857  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.155475 10527 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 12:25:24.205698  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 428598us
I20260709 12:25:24.207965  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.189170 10298 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:24.209837  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.206841  5041 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:24.213240  5148 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:24.214244  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.211107 10536 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:24.221375 10536 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:24.221909  9865 consensus_meta.cc:287] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.592s	user 0.008s	sys 0.004s
I20260709 12:25:24.229449  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:24.231767  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:24.232507  5942 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:24.238265  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:24.243839 10209 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:24.211870 10268 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:24.244828 10529 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:24.260337 10529 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.222515  9865 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:24.260646  4893 catalog_manager.cc:5697] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 8 to 13, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 13 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:24.262681  4893 catalog_manager.cc:5697] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:24.263636  5219 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:24.263069  9865 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:24.245844  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.247874  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.268981  5143 raft_consensus.cc:2376] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19. Re-sending same reply.
I20260709 12:25:24.248507  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.245128  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:24.247159  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:24.270671  5948 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:24.255095  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.250787  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.254799  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.274977  5141 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:24.275266  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:24.275012  5145 raft_consensus.cc:2376] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13. Re-sending same reply.
I20260709 12:25:24.307032  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.307996  5145 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:25:24.305152  5947 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:24.310025  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.310894  5143 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:24.311525  5041 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:24.313918  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.314307  4893 catalog_manager.cc:5697] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 14, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 14 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:24.315302  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.315869  5141 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:24.316788  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.310144 10268 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:24.316118  4893 catalog_manager.cc:5697] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 8 to 11, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 11 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:24.254405 10488 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:24.245019 10298 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.222426 10536 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.338212  5217 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:24.326495  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.317528  5141 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 19: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:24.321903  5947 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:24.343585  5948 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:24.354058  5143 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
W20260709 12:25:24.349344  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 164803us
I20260709 12:25:24.355103  5041 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:24.355480  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:24.356242  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:24.356498  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:24.356727  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:24.357048  5067 raft_consensus.cc:3060] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:24.357496  5071 raft_consensus.cc:3060] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:24.357158  5072 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
W20260709 12:25:24.363386  5223 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.024s	sys 0.008s
W20260709 12:25:24.363966  5223 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.025s	sys 0.008s
I20260709 12:25:24.347477  5947 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 16. Current term is 17. Ops: []
W20260709 12:25:24.365268  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 180615us
I20260709 12:25:24.368528  5067 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:24.374173  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:24.374588  5941 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:24.374737  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.376225  5145 raft_consensus.cc:2468] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:24.383167  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:24.377485  5218 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:24.318672 10442 raft_consensus.cc:2804] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:24.318219 10268 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.376055  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:24.382134  5071 raft_consensus.cc:2468] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:24.391417  5900 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:24.329860 10303 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:24.260727 10529 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:24.259308 10428 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:24.254408 10525 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:24.429920  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:24.434911  5070 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:24.430807  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:24.436584  5941 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
W20260709 12:25:24.437407  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 45092.3us
I20260709 12:25:24.384157 10548 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.320545 10298 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.375648 10545 raft_consensus.cc:2804] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:24.425637 10268 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:24.464879  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:24.474561  5188 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:24.475956  5188 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:24.480727  5900 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:24.485459  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:24.498526  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:24.499137  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 112102us
I20260709 12:25:24.443645 10429 consensus_queue.cc:1059] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:25:24.448379 10503 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:25:24.503473  5188 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:24.483994  9888 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:24.455992 10525 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:25:24.505703  5900 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:24.497963 10488 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:24.498775 10171 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:24.497699 10303 raft_consensus.cc:2804] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:24.519901  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:24.479434 10304 consensus_queue.cc:1059] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:25:24.500026 10086 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:24.500087 10524 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:24.319389  4893 catalog_manager.cc:5697] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 12, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 12 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.517011 10429 consensus_queue.cc:1243] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:25:24.543576 10544 log.cc:927] Time spent T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:25:24.543716 10555 log.cc:927] Time spent T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:25:24.544488 10537 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:24.514787 10553 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:24.550019  9865 raft_consensus.cc:2804] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:24.452114 10548 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:24.550784 10554 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:24.608865  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:24.609965  5223 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:24.554822 10298 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:24.482877 10487 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 12:25:24.580706  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 60866.9us
I20260709 12:25:24.516263 10538 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:24.561928 10303 raft_consensus.cc:697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:24.509377 10442 raft_consensus.cc:697] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:24.517362  9888 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:24.585246  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 146153us
I20260709 12:25:24.523275 10488 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:24.600958  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:24.605993  5068 raft_consensus.cc:1240] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:24.578433 10395 raft_consensus.cc:3055] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:25:24.588153 10490 consensus_meta.cc:287] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.002s
I20260709 12:25:24.625433  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:24.619822  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:24.635749  5076 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:24.645977  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:24.609102 10545 raft_consensus.cc:697] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:24.622128  9865 raft_consensus.cc:697] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:24.661150  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:24.662166  5067 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 11: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:24.663126 10429 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:24.663249  9865 consensus_queue.cc:237] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.663965 10548 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:24.668798  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:24.650135 10395 raft_consensus.cc:740] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:24.689477  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57002 (request call id 1051) took 1022 ms. Trace:
I20260709 12:25:24.651057 10488 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:24.648406  9888 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:24.692482  5224 rpcz_store.cc:276] 0709 12:25:23.666740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.679059 (+ 12319us) service_pool.cc:224] Handling call
0709 12:25:24.689146 (+1010087us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54000a2dd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:24.689381 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897561472}
I20260709 12:25:24.647794 10487 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:24.602543 10529 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:24.609150 10553 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:24.708432  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:24.708724  5067 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 12:25:24.717163  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 131423us
I20260709 12:25:24.719177  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:24.720170  5067 raft_consensus.cc:2393] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:24.652953 10519 consensus_queue.cc:1059] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:25:24.722783  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:24.694173 10303 consensus_queue.cc:237] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:24.725193  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:24.727672  5067 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:24.722460  5073 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:24.730240  5070 raft_consensus.cc:2468] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:24.657886 10490 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:24.731181  5069 raft_consensus.cc:1240] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16. Ops: []
W20260709 12:25:24.734306  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 70870.2us
I20260709 12:25:24.733860  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:24.681964 10315 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:25:24.741623  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:24.733621  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 171713us
I20260709 12:25:24.740064  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:24.749441  5222 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:25:24.750869 10578 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:24.754076  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:24.755700  5074 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:24.756002 10545 consensus_queue.cc:237] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:24.758903 10304 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:24.759866 10490 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:24.761577 10553 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:24.759943 10304 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.730046 10442 consensus_queue.cc:237] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:24.757676 10578 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.766171 10268 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.767948  5075 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:24.744163 10395 consensus_queue.cc:260] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:24.736891 10487 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:24.796788  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:24.799327  5941 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
W20260709 12:25:24.781288  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 45720.7us
I20260709 12:25:24.802217  5041 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:24.806262  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:24.805487  5067 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:24.807188  5948 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:24.807653  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:24.752611 10537 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:24.778034 10529 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:24.782027 10268 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:24.819288  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:24.831768  5898 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:24.823271  5042 connection.cc:256] RPC call timeout handler was delayed by -1.2541979399998127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 12:25:24.800765 10487 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.787504 10524 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:24.790676 10518 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:24.837153  5898 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:24.747398 10315 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.805781 10395 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:24.807118 10553 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:24.832883  5042 connection.cc:256] RPC call timeout handler was delayed by -1.1725685880001038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 12:25:24.842574  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:24.850692  5074 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:24.842270 10304 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:24.834615 10487 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:24.860292  5944 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:24.861426  5948 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:24.842710 10524 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:24.869079  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:24.857493 10586 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:25:24.859054 10315 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:24.853032 10581 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:24.856247 10537 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:24.877257 10515 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 1.169s	user 0.001s	sys 0.009s
I20260709 12:25:24.857398 10304 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:24.879746  5939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1333) took 1408 ms. Trace:
I20260709 12:25:24.880309  5939 rpcz_store.cc:276] 0709 12:25:23.471182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.472258 (+  1076us) service_pool.cc:224] Handling call
0709 12:25:23.496604 (+ 24346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:23.527283 (+ 30679us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.527316 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:23.531658 (+  4342us) log.cc:844] Serialized 26 byte log entry
0709 12:25:23.654996 (+123338us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:23.655036 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.655100 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:24.876445 (+1221345us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:24.879342 (+  2897us) raft_consensus.cc:1685] finished
0709 12:25:24.879373 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.879672 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30159,"raft.queue_time_us":37483,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":25135,"spinlock_wait_cycles":68795648,"thread_start_us":123024,"threads_started":1,"wal-append.queue_time_us":175649}
I20260709 12:25:24.878360  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:24.886236 10171 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:24.890101 10496 raft_consensus.cc:3055] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:24.890698 10496 raft_consensus.cc:740] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:24.891835 10496 consensus_queue.cc:260] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:24.893647 10429 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.894542  5222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 1042) took 1235 ms. Trace:
I20260709 12:25:24.894384 10429 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:24.895215  5222 rpcz_store.cc:276] 0709 12:25:23.658746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.659281 (+   535us) service_pool.cc:224] Handling call
0709 12:25:23.684257 (+ 24976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.750051 (+1065794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.894177 (+144126us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003c8f54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:24.894458 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2448959744}
W20260709 12:25:24.903781  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 88364.5us
I20260709 12:25:24.906714  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:24.848551 10553 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:25:24.931972  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 82275.3us
W20260709 12:25:24.933825  5042 connection.cc:256] RPC call timeout handler was delayed by -1.1407979399996293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
W20260709 12:25:24.934485  5042 connection.cc:256] RPC call timeout handler was delayed by -1.1230685880000237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
W20260709 12:25:24.934803  5042 connection.cc:256] RPC call timeout handler was delayed by -1.1209979399995973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1653s for a response
W20260709 12:25:24.935169  5042 connection.cc:256] RPC call timeout handler was delayed by -1.0804979399995318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1698s for a response
W20260709 12:25:24.935519  5042 connection.cc:256] RPC call timeout handler was delayed by -1.0786979399995289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
W20260709 12:25:24.935909  5042 connection.cc:256] RPC call timeout handler was delayed by -1.0273979399999007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 12:25:24.936303  5042 connection.cc:256] RPC call timeout handler was delayed by -1.0204685879998578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 12:25:24.936682  5042 connection.cc:256] RPC call timeout handler was delayed by -1.0012979399998585s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260709 12:25:24.849018 10518 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:24.909454 10578 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:24.953626  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:24.954807  5948 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:25:24.955341  5948 pending_rounds.cc:85] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 8
I20260709 12:25:24.955588  5948 pending_rounds.cc:107] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:25:24.955812  5948 raft_consensus.cc:2889] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:24.956301  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:24.959689  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:24.961380  5943 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:24.964525 10540 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:24.965291 10540 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:24.967737 10540 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:24.972230 10535 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:24.999238  5042 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:25.006033  5070 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:25.011083  5147 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:25.017604  5899 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:25.017690  5042 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:24.895665 10538 raft_consensus.cc:3055] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:25.021427  5042 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:24.893450 10496 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:25.025848  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:24.852219 10576 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.182s
I20260709 12:25:25.031183  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:24.935053 10553 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:25.024359 10566 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:25:25.074632  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:25.012234 10535 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.027421 10416 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.024665 10538 raft_consensus.cc:740] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:25:25.078132  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 47031.6us
I20260709 12:25:25.034281 10570 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:25.049576 10298 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:25.033912 10424 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:25.048837 10568 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:25.024334 10548 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:25:25.075003 10569 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:25.030908 10567 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:25.131705 10567 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:25.134258 10567 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:25.136574 10584 log.cc:927] Time spent T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 12:25:25.137938 10585 log.cc:927] Time spent T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:25:25.142086 10538 consensus_queue.cc:260] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:25.147490  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 1057) took 1039 ms. Trace:
I20260709 12:25:25.148052  5147 rpcz_store.cc:276] 0709 12:25:24.107821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.109656 (+  1835us) service_pool.cc:224] Handling call
0709 12:25:24.110065 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.012533 (+902468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.147171 (+134638us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b540022ea54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:25.147403 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2071001856}
W20260709 12:25:25.085747  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 280161us
I20260709 12:25:25.110497  5042 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:25.115361  5067 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:25.083446 10535 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:25.117887  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 131569us
I20260709 12:25:25.049420 10578 consensus_queue.cc:1059] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:25:25.048767 10261 consensus_queue.cc:1059] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:25:25.157636 10578 raft_consensus.cc:3055] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:25.158165 10261 consensus_queue.cc:1243] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer 90e5220a62ed42fe83c82d0302d18fac log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:25:25.159682 10571 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:25.160068  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:25.160976  5223 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:25.160737 10260 consensus_queue.cc:260] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:25.164211 10260 raft_consensus.cc:3060] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:25.077972 10550 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 12:25:25.095520 10334 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.168964 10334 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:25.171403 10334 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:25.174690 10487 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:25.176151 10335 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:25.176941 10487 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:25.179674  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:25.180434  5939 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:25.119864 10416 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:25.084022  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 177412us
I20260709 12:25:25.080463 10582 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:25.156437  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:25.091826 10563 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:25.168201  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 1044) took 1504 ms. Trace:
I20260709 12:25:25.080389 10593 consensus_queue.cc:1059] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
W20260709 12:25:25.195675  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 85308.1us
I20260709 12:25:25.204890  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:25.189411  5215 rpcz_store.cc:276] 0709 12:25:23.664241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.664400 (+   159us) service_pool.cc:224] Handling call
0709 12:25:23.665007 (+   607us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:23.737223 (+ 72216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.737257 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:23.747178 (+  9921us) log.cc:844] Serialized 52 byte log entry
0709 12:25:23.928504 (+181326us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:23.928559 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.167609 (+1239050us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b54003b7c54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:25.167678 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.167758 (+    80us) raft_consensus.cc:1685] finished
0709 12:25:25.167778 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.168113 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51362,"raft.queue_time_us":51829,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":10409,"spinlock_wait_cycles":46629632,"thread_start_us":158382,"threads_started":1,"wal-append.queue_time_us":324915,"wal-append.run_cpu_time_us":2100,"wal-append.run_wall_time_us":1004201}
W20260709 12:25:25.202105  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 49530.3us
I20260709 12:25:25.207223  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:25.240113  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.241518  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.241457  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48612.6us
I20260709 12:25:25.243130 10510 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:25.243698  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:25.245393  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:25.246194  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:25.244442  5188 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:25:25.254546  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 409254us
I20260709 12:25:25.255409  5898 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:25.251492  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.181785 10568 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:25.257930  5216 raft_consensus.cc:2393] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 18: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:25.158200 10578 raft_consensus.cc:740] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:25:25.268360  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 61279.6us
I20260709 12:25:25.194080 10550 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.276954  5071 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:25.196777 10563 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:25.283804  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:25.286965  5041 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:25.276337 10548 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:25:25.251268 10529 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:25.295701  5042 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:25.246773 10335 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:25.299459  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:25.304977  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:25.260468 10489 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:25.309691  5223 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:25.309867  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:25.270956 10567 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:25.271976 10400 raft_consensus.cc:2804] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:25.325198  5042 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:25.290271 10563 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:25.238660 10424 consensus_queue.cc:1059] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
W20260709 12:25:25.328230  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.330601 10400 raft_consensus.cc:697] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:25.311352 10593 raft_consensus.cc:3055] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:25.314958 10442 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:25.299626 10570 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:25.304378 10505 raft_consensus.cc:2804] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:25:25.337456  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 66283.6us
I20260709 12:25:25.353116  5115 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:25.354995  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:25.254032  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:25.276101 10510 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:25.331233 10353 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:25.315477 10567 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:25.348188  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:25.323992 10569 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:25.333647 10597 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:25.286399 10550 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:25.316072 10554 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.037s
W20260709 12:25:25.374883  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 120974us
W20260709 12:25:25.346686  9888 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.002s
I20260709 12:25:25.317868 10335 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:25.345228 10593 raft_consensus.cc:740] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:25.377398 10589 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:25.362952 10591 consensus_queue.cc:1059] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:25:25.391454  5224 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:25.359850 10442 raft_consensus.cc:697] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:25.370337 10597 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.400779  5215 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:25:25.402541 10601 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:25.370236 10567 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.374949 10512 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:25.378171 10545 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:25.374286 10505 raft_consensus.cc:697] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:25.486490 10594 log.cc:927] Time spent T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:25:25.488484  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:25.402603 10591 raft_consensus.cc:3055] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:25.491290 10593 consensus_queue.cc:260] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:25.494174 10442 consensus_queue.cc:237] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:25.495083  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:25.496393 10416 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:25.496526  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:25.566380  5220 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:25.389076 10428 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:25.390332  9888 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:25.521715 10400 consensus_queue.cc:237] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:25.356159 10578 consensus_queue.cc:260] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:25.396064 10540 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:25.535633 10335 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:25.374457 10334 consensus_queue.cc:1059] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 12:25:25.489130  5216 raft_consensus.cc:1240] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 16. Current term is 17. Ops: []
W20260709 12:25:25.541199 10488 consensus_meta.cc:287] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.828s	user 0.012s	sys 0.001s
W20260709 12:25:25.546881  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 191950us
I20260709 12:25:25.588460  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:25.554654 10591 raft_consensus.cc:740] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:25.576267  5948 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:25:25.577268  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 216260us
I20260709 12:25:25.578589 10615 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.543021 10597 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:25.549000 10496 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.006s
I20260709 12:25:25.545159 10599 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:25.590008 10553 consensus_meta.cc:287] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.509s	user 0.000s	sys 0.012s
I20260709 12:25:25.594105  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:25.558396  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 210291us
I20260709 12:25:25.370725 10568 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:25.597101  5219 raft_consensus.cc:1240] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:25.607316  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:25.587659 10540 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:25.616292  5115 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:25.612453  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:25.595306 10593 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:25.623312  5115 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:25.638684  5220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:25:25.639825  5220 raft_consensus.cc:606] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:25.600697 10428 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:25.647511  5220 raft_consensus.cc:614] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:25.639978  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.654480  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 188866us
W20260709 12:25:25.656455  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 44781.1us
I20260709 12:25:25.659986  5041 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:25.611251 10334 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:25.620078 10540 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:25.593060 10488 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:25.625046 10524 consensus_meta.cc:287] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.004s
I20260709 12:25:25.627885 10496 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:25.638101 10545 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:25.666951  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:25.610800 10553 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:25.664440  9978 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:25.699730  9927 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:25.700228 10611 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:25.701659 10608 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:25:25.702935  5072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1355) took 1104 ms. Trace:
I20260709 12:25:25.690544 10545 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:25.703457  5072 rpcz_store.cc:276] 0709 12:25:24.598780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.598975 (+   195us) service_pool.cc:224] Handling call
0709 12:25:24.599515 (+   540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.725733 (+126218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.725770 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:24.725817 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:24.725848 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.702695 (+976847us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b540058aa54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:25.702750 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.702865 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289595136}
I20260709 12:25:25.704296 10209 raft_consensus.cc:3055] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:25.704782 10209 raft_consensus.cc:740] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:25.703474  5071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1360) took 1096 ms. Trace:
I20260709 12:25:25.705731 10209 consensus_queue.cc:260] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:25.706548  5071 rpcz_store.cc:276] 0709 12:25:24.606606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.606726 (+   120us) service_pool.cc:224] Handling call
0709 12:25:24.607022 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.278002 (+670980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.703236 (+425234us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b5400328a54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:25.703411 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1539873024}
I20260709 12:25:25.707199 10209 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:25.707715  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.709369 10537 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.710021 10537 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:25.712038  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 456121us
I20260709 12:25:25.712613 10537 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:25.720610 10605 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:25:25.700371  9927 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.705179 10545 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:25.692327  5898 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:25.669646 10424 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:25.723727  5941 raft_consensus.cc:1240] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 19. Current term is 20. Ops: []
W20260709 12:25:25.747141  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 54904.6us
I20260709 12:25:25.687021 10569 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:25.674190 10595 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:25.688759 10550 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:25.757054  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:25.690382 10297 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:25.695231 10510 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:25.609416 10599 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:25.766049  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 131074us
I20260709 12:25:25.762394  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:25.690642 10334 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:25:25.687490 10524 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:25.753027 10539 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:25.795764  5115 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:25.795991  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 1060) took 2316 ms. Trace:
I20260709 12:25:25.808768  5940 rpcz_store.cc:276] 0709 12:25:23.479145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.479287 (+   142us) service_pool.cc:224] Handling call
0709 12:25:23.498731 (+ 19444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.064712 (+565981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.795740 (+1731028us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b54002e5254. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:25.795931 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1299863424}
I20260709 12:25:25.791095  5074 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:25.809576  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:25.798034  5071 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:25.797883  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:25.800633  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:25.816736  5075 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1363) took 1196 ms. Trace:
I20260709 12:25:25.605108 10615 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:25.745579  9927 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:25.803571  5221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 1349) took 2094 ms (client timeout 2142 ms). Trace:
W20260709 12:25:25.796398 10298 consensus_meta.cc:287] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.670s	user 0.014s	sys 0.000s
I20260709 12:25:25.757277 10595 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:25.759886 10569 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:25.817392  5075 rpcz_store.cc:276] 0709 12:25:24.620676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.620848 (+   172us) service_pool.cc:224] Handling call
0709 12:25:24.621320 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.768477 (+147157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.816365 (+1047888us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400327654. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:25.816646 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336554240}
I20260709 12:25:25.763984 10297 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:25.837133  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:25.824957  5221 rpcz_store.cc:269] 0709 12:25:23.709127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.862757 (+153630us) service_pool.cc:224] Handling call
0709 12:25:25.767042 (+1904285us) spinlock_profiling.cc:243] Waited 93.4 ms on lock 0x7b5400318b54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:25.803464 (+ 36422us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214709632}
I20260709 12:25:25.839491  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:25.844614  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:25.845093  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:25.846424  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:25.775839  9888 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:25.846431  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:25.853314  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:25.859622  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.853278  5071 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:25.859445  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:25.781318 10429 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:25.862465  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:25.863445  5067 raft_consensus.cc:2376] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19. Re-sending same reply.
I20260709 12:25:25.865365  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:25.639079 10505 consensus_queue.cc:237] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:25.861419  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:25.792893 10542 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 12:25:25.869467  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:25.838328 10503 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:25.872423  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.842234 10298 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:25.850273 10428 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:25.882225  5115 leader_election.cc:400] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15.
W20260709 12:25:25.882594  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.886516  5115 leader_election.cc:403] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:25.904695  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.858137 10611 consensus_queue.cc:1059] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:25:25.911448 10595 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:25.914114 10615 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:25.893684 10568 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.878327 10620 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:25.918285 10568 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:25.929845 10334 consensus_queue.cc:1059] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:25.931087 10334 consensus_queue.cc:1243] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:25.931461 10595 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15.
I20260709 12:25:25.932538 10595 raft_consensus.cc:3055] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:25.932977 10595 raft_consensus.cc:740] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:25.930127 10611 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.934469 10611 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:25.883437 10429 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:25.888664 10496 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:25.945221  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 330875us
W20260709 12:25:25.947256  5190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.948978  5190 leader_election.cc:336] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.087s (SENT)
I20260709 12:25:25.960538  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:25.945581 10611 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:25.968674  5073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1359) took 1363 ms. Trace:
I20260709 12:25:25.970037  5073 rpcz_store.cc:276] 0709 12:25:24.605492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.614075 (+  8583us) service_pool.cc:224] Handling call
0709 12:25:24.620277 (+  6202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.729820 (+109543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.968297 (+1238477us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54004290d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:25.968581 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234282880}
I20260709 12:25:25.972110 10400 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:25.982991  5069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1362) took 1375 ms. Trace:
I20260709 12:25:25.983660  5069 rpcz_store.cc:276] 0709 12:25:24.607221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.607345 (+   124us) service_pool.cc:224] Handling call
0709 12:25:24.730915 (+123570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.731757 (+   842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.982705 (+1250948us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540041e1d0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853a8ad 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:25.982915 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283370880}
I20260709 12:25:25.991123  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1355) took 1032 ms. Trace:
I20260709 12:25:25.991753  5948 rpcz_store.cc:276] 0709 12:25:24.958890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.963212 (+  4322us) service_pool.cc:224] Handling call
0709 12:25:24.963544 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.595861 (+632317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.990772 (+394911us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b5400006c54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:25.991037 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1408137088}
I20260709 12:25:25.943441 10567 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:25.963900 10535 consensus_meta.cc:287] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.789s	user 0.015s	sys 0.000s
W20260709 12:25:26.022038 10563 consensus_meta.cc:287] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.676s	user 0.011s	sys 0.002s
I20260709 12:25:26.042173  5945 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:26.059378 10563 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:26.061235  5076 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:26.016314 10535 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.043123 10576 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.997470 10553 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:26.062727  5068 raft_consensus.cc:2368] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 18. Current term is 19.
W20260709 12:25:26.063488 10529 consensus_meta.cc:287] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.711s	user 0.014s	sys 0.000s
I20260709 12:25:26.040050 10568 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:26.064160 10529 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:26.063979 10576 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:26.025091  5075 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:26.042228  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:26.065346  5069 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:25:26.066642 10518 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:26.070667  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:26.071970 10576 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:26.071851 10518 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:26.073235 10518 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:26.074437 10518 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:25.966259 10488 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:25.952463 10569 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:26.088447  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:26.089442  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:26.089808  5948 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:26.088903  5940 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:26.097422  5074 raft_consensus.cc:606] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:25:26.098619  5074 raft_consensus.cc:614] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:26.100526  5219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 1058) took 1576 ms. Trace:
I20260709 12:25:26.101082  5219 rpcz_store.cc:276] 0709 12:25:24.524172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.524616 (+   444us) service_pool.cc:224] Handling call
0709 12:25:24.524965 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.612075 (+1087110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.100197 (+488122us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400273dd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:26.100448 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2460551552}
I20260709 12:25:26.103901 10538 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:26.105103  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.111447  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1354) took 1153 ms. Trace:
I20260709 12:25:26.112151 10578 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:26.111922  5941 rpcz_store.cc:276] 0709 12:25:24.958551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.959808 (+  1257us) service_pool.cc:224] Handling call
0709 12:25:24.960302 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.746995 (+786693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.111210 (+364215us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b5400009e54. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:26.111384 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1687930240}
I20260709 12:25:25.965514 10595 consensus_queue.cc:260] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:25.995946 10591 consensus_queue.cc:260] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:26.124893  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.108251  5188 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:26.141224  5948 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:26.080063 10524 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:26.115147 10395 consensus_meta.cc:287] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 1.263s	user 0.014s	sys 0.002s
I20260709 12:25:26.097347  9927 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:26.159983  5041 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:26.107638 10599 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:26.165761  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 279884us
W20260709 12:25:26.166317  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 284242us
I20260709 12:25:26.154150 10400 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:26.106626 10529 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
W20260709 12:25:26.166326 10519 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.000s	sys 0.046s
W20260709 12:25:26.184007 10519 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.000s	sys 0.047s
W20260709 12:25:26.184489 10623 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:25:26.184693  5941 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:25:26.187543 10209 consensus_queue.cc:1059] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:26.188844 10209 raft_consensus.cc:3055] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:26.144517 10512 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:25:26.194152 10488 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:26.200595  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:26.190709 10395 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.166127 10591 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:26.189297 10209 raft_consensus.cc:740] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:26.204543 10536 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:26.168612 10595 raft_consensus.cc:3060] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:26.208192  5941 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 17: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:26.212389  5945 raft_consensus.cc:1240] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:26.222632  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:26.170910  9978 raft_consensus.cc:2804] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:26.215691  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:26.226630  5219 raft_consensus.cc:2410] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:25:26.219578  5222 raft_consensus.cc:1240] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 12:25:26.226910  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:26.224123  5947 raft_consensus.cc:1240] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:25:26.228839  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:26.208545 10400 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:26.235167  5220 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:25:26.250891 10400 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:26.244174  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:26.217064 10395 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:26.254237  9978 raft_consensus.cc:697] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:26.255079 10395 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:26.255327  9978 consensus_queue.cc:237] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:26.262077 10599 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.234752  5941 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:26.264474  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:26.249818  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 77513.5us
I20260709 12:25:26.266247  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:26.220739 10545 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:26.268702  5948 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 11.
I20260709 12:25:26.221916 10519 raft_consensus.cc:3055] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:26.216776 10628 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 12:25:26.285156 10627 log.cc:927] Time spent T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:25:26.267419 10395 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:26.291864 10589 raft_consensus.cc:3055] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:26.296919 10535 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:26.273706 10519 raft_consensus.cc:740] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:26.278144 10209 consensus_queue.cc:260] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.299618  9927 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:26.374217  5042 leader_election.cc:336] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.528s (SENT)
I20260709 12:25:26.382345  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:26.386503  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:26.388290  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:26.398419 10209 raft_consensus.cc:3060] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:26.302528 10589 raft_consensus.cc:740] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:25:26.321770 10597 consensus_meta.cc:287] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: Time spent flushing consensus metadata: real 0.711s	user 0.008s	sys 0.005s
I20260709 12:25:26.262729 10599 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:26.434016 10580 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:26.435511 10580 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:26.438114 10586 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:26.439594 10586 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:26.441805 10628 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:26.442508 10628 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:25:26.324792 10622 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.138s	user 0.002s	sys 0.000s
W20260709 12:25:26.399042  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.429379 10334 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.303890 10395 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:26.451567 10613 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:26.452773 10613 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:26.454063  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 175658us
I20260709 12:25:26.446085 10629 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.490804  5067 raft_consensus.cc:2410] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:25:26.493005 10574 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:26.489764 10519 consensus_queue.cc:260] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:26.444689 10628 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:26.497843  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 101504us
I20260709 12:25:26.507459  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:26.508811  5947 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:25:26.511924  5150 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:26.456247 10613 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:26.521239 10519 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:26.519760 10599 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:26.517395  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.531270  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:26.532558  5220 raft_consensus.cc:3060] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:26.535097 10334 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:26.538569  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:26.539791  5145 raft_consensus.cc:2376] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 11. Re-sending same reply.
I20260709 12:25:26.541849  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:26.543489  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:26.544818  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:26.547911  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:26.550091  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.551134  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.551957  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.494464 10574 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:26.514112 10597 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:26.570319 10574 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:26.571964 10597 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:26.540665  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:26.575068  5073 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:26.580989  5222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:25:26.577261  5944 raft_consensus.cc:1240] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:26.577049 10578 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:26.532909 10629 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:26.531893 10589 consensus_queue.cc:260] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:26.600435 10589 raft_consensus.cc:3060] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:26.600872 10567 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:26.598532  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:26.604596  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:26.607378  5143 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 16: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:26.613482  5218 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:25:26.613219 10628 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:26.637903  5069 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:25:26.654209  5190 leader_election.cc:336] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.546s (SENT)
I20260709 12:25:26.656246  5147 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:26.655545 10334 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:26.672405  5070 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:26.666503  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:26.673620  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:26.673785  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:25:26.676102  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 50140.4us
W20260709 12:25:26.680505  5900 leader_election.cc:336] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.873s (SENT)
I20260709 12:25:26.681427  5900 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:26.683341 10527 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:25:26.686009 10505 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:26.687383 10505 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.687732  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:26.688764  5145 raft_consensus.cc:2410] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:25:26.688936 10395 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:26.691237 10586 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:26.693149 10542 consensus_queue.cc:1059] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:25:26.694105  5899 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:26.694449 10542 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:26.694914 10542 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:26.695454 10579 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:26.698441 10602 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.699304 10602 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.700246  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:26.696487  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:25:26.701458 10297 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.835s	user 0.007s	sys 0.015s
W20260709 12:25:26.702114  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 54758.1us
I20260709 12:25:26.696508 10542 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:26.688540  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:26.704317 10542 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:26.704917  5150 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 17: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:26.705848 10563 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:26.706732 10629 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:26.706564 10569 consensus_queue.cc:1059] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:25:26.707957 10569 consensus_queue.cc:1243] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:26.708709 10569 raft_consensus.cc:3055] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:26.709137 10569 raft_consensus.cc:740] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:26.702087 10297 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:26.716045 10524 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:26.716356 10525 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:26.717026 10334 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.650274 10578 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.723287 10525 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.729939  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:26.730809 10525 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:26.735076  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:26.736008  5943 raft_consensus.cc:1275] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:26.717382  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:26.738360  5220 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 19.
I20260709 12:25:26.671630 10553 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:26.752419  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:26.753691  5218 raft_consensus.cc:2393] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 18: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:25:26.754243  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 58007.3us
I20260709 12:25:26.754549 10569 consensus_queue.cc:260] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:26.756524 10569 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:26.758318 10529 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.758610  9927 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:26.759022 10529 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.763439 10579 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:25:26.716054 10488 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:26.778488 10488 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.779160 10505 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:26.780923  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 172853us
I20260709 12:25:26.786005 10529 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:26.787884 10540 consensus_meta.cc:287] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.100s	user 0.005s	sys 0.019s
I20260709 12:25:26.788581 10540 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.790100  5219 raft_consensus.cc:1240] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:26.791587  5145 raft_consensus.cc:1275] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:26.792917  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:26.792970  5150 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:26.793897  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:26.794576 10488 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:26.792150 10297 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:26.795059  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:26.798736 10576 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:26.800254  5944 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:25:26.719925 10524 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:25:26.801229  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.819479  5141 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:26.761427  9927 consensus_queue.cc:1059] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 12:25:26.823926  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.826265  5947 raft_consensus.cc:2468] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 19.
I20260709 12:25:26.763305 10334 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:26.741441 10580 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:26.863207  5144 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:26.865188  5146 raft_consensus.cc:2410] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
W20260709 12:25:26.870858  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.874826  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:26.877439  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:26.877708  5142 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.881966 10491 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:26.882882 10491 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.887540  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:26.888595  5076 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:26.891072  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:26.891830  5075 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:25:26.892052  5216 raft_consensus.cc:1240] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:26.893558  5216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35624 (request call id 1646) took 1283 ms. Trace:
I20260709 12:25:26.893985  5216 rpcz_store.cc:276] 0709 12:25:25.610084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.616386 (+  6302us) service_pool.cc:224] Handling call
0709 12:25:25.616841 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.892510 (+1275669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.893227 (+   717us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540086b5d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:26.893451 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2877077888}
I20260709 12:25:26.895998 10540 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:26.827514 10524 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:26.899336 10578 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:26.901186 10535 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.902225 10535 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.825179 10553 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.905875 10535 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:26.909876  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 80732.1us
I20260709 12:25:26.916301 10334 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:26.923323  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:26.924384  5150 raft_consensus.cc:2376] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14. Re-sending same reply.
I20260709 12:25:26.925680  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:26.928151 10647 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 12:25:26.930197 10538 raft_consensus.cc:3055] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:26.930760 10538 raft_consensus.cc:740] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:26.932034 10538 consensus_queue.cc:260] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.935096  5142 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:26.936421  5146 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:26.943517  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:26.944579  5072 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 17: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:26.947950 10614 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.948743 10614 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:26.925479  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:26.952379  5150 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:26.953884 10536 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:26.954624 10536 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:26.957291 10536 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:26.959901 10491 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:26.962903  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 91242us
W20260709 12:25:26.966118  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.963064  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:26.969568  5142 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:26.976459  5141 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:26.979036  5075 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
W20260709 12:25:26.977378  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 221106us
I20260709 12:25:26.979187  5148 raft_consensus.cc:2468] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:26.953909 10614 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:26.985150  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:26.992187  5070 raft_consensus.cc:1240] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:26.994343  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.002069  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:26.999246  5072 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:27.005055  5141 raft_consensus.cc:1275] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:27.010775  5216 raft_consensus.cc:2410] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:25:26.997376 10538 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:27.026151  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:27.027691  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 53093.9us
I20260709 12:25:27.029855  5222 raft_consensus.cc:606] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:27.031176  5141 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
W20260709 12:25:27.037398  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.038054  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.039891  5149 raft_consensus.cc:2410] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:27.041025  5217 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:27.046175  5114 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:27.047058  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.048005  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:27.048389  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.048754  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.049358  5146 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:27.050334  5141 raft_consensus.cc:2410] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:25:27.050421  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:27.051538  5146 raft_consensus.cc:2410] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 18 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 12:25:27.052969  5149 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:27.053697  5144 raft_consensus.cc:1275] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:27.021966 10614 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:27.054945  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.059945  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 148163us
W20260709 12:25:27.067076  5900 leader_election.cc:336] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.911s (SENT)
I20260709 12:25:27.060793 10628 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:27.075033  5940 raft_consensus.cc:1275] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:25:27.077713  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.060592 10553 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:27.080852  5141 raft_consensus.cc:1275] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:25:27.079677  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:27.081605  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 169281us
I20260709 12:25:27.080996  5142 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
W20260709 12:25:27.092551  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.090983  5141 pending_rounds.cc:85] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 8
I20260709 12:25:27.093638  5144 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
W20260709 12:25:27.097343  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.098130  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:27.072359 10628 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.095372  5141 pending_rounds.cc:107] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:25:27.104218  5141 raft_consensus.cc:2889] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:27.071090 10614 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:27.072803  9927 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:25:27.107448  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.104725 10628 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:27.117381 10548 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:25:27.117699 10488 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.117486 10661 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:27.118675  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 57492.2us
I20260709 12:25:27.119171 10542 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:27.126959 10660 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:27.129426  5898 leader_election.cc:400] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 18. Current term is 19.
I20260709 12:25:27.167122  5898 leader_election.cc:403] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:27.138738 10602 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:27.115700  5041 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:27.159330  5941 raft_consensus.cc:1240] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:27.161854  5940 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 17: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:27.171200  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.172322  5141 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:27.166527 10491 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:25:27.174806  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:27.175410 10662 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:25:27.176602  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.182296  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 158217us
I20260709 12:25:27.185591  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.186689  5145 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:27.187686 10505 raft_consensus.cc:2804] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:27.188398 10505 raft_consensus.cc:697] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:27.189780 10553 consensus_queue.cc:1059] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:27.191114 10442 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:27.194916  5073 raft_consensus.cc:1240] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:27.185810 10518 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.191359 10553 raft_consensus.cc:3055] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:27.185627 10541 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.205691  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.206835  5141 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:27.181069 10661 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:27.213658  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.212886  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:27.224079  5141 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:27.208233 10525 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 18. Current term is 19.
I20260709 12:25:27.207984 10553 raft_consensus.cc:740] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:27.233553  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:27.243075  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:27.248237  5069 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:27.250877  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:27.209843 10488 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:27.189851 10505 consensus_queue.cc:237] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:27.257189  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.212474 10541 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:27.262737  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:27.212244 10518 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:27.267325  5076 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 19.
I20260709 12:25:27.279176 10578 raft_consensus.cc:3055] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:27.279769 10578 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:27.235945 10586 consensus_queue.cc:1059] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
W20260709 12:25:27.292954  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 163898us
I20260709 12:25:27.296271  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.296945  5945 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:27.293823  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:27.303318 10510 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:27.303726  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:27.306448  5144 raft_consensus.cc:1240] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:25:27.304518  5071 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 12.
I20260709 12:25:27.314563 10524 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:27.280790 10578 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:27.321135  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.323426  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.325223  5145 raft_consensus.cc:2410] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:25:27.329487  5142 raft_consensus.cc:1240] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:27.330307  5145 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:27.327315 10569 consensus_meta.cc:287] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.570s	user 0.015s	sys 0.000s
I20260709 12:25:27.333070  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:27.336199  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.336985  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.330479 10488 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:27.350174 10661 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:27.350864  5147 raft_consensus.cc:1275] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:27.340550  5144 raft_consensus.cc:2393] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 17: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:27.330399 10553 consensus_queue.cc:260] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:27.345968  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:27.347047 10524 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:27.304160 10510 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:27.353686  5142 raft_consensus.cc:1240] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:25:27.354604  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:27.358489  5148 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:27.358451  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:27.354632 10614 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:27.364132 10510 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:27.366351 10553 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:27.398667  5148 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:27.402313 10505 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:27.403137 10505 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:27.406358  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.408273  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.408638  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:27.410018  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.410470 10505 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:27.411018  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.411588 10541 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:27.414600  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:27.408339  5142 raft_consensus.cc:1240] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:27.416157  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.419459 10510 consensus_queue.cc:1048] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:27.428438  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:27.434741  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:27.438506  5220 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:25:27.438999  5220 pending_rounds.cc:85] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 10
I20260709 12:25:27.439245  5220 pending_rounds.cc:107] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:25:27.439489  5220 raft_consensus.cc:2889] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:27.451208  5148 raft_consensus.cc:2468] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:27.454521  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:27.391691 10660 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:27.459932  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.464247  5947 raft_consensus.cc:2410] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:27.467424  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.468228  5147 raft_consensus.cc:2468] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:27.469636  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:27.471305  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.472015  5142 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:27.472765  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.474361  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.473366  5142 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:27.479987  5944 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:27.481825  5144 raft_consensus.cc:2410] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:25:27.480068  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:27.482990  5148 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:27.481313  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.485783  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:27.486666  5147 raft_consensus.cc:2376] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17. Re-sending same reply.
I20260709 12:25:27.449110 10518 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:27.493094  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.504372  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.508814  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.505779  5069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:27.511708  5069 raft_consensus.cc:606] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:25:27.512109  5069 raft_consensus.cc:614] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:27.514281  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.507102 10488 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.516327 10488 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:27.523114  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:27.524015  5220 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:27.524715  5069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:27.525838  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:27.526089  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.527097  5147 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:27.528054  5069 raft_consensus.cc:606] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:25:27.528599  5069 raft_consensus.cc:614] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:27.530344  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:27.515057  5144 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:27.536464 10488 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:27.538395  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:27.542213  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:27.547075 10578 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:27.556218 10505 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:27.557089 10505 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:27.559264  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.559998  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.560250 10505 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:27.556957 10599 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:25:27.527230  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 71607.9us
I20260709 12:25:27.568243 10576 raft_consensus.cc:3055] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:27.581746  5148 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
W20260709 12:25:27.591466  5042 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.501242 10660 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:27.609916  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:27.614264  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.613488  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:27.491967 10628 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:27.616719  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:27.619724  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:27.621203  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.587086 10576 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:27.623432  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.616525 10586 consensus_queue.cc:1243] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 12:25:27.638674  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.639346  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.613852 10488 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.642673  5941 raft_consensus.cc:2468] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:27.643190  5075 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:27.645929  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:27.621604 10510 consensus_queue.cc:1059] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:25:27.652171 10209 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.656616 10488 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:27.642766 10334 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.657279 10209 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:27.658401  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:27.662014 10576 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:27.663269  5948 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:27.652875  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:27.670222  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.677290  5189 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:27.682431  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.686293  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:27.685717  5899 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:27.688406  5220 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
W20260709 12:25:27.689481  5042 leader_election.cc:336] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.043s (SENT)
I20260709 12:25:27.690060  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:27.692566  5218 raft_consensus.cc:2410] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:25:27.678615 10589 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:25:27.684496 10679 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
I20260709 12:25:27.686609 10576 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.699390  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:27.668877 10334 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:27.723163  5188 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:27.716233  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:27.692304 10599 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:27.692792 10518 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:27.704208 10491 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:25:27.727408 10535 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:25:27.734974  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 56594.8us
I20260709 12:25:27.689881 10660 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:27.737995  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:27.738376  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:27.740217  5069 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:25:27.741696 10209 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:27.742368 10488 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:27.743248 10545 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:27.731086 10518 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.744402 10518 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:27.729327 10599 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.746178 10599 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:27.747264  5069 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:27.748102  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:27.728170 10510 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:27.749667  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:27.714259 10519 raft_consensus.cc:3055] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:27.750375 10519 raft_consensus.cc:740] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:27.751101  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:27.729739  5189 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:27.702028 10589 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.724519  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:27.732666 10550 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:27.733424 10676 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.129s	user 0.001s	sys 0.001s
I20260709 12:25:27.749827 10510 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.692745 10579 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
W20260709 12:25:27.761230  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.767110  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.759320  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 43557.8us
W20260709 12:25:27.757328  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.802723  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 73084.5us
I20260709 12:25:27.803527  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:27.805747  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.755802 10589 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:27.751443 10519 consensus_queue.cc:260] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:27.811333  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 51969.7us
I20260709 12:25:27.755856 10550 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:27.819232  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:27.762910 10510 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:27.822636  5070 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:27.809233 10505 raft_consensus.cc:3060] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:27.733459 10424 consensus_queue.cc:1059] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.142s
I20260709 12:25:27.823310 10602 raft_consensus.cc:3055] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:27.823665 10550 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.844962  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:27.821756 10586 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.821292 10548 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:27.822129 10488 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.849411  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:27.863588  5074 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:27.865104  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 61617.2us
W20260709 12:25:27.866295  5900 leader_election.cc:336] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.860s (SENT)
I20260709 12:25:27.866997  5900 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:27.835832 10628 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:27.872283  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:27.846170 10519 raft_consensus.cc:3060] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:27.828653 10614 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:27.833068 10334 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:27.846524 10538 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.838451 10545 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:25:27.870386 10586 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:27.863113 10553 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:27.891130  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.891654 10488 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:27.849123 10602 raft_consensus.cc:740] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:27.859607  9927 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:25:27.875394 10682 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:25:27.866302 10524 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:25:27.918203  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 115294us
W20260709 12:25:27.920356  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 75516.4us
W20260709 12:25:27.933890  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.945216  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:27.914526 10548 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:27.913050 10660 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:27.958227  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 89837.2us
W20260709 12:25:27.972489  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 100332us
I20260709 12:25:27.973461  5898 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:27.960393 10529 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:27.950951 10334 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.916648 10614 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:27.980096  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:27.988124  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.988665  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:27.958293  9927 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:28.000949 10535 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:27.926072 10538 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:27.995792  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:27.996004 10334 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:28.008896  9927 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:28.020037 10488 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:28.021133 10580 raft_consensus.cc:3055] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:27.974027 10553 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:28.025282 10553 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:28.028059 10553 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:28.029016 10683 log.cc:927] Time spent T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:25:28.030802  5115 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:28.032418 10593 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:25:28.033223  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.024619 10580 raft_consensus.cc:740] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:25:28.055195  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.051271  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 55627.8us
W20260709 12:25:28.055174  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 95882.6us
W20260709 12:25:28.045914  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 99796.2us
W20260709 12:25:28.042325  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.063234  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:28.063498  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:27.967135 10524 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:28.069389  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.050029 10661 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:28.018312 10671 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:28.019105 10688 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 12:25:28.072016  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.019922 10690 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:27.995738 10621 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:25:28.015514 10614 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:28.061815 10553 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:28.024323 10334 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:28.071702 10529 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.088265 10589 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:28.089232  5115 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:28.089879  5189 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:28.020239 10599 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:28.098716 10580 consensus_queue.cc:260] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:28.099763 10600 raft_consensus.cc:993] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: : Instructing follower 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:28.084663  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:28.071491 10545 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:25:28.097074 10518 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:28.125105 10518 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:28.128496 10599 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:28.128859  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 47502.8us
I20260709 12:25:28.129222 10488 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:28.129696 10488 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.112543 10529 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:28.130100 10699 raft_consensus.cc:2804] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:28.130398 10600 raft_consensus.cc:1081] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Signalling peer 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:28.130777 10699 raft_consensus.cc:697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:28.131688 10699 consensus_queue.cc:237] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:28.128530 10589 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:28.134836 10580 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:28.018090 10602 consensus_queue.cc:260] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:28.153095  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 64063.4us
I20260709 12:25:28.154136  5115 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:28.153998 10550 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:28.160395 10510 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:28.162756 10578 consensus_meta.cc:287] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.615s	user 0.010s	sys 0.005s
I20260709 12:25:28.164578 10524 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:28.164806  5219 raft_consensus.cc:1240] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:28.165674  5074 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
W20260709 12:25:28.166942 10688 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:28.130112 10488 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:28.168887 10586 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:28.169029  5217 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:25:28.178474  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:28.179204  5075 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:28.172804  5900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.750s (SENT) [suppressed 32 similar messages]
I20260709 12:25:28.175613  5114 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:28.168268 10510 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:28.179867  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:25:28.205495  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.183996  5942 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:39878
I20260709 12:25:28.213593  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:28.214385  5941 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 18: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:28.214781 10586 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:28.164593 10550 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:28.184442 10524 consensus_queue.cc:1059] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:25:28.217751 10524 raft_consensus.cc:3055] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:28.218161 10524 raft_consensus.cc:740] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:28.218674 10529 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:28.219072 10524 consensus_queue.cc:260] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:28.222715  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 47185.8us
W20260709 12:25:28.223649  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.140412 10334 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:28.225595  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 86483us
I20260709 12:25:28.213635  5942 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:28.226989  5222 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 18: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:28.223982 10553 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:28.235646  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 262266us
I20260709 12:25:28.238459  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:28.238588  5942 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:28.241623  5141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 1079) took 1002 ms. Trace:
W20260709 12:25:28.244166  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.248945  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.227011 10614 raft_consensus.cc:2804] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:28.259054  5898 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:28.187120 10578 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:25:28.260586  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:28.266494  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 112457us
W20260709 12:25:28.251217 10706 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:25:28.242304  5141 rpcz_store.cc:276] 0709 12:25:27.246966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.253874 (+  6908us) service_pool.cc:224] Handling call
0709 12:25:27.262931 (+  9057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.624193 (+361262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.624245 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:27.638396 (+ 14151us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.947279 (+308883us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b18004871e0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db0d1 00007fd8a84f6a0f
0709 12:25:28.125880 (+178601us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:28.125947 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.126061 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:28.231526 (+105465us) raft_consensus.cc:1685] finished
0709 12:25:28.231551 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:28.241496 (+  9945us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":333379,"raft.queue_time_us":345052,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":11933,"spinlock_wait_cycles":3926272,"thread_start_us":178308,"threads_started":1,"wal-append.queue_time_us":472041}
I20260709 12:25:28.249615 10524 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:28.270922  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:28.274652  5072 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:28.258834 10602 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:28.262459 10576 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:28.287173  5075 raft_consensus.cc:2468] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 20.
I20260709 12:25:28.247422 10553 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:28.278782 10614 raft_consensus.cc:697] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 17 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:28.293370 10661 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:25:28.303979  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.286211 10538 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:28.304886  5945 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:28.293259 10660 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:25:28.343999  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 49714.9us
I20260709 12:25:28.349916  5041 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:28.396224  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 46453.8us
W20260709 12:25:28.351241  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.352273  5114 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:28.365427 10614 consensus_queue.cc:237] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:28.372268  5190 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.833s (SENT)
I20260709 12:25:28.306897 10593 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:28.312613  5944 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:25:28.361997 10621 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:28.384232  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.385205  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.416054  5221 raft_consensus.cc:2393] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 19: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:28.405045  5190 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:28.396795  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:28.404196  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 52080.3us
W20260709 12:25:28.411888  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 46718us
I20260709 12:25:28.379820 10538 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.378275 10334 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:28.421447  5900 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.888s (SENT)
I20260709 12:25:28.422493  5900 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:28.422585  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:28.437034 10717 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:28.437474  5070 raft_consensus.cc:2393] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 20: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:28.438078 10717 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:28.438884 10518 raft_consensus.cc:3055] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:28.439391 10518 raft_consensus.cc:740] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:25:28.440147  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.451772  5943 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:28.449458  5072 raft_consensus.cc:2468] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 20.
I20260709 12:25:28.400624 10697 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:28.414325 10545 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:28.448354  5218 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:28.416613 10586 raft_consensus.cc:2804] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:28.460378  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:28.501032  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.418740 10548 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:25:28.506839  5042 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.543s (SENT)
I20260709 12:25:28.452762 10589 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.509093  5216 raft_consensus.cc:1275] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:25:28.511505  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 59753.4us
I20260709 12:25:28.424439 10621 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:28.461143 10519 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:28.518931  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 97746.7us
I20260709 12:25:28.463831 10545 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.516456  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.423583 10593 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:28.453459 10538 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:28.525121  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 84185.3us
I20260709 12:25:28.532087  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:28.533694  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:25:28.538159  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.537609 10519 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:28.536581 10488 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:28.540516 10519 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:28.541173 10488 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:28.542457  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:28.460074 10334 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:28.514262 10714 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:28.523875 10545 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:28.542214 10716 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:25:28.541361 10519 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:28.570838  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 198621us
I20260709 12:25:28.579581 10538 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:28.572449 10488 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:28.551656 10518 consensus_queue.cc:260] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:28.580920  5218 raft_consensus.cc:1240] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:25:28.581146 10695 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:28.581657  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:25:28.584949  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 78145us
I20260709 12:25:28.635248  5220 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:28.587669 10721 log.cc:927] Time spent T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:28.591910 10593 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:28.592383  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.513585 10697 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:28.598995 10718 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:28.600117 10717 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:28.603526 10334 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:28.604019 10553 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:28.607108 10614 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.655646 10614 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:28.634868 10660 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:28.627837 10519 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.657941 10519 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.658241 10614 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:28.614670 10505 consensus_meta.cc:287] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.778s	user 0.004s	sys 0.008s
I20260709 12:25:28.657931 10660 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:28.659257 10505 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:28.615859 10724 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:25:28.661198 10660 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:28.688647  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.509793 10548 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:28.621737  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.622927  5188 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:28.626722  5942 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:28.621270  5899 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:28.639683  5943 raft_consensus.cc:2468] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
W20260709 12:25:28.700526  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 79337us
W20260709 12:25:28.643358  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.650931 10488 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:28.651952  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.652283 10697 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:28.515053 10589 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:28.673130  5042 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.729s (SENT)
I20260709 12:25:28.670754 10519 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:28.674813  5190 leader_election.cc:336] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.178s (SENT)
W20260709 12:25:28.706926  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 84066.9us
I20260709 12:25:28.637833 10695 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:28.696731  5220 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:28.635509 10586 raft_consensus.cc:697] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 20 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:28.712302  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.700630 10698 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:28.718679  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.720924  5942 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:28.708252  9927 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:28.723991  5220 raft_consensus.cc:2393] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 18: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:28.724375  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:28.725482  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 157939us
I20260709 12:25:28.710701 10697 raft_consensus.cc:2804] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:28.726500  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:28.731489 10729 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:28.732717 10727 consensus_queue.cc:1059] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:25:28.733474 10488 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:28.734727  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 60051.8us
I20260709 12:25:28.733951 10548 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:28.735762 10589 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:28.736133 10519 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:28.736263 10548 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.736672 10695 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:28.737751 10550 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.737999 10695 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:28.738327 10550 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:28.741190 10550 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:28.741884  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 68778us
I20260709 12:25:28.742808  5189 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:28.743266  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:28.744314  5069 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:25:28.745630 10621 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:28.745680  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:28.746439  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:28.747014  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.747879 10586 consensus_queue.cc:237] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:28.749521  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:28.752238 10728 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.726850  9927 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.753616 10518 raft_consensus.cc:3060] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:28.752918 10728 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:28.755374 10545 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:28.755985 10728 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:28.756906  5946 raft_consensus.cc:1240] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:28.757267 10697 raft_consensus.cc:697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 20 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:28.752888 10714 raft_consensus.cc:2804] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:28.761909  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:28.762426  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:28.763439  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:28.737085 10548 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:28.772861  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 242827us
I20260709 12:25:28.774696  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.778080  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.774072 10714 raft_consensus.cc:697] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:28.782423  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.783341 10714 consensus_queue.cc:237] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:28.786557  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.753388  9927 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:28.793041  5941 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 20.
I20260709 12:25:28.775833  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.802806  5189 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:28.772208 10505 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:25:28.807134 10535 consensus_queue.cc:1048] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:28.774293 10692 consensus_queue.cc:1059] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:25:28.795025 10621 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.810863 10621 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:28.758596 10697 consensus_queue.cc:237] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:28.810257 10692 raft_consensus.cc:3055] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:28.812986  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.817760  5070 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
W20260709 12:25:28.820097  5042 leader_election.cc:336] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.791s (SENT)
I20260709 12:25:28.826011  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.830230 10545 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:28.834200 10621 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:28.834802 10692 raft_consensus.cc:740] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:28.834954 10548 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:28.835867 10692 consensus_queue.cc:260] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:28.837152 10692 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:28.818462 10729 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:25:28.766810 10698 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:25:28.843798  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.846140  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:28.847435  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.848611  5941 raft_consensus.cc:2410] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:25:28.850077  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:28.851254  5946 raft_consensus.cc:2393] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 19: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
W20260709 12:25:28.852440  5114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.856863  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.855330  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:28.859601  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.851835 10602 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:28.863919  5223 raft_consensus.cc:2368] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 18. Current term is 19.
W20260709 12:25:28.864465  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 49900.4us
I20260709 12:25:28.861713 10529 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:28.863391 10660 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:25:28.875358 10586 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:28.876149  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.876060 10586 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:28.866436 10602 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.877964 10602 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:28.889810 10586 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:28.890359  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.891896  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.892050  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.892282  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.892935  5944 raft_consensus.cc:2410] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:25:28.893181  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.894160  5221 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:28.891925  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.895362  5217 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 18: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:28.892365  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:28.896199  5946 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:28.890836  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.902235  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.904114  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:28.904569  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.905403  5218 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:28.905311 10518 consensus_queue.cc:1059] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:28.906560 10660 raft_consensus.cc:3055] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:28.906991 10660 raft_consensus.cc:740] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:28.908046 10660 consensus_queue.cc:260] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:28.909603  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.909978  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.909439 10660 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:28.918082  5190 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:28.907450  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:28.930600  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:28.938375  5946 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:28.944864 10550 consensus_queue.cc:1048] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:28.945758 10548 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:25:28.946293  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 63966.7us
I20260709 12:25:28.949586 10699 consensus_queue.cc:1048] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:28.959316 10695 consensus_queue.cc:1048] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:25:28.950646  5190 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:28.951254 10510 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:28.929435 10698 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:28.953094  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.971632  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.971984  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.972563  5944 raft_consensus.cc:2393] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 20: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:28.972895  5945 raft_consensus.cc:2368] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 18. Current term is 19.
W20260709 12:25:28.957046  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 61635.9us
I20260709 12:25:28.973584  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.973896  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.974361  5943 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:28.975234  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:28.975657  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.975886  5218 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:28.976507  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:28.977133  5947 raft_consensus.cc:3060] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:28.977946  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.960139  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:28.974709  5942 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 18: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:28.978853  5941 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:28.958799 10586 raft_consensus.cc:493] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.959024 10518 raft_consensus.cc:2764] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:25:28.976684  5939 raft_consensus.cc:2410] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:25:28.979899 10586 raft_consensus.cc:515] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:28.980504 10602 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:28.982543 10586 leader_election.cc:290] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:28.959441  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 54603us
I20260709 12:25:28.987962  5947 raft_consensus.cc:2468] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:28.991326  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
W20260709 12:25:29.000892  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 70398.2us
I20260709 12:25:29.001580  5939 raft_consensus.cc:3060] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:28.992187  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:28.993999 10510 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:28.995456 10719 consensus_queue.cc:1048] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:25:29.006181  5042 leader_election.cc:336] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.933s (SENT)
I20260709 12:25:28.996459 10715 consensus_queue.cc:1048] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:29.009390  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:28.958666 10579 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:29.011740 10519 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:29.012560 10519 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:29.015605 10665 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:29.016345  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:29.018865  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:29.019814 10519 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:29.010321  5899 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:29.027555  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:29.028621  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:29.026779  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:29.031929  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:29.026418  5114 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:29.033897  5939 raft_consensus.cc:2410] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:25:29.037202  5074 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:29.037619  5219 raft_consensus.cc:2368] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 20. Current term is 21.
I20260709 12:25:29.037408  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:29.038427  5943 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:29.040995  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:29.030251 10579 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:29.045441  5041 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:29.045450 10697 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:29.029935 10586 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.036689 10519 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:29.074323 10519 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:29.077042 10519 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:29.053849 10579 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:29.078379 10579 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:29.080526  5074 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:25:29.080619  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:29.082507  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:29.083894  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:29.081931  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:29.085805  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:29.086800  5940 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:29.051097 10660 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.048745 10589 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:29.088783 10589 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.046919 10580 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:29.089246  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:29.067277 10602 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:29.040525 10732 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:29.051698  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:29.088251 10545 raft_consensus.cc:2804] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:29.092073  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:29.093056  5943 raft_consensus.cc:1275] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:29.103955  5215 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
W20260709 12:25:29.096300  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 44677.1us
I20260709 12:25:29.093286  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:29.112777  5940 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:29.100603 10732 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.089248 10589 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:29.132596  5947 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:29.135821  5899 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:29.129599 10665 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:25:29.161778 10727 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:29.161916 10524 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:29.162739 10524 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:29.138924 10740 log.cc:927] Time spent T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:25:29.114351 10715 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:29.148983 10730 consensus_queue.cc:1059] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:29.149057  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:29.151089  5042 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.559s (SENT)
I20260709 12:25:29.096493 10519 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:29.144418  5940 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 21.
I20260709 12:25:29.100520 10728 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:25:29.130347 10732 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:29.089824 10586 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:29.162462 10727 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:29.131567 10545 raft_consensus.cc:697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:29.090014 10580 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:29.090464  9927 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:29.181491 10545 consensus_queue.cc:237] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:29.195988  5115 leader_election.cc:400] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 18. Current term is 19.
I20260709 12:25:29.196530  5115 leader_election.cc:403] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:29.197721  5115 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:29.130182 10660 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:29.194762 10593 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:29.218037  9927 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:29.219247 10660 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:29.201361 10602 consensus_queue.cc:1048] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:25:29.201731 10614 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:29.225068  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 74028us
I20260709 12:25:29.249086 10589 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:29.251848 10589 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:29.252621 10524 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:29.255347 10730 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:29.256907 10727 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:29.267905  5042 leader_election.cc:336] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.084s (SENT)
I20260709 12:25:29.268626  5042 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:29.307456 10519 raft_consensus.cc:3055] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:29.308261 10519 raft_consensus.cc:740] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:29.312265 10586 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:29.339171  5190 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.097s (SENT)
I20260709 12:25:29.340150  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:29.342119 10510 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:29.249583 10580 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:29.363569  5900 leader_election.cc:336] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.822s (SENT)
I20260709 12:25:29.364198  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:29.364651  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 96802us
I20260709 12:25:29.365681 10524 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:29.365779 10717 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:25:29.322825 10744 log.cc:927] Time spent T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.104s	user 0.000s	sys 0.003s
I20260709 12:25:29.371827  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:29.371680  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:29.393460  5074 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:25:29.366671 10519 consensus_queue.cc:260] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:29.418172  5190 leader_election.cc:336] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 2.069s (SENT)
I20260709 12:25:29.398645  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:29.433228 10524 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:29.449064  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 251448us
I20260709 12:25:29.450486  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:29.451915  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:29.453397  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 285681us
I20260709 12:25:29.445919 10524 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.458386  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:29.458843  5115 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:29.461272  5215 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:29.457131  5189 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:29.466109  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:29.467839  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:29.475841  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:29.477665  5223 raft_consensus.cc:2393] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:25:29.478473  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 60390.2us
I20260709 12:25:29.480719  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:29.481238  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:29.482299  5067 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:29.481336  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:29.455931 10586 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:29.463402 10524 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:29.486227  5948 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:29.489952  5115 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:29.472821 10728 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:29.501096  5071 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:29.489928 10586 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.512148 10586 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:29.491712 10589 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:29.485306 10593 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:29.483000 10730 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:29.485122 10491 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.526870 10491 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:29.530161 10491 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:29.542982  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 53152.2us
I20260709 12:25:29.550550  5115 leader_election.cc:400] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 20. Current term is 21.
I20260709 12:25:29.551023  5115 leader_election.cc:403] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:29.542655  5215 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:29.543663 10730 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:29.556367  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 66938.3us
W20260709 12:25:29.558686 10746 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:25:29.558545  5190 leader_election.cc:336] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.983s (SENT)
I20260709 12:25:29.559878  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:29.560514 10745 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:25:29.560952 10491 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:29.566402  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.567094  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.567179  5145 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:29.567232  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.567899  5141 raft_consensus.cc:2376] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14. Re-sending same reply.
I20260709 12:25:29.574549  5146 raft_consensus.cc:1240] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:29.574682  5149 raft_consensus.cc:1275] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:25:29.577783  5149 pending_rounds.cc:85] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 7
I20260709 12:25:29.578085  5149 pending_rounds.cc:107] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:25:29.578365  5149 raft_consensus.cc:2889] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:29.564643 10730 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.582419 10730 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:29.582995 10491 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:29.585907  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.587424  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.587726  5144 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:25:29.588071  5144 pending_rounds.cc:85] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 7
I20260709 12:25:29.588085  5149 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:29.588371  5144 pending_rounds.cc:107] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:25:29.589082  5144 raft_consensus.cc:2889] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:29.590257  5142 raft_consensus.cc:2393] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 17: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:29.558370 10713 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:29.590631  5145 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:29.558494 10728 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:25:29.558590 10749 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:29.601954  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.595381  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.594937  5148 raft_consensus.cc:1275] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:25:29.596859 10751 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:25:29.597837  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.596060  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.561744 10518 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:29.606297  5146 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:29.600687  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.607957  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.594746  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.608940  5147 raft_consensus.cc:2368] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 17. Current term is 18.
I20260709 12:25:29.569574 10614 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 18. Current term is 19.
I20260709 12:25:29.603999  5141 raft_consensus.cc:2468] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:29.606303  5142 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:29.607120  5143 raft_consensus.cc:2468] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:29.611413  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.612125  5144 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:29.563340 10732 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:29.617053  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:29.594265 10730 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:29.619401  5943 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:29.619503  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.623234  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.622035  5146 raft_consensus.cc:2376] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12. Re-sending same reply.
I20260709 12:25:29.633206  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.635442  5145 raft_consensus.cc:3060] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:29.602409 10749 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.638835 10749 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:29.605609 10713 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 21 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:29.642670 10713 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:29.646263  5141 raft_consensus.cc:2410] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:25:29.648180  5067 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:29.651823  5150 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:29.655093  5145 raft_consensus.cc:2468] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:29.658066  5071 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:29.659016  5075 raft_consensus.cc:2368] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15.
I20260709 12:25:29.658959  5899 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:29.671643  5948 raft_consensus.cc:1275] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:25:29.672361  5948 pending_rounds.cc:85] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 10
I20260709 12:25:29.672677  5948 pending_rounds.cc:107] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:25:29.672984  5948 raft_consensus.cc:2889] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:29.613360 10518 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:29.664686 10660 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:29.691687  5114 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:29.660538 10697 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:25:29.698294 10660 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:29.698575  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.698907 10524 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:29.699687  5146 raft_consensus.cc:2376] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15. Re-sending same reply.
I20260709 12:25:29.706593 10732 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:29.712598  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:29.721302  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:29.722215  5948 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:25:29.724408 10660 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:29.729492 10756 log.cc:927] Time spent T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:25:29.729724  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.732461  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.732647  5948 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:29.733255  5148 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:29.734339  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.735010  5148 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:29.434368  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:29.737107  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.737946  5148 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 19.
I20260709 12:25:29.739893  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.740831  5142 raft_consensus.cc:2376] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17. Re-sending same reply.
I20260709 12:25:29.697746 10614 raft_consensus.cc:2804] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:25:29.756175  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:29.722277 10757 log.cc:927] Time spent T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:29.749464  5041 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:29.766185  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.766372  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.768261  5147 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:29.773324  5143 raft_consensus.cc:2368] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 18. Current term is 19.
I20260709 12:25:29.714272 10695 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:29.716562 10732 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:29.792563 10732 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:29.778015 10524 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:29.780826 10717 raft_consensus.cc:2804] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:29.795032 10717 raft_consensus.cc:697] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:29.796052 10717 consensus_queue.cc:237] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:29.790498 10692 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:25:29.696973 10730 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:25:29.697926 10586 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:29.817461  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.818387  5143 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:29.848738  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:29.862671  5218 raft_consensus.cc:3060] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:29.782426 10614 raft_consensus.cc:697] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 18 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:29.878260  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:29.873868  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.879031  5143 raft_consensus.cc:3060] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:29.879829  5146 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:29.871713 10750 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.804278 10695 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:29.888605  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:29.865001  4893 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43222 (request call id 144) took 27094 ms (client timeout 15000 ms). Trace:
I20260709 12:25:29.891601  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.903363  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:29.892468  5146 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 13.
I20260709 12:25:29.915058 10695 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:29.932086 10614 consensus_queue.cc:237] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:29.954695  5143 raft_consensus.cc:2468] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:29.890520 10750 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:29.971542  5218 raft_consensus.cc:2468] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:29.967871 10749 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:29.962309 10586 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:29.995180  5142 raft_consensus.cc:2368] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 17. Current term is 18.
I20260709 12:25:29.994074 10732 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:30.005553 10749 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:30.040469  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:30.049418  5069 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.070607 10750 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:30.043569 10728 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 20. Current term is 21.
I20260709 12:25:30.058908 10719 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:30.084149  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 463564us
I20260709 12:25:30.084780 10719 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:30.087114 10719 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:30.090065  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:30.093341  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7f46906b2d49fe83fd7c7729fc9d53" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:30.094017  5900 leader_election.cc:336] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.823s (SENT)
I20260709 12:25:30.094527  5900 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.095363 10661 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:30.096184  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:30.098304  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:30.100318 10762 log.cc:927] Time spent T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.185s	user 0.001s	sys 0.001s
I20260709 12:25:30.100548  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:30.101504  5218 raft_consensus.cc:2393] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 18: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:30.103901  5947 raft_consensus.cc:1275] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:25:30.104267  5947 pending_rounds.cc:85] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 10
I20260709 12:25:30.104547  5947 pending_rounds.cc:107] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:25:30.104779  5947 raft_consensus.cc:2889] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:30.106961  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:30.109349  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:30.110009  5948 raft_consensus.cc:3060] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:25:30.114964  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.119565  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:30.120301  5948 raft_consensus.cc:2468] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:30.126302  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:30.127044  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:30.133666  5940 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:30.128728  5142 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:30.133883  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:30.127764  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:30.135025  5947 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:25:30.135056  5943 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:30.136560  5147 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 21.
I20260709 12:25:30.138969  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.137521  5941 raft_consensus.cc:1275] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:25:30.139252  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.135056  5218 raft_consensus.cc:3060] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:30.140893  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:30.142031  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:30.142848  5219 raft_consensus.cc:3060] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:30.117347 10763 log.cc:927] Time spent T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.202s	user 0.003s	sys 0.000s
I20260709 12:25:30.150604  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.151649  5142 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:30.155481  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52395.1us
I20260709 12:25:30.166956  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:30.167873  5946 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 16: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:30.175346  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:30.176543  5939 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:30.186195  5943 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:30.189581  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:30.192840  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:30.195235  5142 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:30.196600  5218 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:30.163424 10553 consensus_queue.cc:1059] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:25:30.198482  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.199348  5142 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:30.193902 10695 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:30.200362  5947 raft_consensus.cc:2468] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:30.200721 10695 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:30.201151 10695 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:30.200387  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.203066  5142 raft_consensus.cc:2368] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 18. Current term is 19.
I20260709 12:25:30.195593 10767 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:30.203914  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.174178 10732 consensus_queue.cc:1048] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:25:30.206167  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.203743  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.206748  5142 raft_consensus.cc:2376] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17. Re-sending same reply.
I20260709 12:25:30.207324  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:30.207609  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.208014  5947 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
W20260709 12:25:29.892133  4893 rpcz_store.cc:269] 0709 12:25:02.709166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.709495 (+   329us) service_pool.cc:224] Handling call
0709 12:25:29.803248 (+27093753us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:25.475729 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:25.802444 (+326715us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:25.823490 (+ 21046us) write_op.cc:435] Acquired schema lock
0709 12:25:25.823516 (+    26us) tablet.cc:662] Decoding operations
0709 12:25:25.993688 (+170172us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:25.993806 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:25.993842 (+    36us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:25:25.995084 (+  1242us) tablet.cc:701] Row locks acquired
0709 12:25:25.995149 (+    65us) write_op.cc:260] PREPARE: finished
0709 12:25:25.995367 (+   218us) write_op.cc:270] Start()
0709 12:25:25.995566 (+   199us) write_op.cc:276] Timestamp: P: 1783599925995280 usec, L: 0
0709 12:25:25.995588 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:25:26.117937 (+122349us) log.cc:844] Serialized 5543 byte log entry
0709 12:25:26.551366 (+433429us) op_driver.cc:464] REPLICATION: finished
0709 12:25:27.076148 (+524782us) write_op.cc:301] APPLY: starting
0709 12:25:27.076335 (+   187us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:27.414542 (+338207us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:27.446785 (+ 32243us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:28.167253 (+720468us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:28.167651 (+   398us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=28,mrs_lookups=0
0709 12:25:28.167715 (+    64us) write_op.cc:312] APPLY: finished
0709 12:25:28.168230 (+   515us) log.cc:844] Serialized 128 byte log entry
0709 12:25:28.287682 (+119452us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:28.299551 (+ 11869us) write_op.cc:454] Released schema lock
0709 12:25:28.308311 (+  8760us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:28.394919 (+ 86608us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:28.401644 (+  6725us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2422182,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":509533,"apply.run_cpu_time_us":71403,"apply.run_wall_time_us":1308530,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62619,"num_ops":14,"prepare.queue_time_us":518277,"prepare.run_cpu_time_us":22221,"prepare.run_wall_time_us":886766,"raft.queue_time_us":242263,"raft.run_cpu_time_us":44337,"raft.run_wall_time_us":445431,"replication_time_us":555430,"thread_start_us":996256,"threads_started":5,"wal-append.queue_time_us":317360,"wal-append.run_cpu_time_us":17283,"wal-append.run_wall_time_us":2327243}]]}
I20260709 12:25:30.203787 10767 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:30.213624 10767 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:30.219130  5069 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:30.190729 10717 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:25:30.230360  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.231694 10732 consensus_queue.cc:1048] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:30.231709  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.233155  5141 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:30.233316  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:30.235278 10781 log.cc:927] Time spent T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:25:30.236279  5898 leader_election.cc:400] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15.
I20260709 12:25:30.236670  5898 leader_election.cc:403] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:30.239954  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.242842  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:30.243196  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.243507  5145 raft_consensus.cc:2368] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 20. Current term is 21.
I20260709 12:25:30.242082 10545 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:30.244179  5219 raft_consensus.cc:2468] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:30.246563  5141 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:30.247509  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.248970  5142 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:30.249847  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.250540  5142 raft_consensus.cc:2393] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 18: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:30.250444 10747 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 14. Current term is 15.
I20260709 12:25:30.251956  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.252744 10761 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:30.253376 10761 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.253492  5145 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:25:30.254629  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.255079  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.256422  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.257022  5145 raft_consensus.cc:2393] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 18: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:30.258172 10728 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.258288  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.258942  5141 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:30.258718 10728 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:30.260624 10542 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:30.261003 10728 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:30.262769  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.263036 10586 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:30.263928 10586 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:30.264000  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.264671 10586 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.264827  5148 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:30.265264  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.265190 10586 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:30.266775 10661 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.267383 10661 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:30.246395 10545 consensus_queue.cc:1243] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:30.271539  5142 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:25:30.253799 10761 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:30.277488  5115 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.278620  5947 pending_rounds.cc:85] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 10
I20260709 12:25:30.279044  5947 pending_rounds.cc:107] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:25:30.279277  5947 raft_consensus.cc:2889] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:30.257350  5190 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.280428  5115 leader_election.cc:400] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 18. Current term is 19.
I20260709 12:25:30.281845  5115 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.278645  5141 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:30.283531  5042 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.283659  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:30.285384  5900 leader_election.cc:336] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.934s (SENT)
I20260709 12:25:30.285915  5900 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.287761  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.289108  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:30.289963 10774 raft_consensus.cc:2764] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:25:30.290045 10749 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:30.291157 10510 raft_consensus.cc:2764] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:25:30.285202 10730 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:30.285004 10545 raft_consensus.cc:2764] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 12:25:30.301141  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.292683 10728 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:30.291123 10732 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:30.306836  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:30.309728  5218 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:30.310355  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.296113 10578 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:30.312671  5115 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:30.315186  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 57913.3us
I20260709 12:25:30.315959  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.312829  5189 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:30.309244 10749 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:30.314143 10774 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:30.323563 10661 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:30.300201 10518 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:30.327107 10695 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:30.331821  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.335858  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.336010  5149 raft_consensus.cc:2468] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:30.329393 10767 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:30.339349  5900 leader_election.cc:336] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Timed out: RequestConsensusVote RPC to 127.1.123.67:46855 timed out after 1.750s (SENT)
W20260709 12:25:30.341130  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 57706.8us
I20260709 12:25:30.329762 10750 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:30.342183 10550 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:30.356408 10697 consensus_queue.cc:1048] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:30.352810 10510 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:30.338464 10695 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:30.358209 10550 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:30.358759 10614 consensus_queue.cc:1048] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:30.360579  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.378284 10719 raft_consensus.cc:3055] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:30.378842 10719 raft_consensus.cc:740] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:30.380074 10719 consensus_queue.cc:260] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:30.374014  5113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.374853  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:30.377914 10550 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:30.373693  5218 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:30.348718  5115 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:30.399930  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 51365.8us
W20260709 12:25:30.394258  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44389.2us
I20260709 12:25:30.401926  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.401942  5115 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.349079 10586 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:30.345821 10713 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:30.404484 10593 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:30.407714 10730 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:25:30.408391  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 72650.5us
I20260709 12:25:30.413669  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.394016 10553 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:30.411779  5115 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:30.420374  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.425052  5068 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:30.410233 10713 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.428735  5190 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.421885  5115 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:30.413055 10593 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.430493  5223 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:25:30.432981  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.433780 10769 raft_consensus.cc:2804] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:30.436574 10769 raft_consensus.cc:697] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 19 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:30.437487 10769 consensus_queue.cc:237] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:30.438146  5068 raft_consensus.cc:3060] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.422739 10774 raft_consensus.cc:2804] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:30.440558 10774 raft_consensus.cc:697] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 18 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:25:30.440991 10760 log.cc:927] Time spent T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.542s	user 0.004s	sys 0.001s
I20260709 12:25:30.441598 10774 consensus_queue.cc:237] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:30.443225 10578 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.443774 10578 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:30.444252  5070 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 17. Current term is 18. Ops: []
W20260709 12:25:30.434394 10786 log.cc:927] Time spent T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:25:30.433799 10553 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:30.451560 10553 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:30.451936 10553 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:30.454216 10767 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:30.454341 10593 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:30.418579 10586 consensus_queue.cc:1048] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:25:30.463122 10519 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:30.455660 10713 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:30.475122  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 46469.1us
I20260709 12:25:30.417981 10694 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:30.496946  5900 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.465410 10550 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:30.502648  5218 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:25:30.420475 10719 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.504616  5144 raft_consensus.cc:1275] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:30.509173  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:30.429606 10545 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:30.517066 10593 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:30.519044  5224 raft_consensus.cc:1240] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:30.519268  5148 raft_consensus.cc:2368] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17.
I20260709 12:25:30.520475  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:30.522095  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:30.522969  5943 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:30.523645 10518 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:30.527083 10578 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:30.527807 10767 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:30.528354 10767 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:30.505348 10694 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:30.532804  5940 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:30.535820  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:30.536921  5943 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:30.517207 10545 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:30.505113 10699 raft_consensus.cc:2804] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:30.544826  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:30.540753  5900 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.541975 10593 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:30.542110 10550 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:30.547189  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 115824us
I20260709 12:25:30.549405  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.551220  5148 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:30.553987  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.556861  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.558640  5148 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:25:30.560825 10713 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:30.562058  5115 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:30.562574  5218 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:30.564531 10730 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:30.565081  5115 leader_election.cc:304] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:30.566458  5115 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:30.566409 10730 raft_consensus.cc:2749] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:30.567291 10713 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:30.567590 10730 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:30.568295  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:30.570662  5943 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:30.567870  5115 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:30.577474 10730 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:30.580065  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.580839  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:30.581779  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:30.586266  5148 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:25:30.588209  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.589205  5144 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:30.589747  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50129.1us
I20260709 12:25:30.589219  5940 raft_consensus.cc:1275] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:30.590981  5068 raft_consensus.cc:1275] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:30.591691  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:30.592931  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 52275.6us
I20260709 12:25:30.594101  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.595333  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.596340  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:30.598907  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f30d7c9f3244999f248bc1aed226c7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.600395  5142 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:25:30.601495  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:30.602010  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.603753  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.604302  5142 raft_consensus.cc:2376] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19. Re-sending same reply.
I20260709 12:25:30.605167  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.605618  5142 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 20.
I20260709 12:25:30.607831  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.608860  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.609901  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:30.610877  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.611907  5943 raft_consensus.cc:2468] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:30.613804  5189 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:30.538892 10694 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:30.618235  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.602773 10614 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:30.623281  5149 raft_consensus.cc:2376] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18. Re-sending same reply.
I20260709 12:25:30.623535  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.624173 10699 raft_consensus.cc:697] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:30.626461  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:30.627144  5072 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:30.631894  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:30.630026  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 46797.8us
I20260709 12:25:30.634290  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.598619 10750 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:30.637149  5072 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:30.543365 10695 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 12:25:30.644708  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:30.616252 10589 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.646590  5141 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:25:30.621781 10747 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:30.625223 10699 consensus_queue.cc:237] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:30.648486  5069 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:25:30.628572 10518 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:30.651021  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:30.658696  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:30.616573 10578 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:30.665140  5068 raft_consensus.cc:1275] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:30.633870 10602 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:30.621079 10550 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:25:30.616910 10553 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:30.622563 10660 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:25:30.690999  5069 raft_consensus.cc:2468] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:30.681674  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.646760 10614 raft_consensus.cc:2804] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:30.676733 10518 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:30.728557  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 92541.9us
I20260709 12:25:30.647074 10580 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:25:30.742631  5067 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:30.746758  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:30.662235 10589 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:30.781328 10778 log.cc:927] Time spent T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.087s	user 0.000s	sys 0.005s
I20260709 12:25:30.781960 10602 raft_consensus.cc:697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:30.782191 10715 raft_consensus.cc:3055] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:25:30.784866  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 103339us
I20260709 12:25:30.697202 10767 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:30.701475 10614 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.743808 10518 raft_consensus.cc:3060] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:30.692086 10491 consensus_queue.cc:1059] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
W20260709 12:25:30.798533 10761 consensus_meta.cc:287] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.526s	user 0.013s	sys 0.000s
I20260709 12:25:30.693061 10791 consensus_queue.cc:1048] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 12:25:30.802551  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:30.794092 10797 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.179s	user 0.002s	sys 0.000s
W20260709 12:25:30.802397 10789 log.cc:927] Time spent T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:25:30.808969  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 61887.3us
I20260709 12:25:30.834810  5042 leader_election.cc:400] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17.
I20260709 12:25:30.835184  5042 leader_election.cc:403] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:30.811450 10553 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:30.807451 10614 raft_consensus.cc:3060] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:30.851256 10790 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 16. Current term is 17.
W20260709 12:25:30.813988  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 78078us
I20260709 12:25:30.853180  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:30.854686  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.839753  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.828244 10602 consensus_queue.cc:237] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:30.830863  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:30.799427 10715 raft_consensus.cc:740] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:30.866178  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:30.692212 10545 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:30.840798 10802 raft_consensus.cc:3055] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:25:30.843015 10792 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.207s	user 0.002s	sys 0.000s
W20260709 12:25:30.843873 10788 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:25:30.778734 10747 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:25:30.856184  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.865079  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.812829 10748 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:30.853434 10589 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:30.882185  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 217519us
I20260709 12:25:30.904250  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.874675 10790 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:30.875555 10542 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:30.912215 10542 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:30.912691 10542 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:30.873512 10550 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:30.821774 10761 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:30.875315 10802 raft_consensus.cc:740] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:30.860327 10732 consensus_queue.cc:1048] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:25:30.899781 10661 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:30.933684 10761 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:30.934042 10794 log.cc:927] Time spent T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:25:30.934476  5147 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:30.935156 10749 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.935647 10749 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:30.938727  5218 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:25:30.902999 10665 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:30.950673 10812 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:30.920189  5144 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:30.871528 10800 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:25:30.953235  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:30.963656  5939 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:30.956919  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:30.909435 10767 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:30.964860  5142 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:30.969673 10542 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:30.964506  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:30.913897 10578 raft_consensus.cc:697] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
W20260709 12:25:30.993044  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 66310.1us
I20260709 12:25:30.906229 10491 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 12:25:30.999848  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:31.001085  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:31.000478  5218 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 18
W20260709 12:25:30.993520  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 97325us
I20260709 12:25:31.000475  5114 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:31.012710  5224 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:31.021200  5939 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:31.022882  5114 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:31.035247  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 236598us
I20260709 12:25:31.037228  5900 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:30.989794 10660 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:25:31.040865  5900 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:31.004477 10580 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:31.035375  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:31.047021  5218 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:30.992518 10665 consensus_queue.cc:1048] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:25:31.014952 10769 raft_consensus.cc:2804] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:31.114055 10769 raft_consensus.cc:697] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 17 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:31.115088 10769 consensus_queue.cc:237] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:31.048481 10519 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:31.012173 10589 consensus_queue.cc:1048] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:25:31.133862 10815 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.141s
I20260709 12:25:31.135699 10660 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:31.136312 10660 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.136688 10660 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:31.145733 10801 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.103s
I20260709 12:25:31.161011 10816 consensus_queue.cc:1059] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.143s
I20260709 12:25:31.162856 10816 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:31.163376 10816 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:31.164206 10519 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:31.164907 10519 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:31.165642 10775 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.198s
I20260709 12:25:31.167901 10755 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.168802 10755 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:31.170511  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:31.171559 10755 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:31.175813 10749 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:31.178174 10694 consensus_meta.cc:287] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.002s
W20260709 12:25:31.178447 10788 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:25:31.124903 10578 consensus_queue.cc:237] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:31.165234  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33342 (request call id 1114) took 1109 ms. Trace:
I20260709 12:25:31.168149 10579 raft_consensus.cc:3055] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:31.181579  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:31.191871  5147 rpcz_store.cc:276] 0709 12:25:30.056010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.233003 (+176993us) service_pool.cc:224] Handling call
0709 12:25:31.165046 (+932043us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b540019f854. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:31.165170 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1610596096}
I20260709 12:25:31.147699 10790 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:31.164489 10816 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:31.147490 10747 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:31.194419 10579 raft_consensus.cc:740] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:31.224733  5074 raft_consensus.cc:1240] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:31.196525 10621 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:31.236713  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 183740us
I20260709 12:25:31.250471 10822 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.251533 10822 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:31.263290  5074 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:31.264154  5074 raft_consensus.cc:606] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:31.268466 10791 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:31.269381 10728 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:31.239353  4891 catalog_manager.cc:5697] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 15, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 15 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:31.270042 10728 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:31.274700  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:31.271986  4891 catalog_manager.cc:5697] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 17, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 17 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:31.238961 10790 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:31.247272 10816 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:31.278877 10660 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:31.279378 10728 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:31.306378 10819 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:31.306958 10819 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:31.308826 10761 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:31.309444 10761 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:31.292734 10719 consensus_meta.cc:287] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.771s	user 0.009s	sys 0.001s
I20260709 12:25:31.195748 10694 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:31.244273 10621 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:31.295405  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:31.295909 10831 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:31.296218  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:31.318096  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:31.297472  5074 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
W20260709 12:25:31.318368  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 277580us
W20260709 12:25:31.318385  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 51721.9us
I20260709 12:25:31.314862  5224 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:25:31.325639  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:31.327298  5900 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:31.298208 10801 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:31.323783  5074 raft_consensus.cc:606] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:31.330534  5900 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:31.331224  5215 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:31.333024 10825 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:31.329990 10747 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:31.294345 10579 consensus_queue.cc:260] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:31.336284 10815 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:31.333189 10824 log.cc:927] Time spent T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:25:31.294999 10614 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:31.341185  5900 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:31.345981 10695 raft_consensus.cc:993] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: : Instructing follower 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:31.346668 10695 raft_consensus.cc:1081] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 LEADER]: Signalling peer 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:31.326809 10831 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:31.355504  5223 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:31.356971 10825 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:31.251416 10715 consensus_queue.cc:260] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:31.358011 10747 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:31.358351 10833 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:31.358896 10833 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:31.362704 10614 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:31.339779 10801 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:31.448491 10831 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:31.395027  4891 catalog_manager.cc:5697] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:31.452412 10819 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:31.463419 10519 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:31.474560  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:31.460757  4891 catalog_manager.cc:5697] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 13 to 16, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 16 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:31.477674  4891 catalog_manager.cc:5697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 16 to 17, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 17 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:31.485741 10660 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:31.373657 10790 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:31.479671  4891 catalog_manager.cc:5697] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:31.498504 10769 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:31.510246 10769 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:31.512931 10769 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:31.514196 10833 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:31.523741 10621 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:31.524576  5223 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:31.525876  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:31.462309 10834 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:25:31.528299  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:31.523794  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:31.530055  5219 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:31.530500 10802 consensus_queue.cc:260] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:31.531754 10802 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:31.532420 10810 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:31.532850 10810 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:31.533190 10810 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:31.497108 10822 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:31.537283  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:31.538138  5068 raft_consensus.cc:2393] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:31.539042  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:31.539588  5939 raft_consensus.cc:1275] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
W20260709 12:25:31.540971 10518 consensus_meta.cc:287] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.719s	user 0.012s	sys 0.002s
I20260709 12:25:31.541453 10518 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:31.542119  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:31.543725 10518 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:31.544565  5943 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 17: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:31.548530 10730 raft_consensus.cc:493] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:31.549160 10730 raft_consensus.cc:515] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:31.553032  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:31.554833  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:31.555667  5948 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:31.557319  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:31.555339  5941 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:25:31.557914  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 149835us
I20260709 12:25:31.558120  5071 raft_consensus.cc:2393] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 17: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:31.563570  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:31.571868  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:31.573108  5943 raft_consensus.cc:2368] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 20. Current term is 21.
I20260709 12:25:31.574872  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:31.575892  5141 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:31.576488  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:31.553783 10798 log.cc:927] Time spent T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.503s	user 0.006s	sys 0.000s
I20260709 12:25:31.559017 10728 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:31.578367  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39878 (request call id 1425) took 1097 ms. Trace:
I20260709 12:25:31.577216  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:31.579906  5149 raft_consensus.cc:2393] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 16: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:31.578836 10728 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:31.584044  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:31.578856  5946 rpcz_store.cc:276] 0709 12:25:30.481276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.481415 (+   139us) service_pool.cc:224] Handling call
0709 12:25:30.520598 (+ 39183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.531392 (+ 10794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.531445 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.540303 (+  8858us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.625002 (+ 84699us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b18000e0b60. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db0d1 00007fd8a84f6a0f
0709 12:25:30.705907 (+ 80905us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:30.705965 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.706085 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:31.370429 (+664344us) spinlock_profiling.cc:243] Waited 9.23 ms on lock 0x7b2400037178. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a52d49c4 00007fd8a52d2acb 00007fd8a548ddc9 00007fd8a548da55 00007fd8a548d09d 00007fd8a80bf10e 00007fd8a80d688c 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221
0709 12:25:31.578080 (+207651us) raft_consensus.cc:1685] finished
0709 12:25:31.578101 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.578254 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8322,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":6208,"spinlock_wait_cycles":88049792,"thread_start_us":80641,"threads_started":1,"wal-append.queue_time_us":253183}
I20260709 12:25:31.601020  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:31.606074  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:31.607052  5147 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
W20260709 12:25:31.609252  5897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.613302  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:31.603984  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:31.615885  5217 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:25:31.617331  5899 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:31.619532  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:31.620829  5070 raft_consensus.cc:1240] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:31.621174 10728 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:31.622100  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:31.628649  5899 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:31.634130  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:31.638933  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:31.639171  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:31.639820  5149 raft_consensus.cc:2393] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 20: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:31.642014  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:31.645736  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:31.646452  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:31.649180  5899 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:31.651425  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:31.640368 10518 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:31.644616 10822 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:31.661679 10621 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:31.705936 10621 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:31.668634 10553 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:31.664052 10748 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:31.716665  5217 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 21.
I20260709 12:25:31.717864 10621 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:31.648988 10816 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:25:31.720844  5899 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:31.757083  5070 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:25:31.811096  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.746896 10822 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:31.813407  5943 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:31.761247 10518 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:31.619585  4891 catalog_manager.cc:5697] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 10 to 11, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 11 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:31.910539  5944 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:25:31.863536 10834 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 12:25:31.906330 10815 consensus_meta.cc:287] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.003s
I20260709 12:25:31.789592 10728 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:32.012437  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 103438us
I20260709 12:25:31.904326 10828 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:31.962169 10518 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 21 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:31.960659 10826 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:32.052182 10826 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:31.988162 10553 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:31.906939 10822 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:32.016227 10815 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:32.095486 10845 log.cc:927] Time spent T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 12:25:32.097607 10846 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 12:25:32.107834 10856 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.108577 10856 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.111410 10856 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:32.112095 10856 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.114107 10826 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.071067  4891 catalog_manager.cc:5697] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 9 to 14, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 14 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:32.118373  4891 catalog_manager.cc:5697] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:32.112609 10856 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:32.016631 10728 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:32.124286  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 133181us
I20260709 12:25:32.147478  5947 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:32.150784 10518 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:32.157811  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:32.067129 10553 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.044190 10828 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:32.320067 10769 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:32.163589 10815 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:32.192389 10614 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.321058 10769 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.342456 10815 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.325362 10843 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.058367 10831 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 12:25:32.362604  4891 catalog_manager.cc:5697] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 8 to 13, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 13 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:32.381101 10550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.011s	sys 0.062s
I20260709 12:25:32.367862 10815 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:32.387410  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 353117us
I20260709 12:25:32.357250 10614 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:32.392688  5189 leader_election.cc:400] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 20. Current term is 21.
I20260709 12:25:32.393042  5189 leader_election.cc:403] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:32.393325 10825 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:32.395857 10825 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.398828 10825 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:32.399024  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.384811 10694 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:32.387650 10550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.011s	sys 0.063s
I20260709 12:25:32.405757 10519 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.373358 10843 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.391245 10728 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.449054  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:32.452551  5898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.427013 10822 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:32.421916 10868 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac for earlier term 20. Current term is 21.
I20260709 12:25:32.449146 10813 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.455044 10856 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.441916 10809 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:25:32.448953 10519 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.457170 10518 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.445217 10866 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:25:32.466216 10553 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:32.497027  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.467621 10822 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.485905 10828 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:32.469484 10801 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.429052  4891 catalog_manager.cc:5697] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:32.498528 10518 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.518805 10875 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.521678 10875 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.525068 10875 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.526886 10875 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:25:32.528088 10621 consensus_meta.cc:287] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 0.783s	user 0.010s	sys 0.000s
I20260709 12:25:32.504494 10728 consensus_queue.cc:1048] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:25:32.529345  4891 catalog_manager.cc:5697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 9 to 17, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 17 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:32.536620  4891 catalog_manager.cc:5697] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 8 to 14, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 14 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:32.538269 10865 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:25:32.494127 10813 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.541296 10813 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:32.517620 10769 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.543020 10621 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.539147  4891 catalog_manager.cc:5697] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 10 to 20. New cstate: current_term: 20 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:32.547065 10621 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:32.548604 10810 consensus_meta.cc:287] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Time spent flushing consensus metadata: real 1.015s	user 0.012s	sys 0.001s
I20260709 12:25:32.550925 10519 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.551010 10810 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:32.553333  5068 raft_consensus.cc:1240] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:32.505393 10822 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.553193 10828 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.558241 10828 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.561053 10828 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:32.549916  4891 catalog_manager.cc:5697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 14, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 14 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:32.570900 10843 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:32.574353 10864 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.574982 10864 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.575331  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:32.579658  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 76612us
I20260709 12:25:32.555696 10519 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:32.587316  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:32.588088  5948 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:32.556324 10614 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:32.594693  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:32.600601  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:32.601053  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.601367  5941 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 20.
I20260709 12:25:32.598542  5189 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:32.604809  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.607332  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.590173 10717 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.609721  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.611876  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.579993 10864 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:32.601768 10621 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:32.596635 10804 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
W20260709 12:25:32.614975 10817 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 1.564s	user 0.006s	sys 0.010s
I20260709 12:25:32.616777  5189 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:32.621841 10761 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.622807 10518 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:32.623550 10861 log.cc:927] Time spent T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:25:32.625494 10747 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.627038  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39866 (request call id 1130) took 1089 ms. Trace:
I20260709 12:25:32.628010 10822 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:32.647068  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.591730 10519 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.631875 10542 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:32.633066  5941 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:39870
W20260709 12:25:32.631944 10802 consensus_meta.cc:287] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Time spent flushing consensus metadata: real 1.100s	user 0.011s	sys 0.003s
I20260709 12:25:32.611743 10835 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:32.628352  5074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56034 (request call id 27) took 1331 ms. Trace:
I20260709 12:25:32.645606  5947 rpcz_store.cc:276] 0709 12:25:31.537908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.538714 (+   806us) service_pool.cc:224] Handling call
0709 12:25:32.626830 (+1088116us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b540000b754. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:32.626945 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1397649664}
I20260709 12:25:32.651590  5941 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:32.662791  5941 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:32.660933  5074 rpcz_store.cc:276] 0709 12:25:31.297213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.297349 (+   136us) service_pool.cc:224] Handling call
0709 12:25:32.628146 (+1330797us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b4c00334f80. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6807a20 00007fd8a6818a29 00007fd8a67de644 00007fd8a6805f2d 00007fd8a6805b93 00007fd8a8535c92 00007fd8a853584a 00007fd8ad50360a 00007fd8a6a897d4 00007fd8a6a89762 00007fd8a6a89691
0709 12:25:32.628259 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99956992}
I20260709 12:25:32.653026  5073 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:25:32.640151  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 131947us
I20260709 12:25:32.665880  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:32.667559  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:32.668805  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:32.675096  5115 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:32.677845 10614 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:32.678294  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.679708 10876 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:32.682543 10579 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:32.684801 10769 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:32.677177  5115 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:32.689623  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:32.692384  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.695159  5948 raft_consensus.cc:1275] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:32.695518  5947 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:32.697701  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:32.698659  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.699524  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.701128  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.702559  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.704784  5222 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 15: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:32.649015 10761 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:32.633577 10717 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:32.652938 10864 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:32.641110 10621 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.743232  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:32.744642 10621 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.720237  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.745199  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:32.713891 10769 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:32.745858  5142 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:32.745636  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:32.756035  5142 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 21.
I20260709 12:25:32.761794 10621 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:32.762503 10621 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.764729 10621 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:32.765374  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:32.641034 10804 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.726083  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:32.766664  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:32.761879  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.767803 10804 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.664019 10835 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.768483 10835 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:32.662585 10842 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.732060 10519 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.734956 10695 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:32.735010  5222 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:32.737849 10810 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:32.739683 10717 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:32.565367  4891 catalog_manager.cc:5697] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 14, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 14 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:32.653303 10747 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:32.741711  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:32.713491 10876 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:32.790305  5948 raft_consensus.cc:1275] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:32.742538  5072 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 12.
I20260709 12:25:32.768561 10831 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:32.729846  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:32.800521  5147 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:32.772318  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.803342  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.803575  5219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:25:32.804241  5219 raft_consensus.cc:606] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:25:32.804620  5219 raft_consensus.cc:614] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:32.730384 10802 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:32.775221  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:32.778029  5146 raft_consensus.cc:1275] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:32.776701  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39866 (request call id 1127) took 1924 ms. Trace:
I20260709 12:25:32.686547 10873 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:32.787737  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:32.783694  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:32.730582 10822 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.728300 10864 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.766090  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:32.826207  5142 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:32.769891 10748 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.814719  5900 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:32.811187  5942 rpcz_store.cc:276] 0709 12:25:30.851973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.852144 (+   171us) service_pool.cc:224] Handling call
0709 12:25:30.852546 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.852903 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.852933 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.853434 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.896559 (+ 43125us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:30.896608 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.896688 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.598925 (+1702237us) spinlock_profiling.cc:243] Waited 50.9 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:32.727346 (+128421us) raft_consensus.cc:1685] finished
0709 12:25:32.727371 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.776623 (+ 49252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1062,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":117047680,"thread_start_us":42520,"threads_started":1,"wal-append.queue_time_us":139270}
I20260709 12:25:32.778591 10804 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.725296 10761 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:32.789126  5115 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:32.824362 10822 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.830448  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:32.835044  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:32.809943 10810 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.844480  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:32.832371  5941 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.855444  5147 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:32.746467 10769 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.814277 10873 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.852839  5142 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:32.830317 10864 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:32.824652 10816 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:32.867877  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.737407 10815 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:32.875751 10815 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:32.876678  5940 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:32.814462 10878 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:25:32.813832 10875 consensus_queue.cc:1048] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:25:32.780551 10842 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.841804 10801 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:32.798497 10695 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.842263 10518 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:32.885281  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.890758  5941 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:32.883505  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.861147 10876 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:32.902753  5215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 1112) took 2465 ms. Trace:
I20260709 12:25:32.909262  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.909699  5073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:32.915861  5073 raft_consensus.cc:606] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:25:32.918010  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.919191  5073 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 18: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:32.913388  5215 rpcz_store.cc:276] 0709 12:25:30.426647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.426783 (+   136us) service_pool.cc:224] Handling call
0709 12:25:30.427126 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.331788 (+904662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.892130 (+1560342us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54001437d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:32.892452 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2078758272}
I20260709 12:25:32.864076 10873 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:32.926368  5216 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:32.855160 10804 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:32.930543  5071 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:32.900314 10875 consensus_queue.cc:1243] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:25:32.856982  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 67948.8us
I20260709 12:25:32.863706 10810 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:32.939481  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 123911us
I20260709 12:25:32.939765  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.841816 10748 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:32.931084 10839 log.cc:927] Time spent T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 1.470s	user 0.011s	sys 0.013s
W20260709 12:25:32.931320 10833 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.014s	sys 0.012s
I20260709 12:25:32.926106 10870 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.926976 10802 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.945879  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:32.911218 10518 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:32.950316 10833 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.015s	sys 0.012s
I20260709 12:25:32.961333  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.957616 10890 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 12:25:32.957720 10870 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:32.966385 10755 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:32.968806  5145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 1150) took 1396 ms. Trace:
I20260709 12:25:32.983966  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:32.984611  5076 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:32.981721  5145 rpcz_store.cc:276] 0709 12:25:31.572200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.572348 (+   148us) service_pool.cc:224] Handling call
0709 12:25:31.582466 (+ 10118us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.602524 (+ 20058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.602558 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.186146 (+583588us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b1800317980. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a541c2a6 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a
0709 12:25:32.743564 (+557418us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b2c00193f58. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:25:32.744277 (+   713us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.842175 (+ 97898us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:32.842220 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.842298 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.968469 (+126171us) raft_consensus.cc:1685] finished
0709 12:25:32.968500 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.968680 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5334,"raft.queue_time_us":194,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":20993,"spinlock_wait_cycles":35990144,"thread_start_us":97457,"threads_started":1,"wal-append.queue_time_us":194792}
I20260709 12:25:32.990075  5224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57002 (request call id 1121) took 1659 ms. Trace:
W20260709 12:25:32.990581 10885 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:25:32.991230 10884 log.cc:927] Time spent T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:32.991550 10755 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:32.993040 10761 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:32.992791 10755 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:32.979182 10550 consensus_queue.cc:1048] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:25:32.960675 10802 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:32.998642 10870 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:32.990677 10695 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:32.999734 10802 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:32.881810  4891 catalog_manager.cc:5697] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 12 to 15, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 15 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:33.002651 10518 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.002079  4891 catalog_manager.cc:5697] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 10 to 12, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 12 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:33.003631 10842 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:32.960491 10876 consensus_queue.cc:1048] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:25:33.005549 10730 leader_election.cc:290] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:33.006685 10747 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:32.997641 10822 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.027204  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:33.028939  5898 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:33.039237  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.040129  5148 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:33.040330 10810 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:33.042047  5114 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:33.042688  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5deada35125f4c008544b2ff246b039c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:32.979938 10856 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:32.990702  5224 rpcz_store.cc:276] 0709 12:25:31.331176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.331363 (+   187us) service_pool.cc:224] Handling call
0709 12:25:31.331730 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.342847 (+ 11117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.342875 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.343428 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.404489 (+ 61061us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:31.417598 (+ 13109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.417733 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.807713 (+1389980us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:32.989726 (+182013us) raft_consensus.cc:1685] finished
0709 12:25:32.989755 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.990000 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10684,"raft.queue_time_us":15384,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":126,"spinlock_wait_cycles":86794496,"thread_start_us":60746,"threads_started":1,"wal-append.queue_time_us":73472}
I20260709 12:25:33.044633  5076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:33.045652  5076 raft_consensus.cc:606] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:33.046031  5076 raft_consensus.cc:614] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:33.046744  5114 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:33.052079  5042 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.054740  5114 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:33.057781  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 97057.4us
I20260709 12:25:33.058408  5190 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:33.065234 10875 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.065958  5947 raft_consensus.cc:1217] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 13.8->[14.9-18.11]   Dedup: 14.9->[16.10-18.11]
I20260709 12:25:33.066175 10875 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:33.066655 10578 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.067314 10578 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:33.066752  5947 pending_rounds.cc:85] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 9
I20260709 12:25:33.070385  5947 pending_rounds.cc:107] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:25:33.070784  5947 raft_consensus.cc:2889] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:33.070606 10901 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:33.072705  5115 leader_election.cc:304] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:33.075201  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:33.069703 10578 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:33.053915 10870 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:33.044487 10518 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:33.058434 10728 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:33.061043 10878 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:33.057952 10871 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:33.073271 10719 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:33.058154 10747 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:25:33.111789 10890 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.085342 10879 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:33.082597 10602 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:33.062137 10873 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:33.125466  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 172910us
I20260709 12:25:33.110165 10878 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.114809 10749 raft_consensus.cc:2749] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:33.003831  4891 catalog_manager.cc:5697] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 12 to 13, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 13 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:33.061374 10856 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.122197 10614 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:33.122159 10879 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.123699 10728 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:33.128983 10810 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.082356 10901 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:33.144403  5042 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.135969 10878 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.128458 10719 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.138154 10769 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:33.154332  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 73108.5us
I20260709 12:25:33.145282 10804 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:33.155983  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.145001 10614 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:33.164047 10614 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:33.165906 10697 consensus_queue.cc:1059] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:33.125041 10890 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.167119 10901 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:33.170081  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.145241 10879 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:33.183058  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:33.191964  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 119384us
I20260709 12:25:33.151971 10810 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.154902 10719 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.195168  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.148892 10728 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:33.199576  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.194794  5042 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.212071  5941 raft_consensus.cc:2410] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:25:33.200731  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:33.214270  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.215106  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:33.216183  5222 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:33.201795  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.205956  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:33.203033  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.221802 10761 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:33.192444 10890 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:33.222102  5147 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:33.195248 10900 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.205847  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.223243  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.221834  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.224648  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.225033  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.222030  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.189882 10908 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:33.212639 10717 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:33.196105 10856 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.241036  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.242524  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.199687  4891 catalog_manager.cc:5697] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:33.242852 10719 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:33.243825  4891 catalog_manager.cc:5697] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 13 to 15, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 15 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:33.244769 10728 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.245083  5144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33342 (request call id 1143) took 1694 ms. Trace:
I20260709 12:25:33.246122 10804 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:33.232257 10749 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:33.247305 10749 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:33.245687  5144 rpcz_store.cc:276] 0709 12:25:31.551046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.551287 (+   241us) service_pool.cc:224] Handling call
0709 12:25:31.551822 (+   535us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:31.552395 (+   573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.552433 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:31.553223 (+   790us) log.cc:844] Serialized 52 byte log entry
0709 12:25:31.753147 (+199924us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:32.123866 (+370719us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.124454 (+   588us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.124688 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.124771 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.751208 (+626437us) spinlock_profiling.cc:243] Waited 7.66 ms on lock 0x7b480004c758. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ad464aa1 00007fd8ad45bb37 00007fd8ad45b9ca 00007fd8ad55326f 00007fd8ad552f37 00007fd8ad56755c 00007fd8ad567508 00007fd8ad567488 00007fd8ad567443 00007fd8ad5666ec 00007fd8ac490a96
0709 12:25:33.241855 (+490647us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b18002e0d80. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a544e3f3 00007fd8a54507c1 00007fd8a5450779 00007fd8a5450709 00007fd8a54506d1 00007fd8a544f9cd 0000000000388634 0000000000388479 00007fd8a543183a
0709 12:25:33.244856 (+  3001us) raft_consensus.cc:1685] finished
0709 12:25:33.244879 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.245021 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3409,"prepare.queue_time_us":112150,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":34213,"raft.queue_time_us":7085,"raft.run_cpu_time_us":8470,"raft.run_wall_time_us":653108,"spinlock_wait_cycles":25936640,"thread_start_us":499499,"threads_started":2,"wal-append.queue_time_us":310748}
I20260709 12:25:33.201304 10810 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:33.249557  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a42608cf874aa6943aaf8e349714b9" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.231401 10900 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:33.250921  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:33.251557  5076 raft_consensus.cc:3055] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:33.251849  5076 raft_consensus.cc:740] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:33.230094 10871 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:33.207564 10835 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.252664  5076 consensus_queue.cc:260] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.252882 10871 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:33.254426 10850 log.cc:927] Time spent T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 1.291s	user 0.010s	sys 0.000s
I20260709 12:25:33.254913 10835 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.255671  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1679) took 1669 ms. Trace:
I20260709 12:25:33.257388  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.258706  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:33.245036  5141 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:33.256265  5150 rpcz_store.cc:276] 0709 12:25:31.586197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.586346 (+   149us) service_pool.cc:224] Handling call
0709 12:25:31.586786 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.587298 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.587340 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.622108 (+ 34768us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.731516 (+109408us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:31.731560 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.731631 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.141297 (+1409666us) spinlock_profiling.cc:243] Waited 7.76 ms on lock 0x7b480004c758. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ad464aa1 00007fd8ad45bb37 00007fd8ad45b9ca 00007fd8ad55326f 00007fd8ad552f37 00007fd8ad56755c 00007fd8ad567508 00007fd8ad567488 00007fd8ad567443 00007fd8ad5666ec 00007fd8ac490a96
0709 12:25:33.241503 (+100206us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:33.255466 (+ 13963us) raft_consensus.cc:1685] finished
0709 12:25:33.255486 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.255606 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9938,"raft.queue_time_us":5652,"raft.run_cpu_time_us":25004,"raft.run_wall_time_us":1539854,"spinlock_wait_cycles":42541696,"thread_start_us":109079,"threads_started":1,"wal-append.queue_time_us":162026}
I20260709 12:25:33.260757  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.262455  5220 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:33.236545  5042 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:33.253196 10819 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:33.265796  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.266607  5221 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:33.266758  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.267642  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.265911  5076 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:33.268376  5144 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:33.269695  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.270787  5939 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:33.272782  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.273576  5072 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 17.
I20260709 12:25:33.276656  5115 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:33.199285 10864 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.282847  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.283746  5943 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:33.275065  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:33.288515  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 104644us
I20260709 12:25:33.289587  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.299237  5221 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:25:33.299226  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.303131  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:33.307024  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.308223  5224 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 19: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:33.300941  5072 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:33.309079  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:33.310583  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 74122.9us
I20260709 12:25:33.312252  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:33.313750  5898 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:33.316659  5217 raft_consensus.cc:1240] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:33.318363  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:33.324173  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.296764 10728 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:33.326344 10908 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:25:33.328699 10862 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:33.329227 10862 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.329726 10862 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:33.322921 10856 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:33.326377 10728 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:33.330672 10856 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:33.331032 10728 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:33.331097 10856 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.331604 10908 raft_consensus.cc:3055] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:33.332042 10908 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:33.332396 10804 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:33.330163  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:33.335654 10908 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:33.337172 10908 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:33.339450 10749 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:33.344784 10728 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:33.345567 10870 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:33.346344 10870 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.346848 10870 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:33.347751 10728 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.351455 10879 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:33.322660 10810 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:33.347965  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:33.352514 10900 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:33.407239  5224 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:33.352979  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:33.361511  4891 catalog_manager.cc:5697] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 13, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 13 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:33.368916 10875 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.371769  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee83a16af0a40998fc58f3e882c6080" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.372799 10878 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:33.379341 10864 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.384086  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:33.387913 10873 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:33.389117  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:33.389715  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 77734.9us
I20260709 12:25:33.393163  5222 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 21.
I20260709 12:25:33.423151  5073 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 17: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:33.393834 10910 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.424985 10910 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:33.426189  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.427019  5070 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 12.
I20260709 12:25:33.427717 10910 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:33.394367  5142 raft_consensus.cc:1275] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:33.396651  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
W20260709 12:25:33.399186  5187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.430595 10713 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:33.431203 10713 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:33.432540  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.433231  5221 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:33.408241 10810 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.401845  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:33.405929  5076 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 21.
W20260709 12:25:33.408375  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 55509.5us
I20260709 12:25:33.407862  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.448230 10878 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:33.411541  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.450011  5939 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:33.451342  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.452502  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.412560  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f48767a7ee419197aff24610838e25" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:33.453836  5942 raft_consensus.cc:3060] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:33.415524  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.458392  5224 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 20.
I20260709 12:25:33.457587  4891 catalog_manager.cc:5697] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 14, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 14 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:33.459390  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.459766  4891 catalog_manager.cc:5697] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:33.413179 10866 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:33.461800 10866 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.462198 10866 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:33.463894 10713 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:33.466570  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.467284 10862 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.465248  5900 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:33.401997  5115 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.1.123.68:43449 timed out after 1.847s (SENT) [suppressed 22 similar messages]
I20260709 12:25:33.469405  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:33.470523  5939 raft_consensus.cc:2393] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 14: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:33.470768  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.439867  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.440524  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.440435  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.444495  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:33.449311  5148 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:33.476325  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.477414  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.435492 10856 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:33.484076  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.484897  5217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.479820  5942 raft_consensus.cc:2468] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 20.
I20260709 12:25:33.419971  5067 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:33.480468  5224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.481839  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.490177  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.500330 10882 consensus_queue.cc:1059] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:33.501211  5041 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.501864  5218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.503491  5041 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.503716 10917 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:33.504904 10910 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:33.506268  5041 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.508688 10910 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:33.510602  5900 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.512053 10518 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:33.486158 10812 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.514623 10862 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.514912 10812 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:33.470467 10810 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.472633  4891 catalog_manager.cc:5697] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 9 to 12, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 12 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:33.516772  5941 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 12.
I20260709 12:25:33.502101 10804 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:33.521839  5898 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:33.522449  5041 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.487857 10621 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.497828 10579 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:33.535113  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:33.535614  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:33.539122  5076 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:33.544284  5041 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.533140 10833 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:33.554818 10833 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 21 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:33.556370 10833 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.557052 10910 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.557755 10910 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:33.558139 10748 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:33.560281 10910 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:33.549157 10913 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:33.501848 10882 consensus_queue.cc:1059] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:25:33.528529 10804 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:33.563136 10804 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.564631 10804 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:33.537732 10908 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:33.536780 10621 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:33.567646  5067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58050 (request call id 1883) took 1969 ms (client timeout 2240 ms). Trace:
I20260709 12:25:33.523583 10862 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
W20260709 12:25:33.568157  5067 rpcz_store.cc:269] 0709 12:25:31.598629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.600668 (+  2039us) service_pool.cc:224] Handling call
0709 12:25:33.567446 (+1966778us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54004290d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:33.567555 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4123345664}
I20260709 12:25:33.548293  4891 catalog_manager.cc:5697] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 14 to 16, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 16 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:33.570900 10870 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:33.572566 10879 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:33.572824  5947 raft_consensus.cc:2368] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 17. Current term is 18.
I20260709 12:25:33.574294 10910 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:33.575899  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:33.577041 10910 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:25:33.580196  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 178208us
I20260709 12:25:33.584823  5041 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.586725  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:33.545907 10761 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:33.595074  5076 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:33.568223 10621 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:33.587162 10910 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:33.601135  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.603312  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.604419 10913 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:33.605001 10913 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:33.571026  4891 catalog_manager.cc:5697] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 13, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 13 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:33.589816 10717 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:33.617687  5114 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:33.623915  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:33.623661 10873 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.626187  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:33.633905 10856 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:33.635717  5946 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:33.640278  5068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49640 (request call id 1175) took 2083 ms. Trace:
I20260709 12:25:33.641292  5068 rpcz_store.cc:276] 0709 12:25:31.556692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.563070 (+  6378us) service_pool.cc:224] Handling call
0709 12:25:31.563479 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.554202 (+990723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.639853 (+1085651us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b540024efd4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853be01 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2 00007fd8a687648c 00007fd8a68758c6
0709 12:25:33.640140 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2275905792}
I20260709 12:25:33.645769 10890 raft_consensus.cc:993] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: : Instructing follower 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:33.646466 10890 raft_consensus.cc:1081] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 LEADER]: Signalling peer 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:33.647442 10890 raft_consensus.cc:3055] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:33.647781 10890 raft_consensus.cc:740] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:33.648536 10890 consensus_queue.cc:260] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.649029 10843 consensus_queue.cc:1059] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:33.651577  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.656340 10905 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:33.657052 10905 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.657106  5114 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:33.657482 10905 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:33.658478  5114 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:33.658746 10730 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:33.659583 10730 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:33.660101 10730 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:33.661306  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:33.651465 10862 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:25:33.662258 10614 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:33.662649 10862 raft_consensus.cc:3055] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:33.663246 10862 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:33.661306 10749 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:33.663879 10749 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.663933  5115 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:33.664338 10749 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.665109  5115 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:33.667564  5899 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:33.625612 10873 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:33.672278  5899 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:33.627991 10728 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:33.649478 10890 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:33.605034 10810 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:33.653403 10843 raft_consensus.cc:3055] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:33.686524  5899 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:33.696269  5899 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:33.672748 10769 raft_consensus.cc:2804] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:33.673004 10748 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:33.683137 10518 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:33.676290 10822 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:33.683117 10904 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:33.683130 10614 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:33.713907  5075 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 13: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:33.715358  5068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:33.717907  5068 raft_consensus.cc:606] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:33.718384  5068 raft_consensus.cc:614] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:33.694891 10843 raft_consensus.cc:740] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:33.721612 10843 consensus_queue.cc:260] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.722703 10843 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:33.710783 10904 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.723496 10904 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:33.726477 10730 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.727315 10810 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:33.705231 10815 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:33.664256 10862 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:33.712265 10769 raft_consensus.cc:697] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 20 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:33.719416 10866 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.737931 10769 consensus_queue.cc:237] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:33.739501 10866 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.741811  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:33.731304 10815 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:33.742630 10815 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:33.742686 10904 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:33.744213  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:33.744958 10904 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:33.745216  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:33.745855  5219 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Rejecting Update request from peer 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 18. Current term is 19. Ops: [18.12-18.12]
W20260709 12:25:33.753659  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 72173.1us
I20260709 12:25:33.754835  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:33.755220  5039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.756580  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.757586  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.758436  5074 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 18.
I20260709 12:25:33.765651  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:33.774911 10862 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:33.719632 10748 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:25:33.783906 10812 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:33.788385  5076 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:33.790518  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 88756.8us
I20260709 12:25:33.795157  5067 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:33.734112 10810 raft_consensus.cc:493] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:33.720319 10871 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:33.801976  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 47230.7us
I20260709 12:25:33.809334  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.809955  5115 leader_election.cc:400] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 17. Current term is 18.
I20260709 12:25:33.810333  5115 leader_election.cc:403] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:25:33.811352 10873 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.812016 10873 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:33.746502 10870 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:33.808113 10904 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:33.813922 10904 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:33.814306 10904 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:33.795047  5899 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:33.720291 10614 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:33.821039  5067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:33.823040  5189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.824705  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.831321  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:33.823076  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:33.832109  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:33.832688  5149 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:33.826210 10715 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:33.837749  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:33.840312 10614 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:33.844098 10904 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:33.851457 10905 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:33.852428  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.854604  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:33.941733  5220 raft_consensus.cc:2368] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 18. Current term is 19.
I20260709 12:25:33.855384  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:33.861057 10856 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:33.863811  5223 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:33.822275 10810 raft_consensus.cc:515] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.887060  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:33.946573  5068 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:33.888314  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:33.947886  5076 raft_consensus.cc:2368] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13.
I20260709 12:25:33.948495  5946 raft_consensus.cc:2393] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:33.888803  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:33.949362  5947 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.889932  5940 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:39870
I20260709 12:25:33.950306  5940 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:33.950688  5940 raft_consensus.cc:3060] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:33.890540  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.894520  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:33.896148  5898 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:33.952843  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 56772.4us
I20260709 12:25:33.893784 10804 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:25:33.857056 10749 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:33.911051 10815 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:33.965591  5898 leader_election.cc:400] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13.
I20260709 12:25:33.965870  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.966079  5898 leader_election.cc:403] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:25:33.915103  5067 raft_consensus.cc:2468] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 21.
I20260709 12:25:33.880651  5149 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 19.
I20260709 12:25:33.867758 10913 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:33.928157  5939 raft_consensus.cc:1240] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:33.936363 10809 consensus_queue.cc:1048] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:33.944617  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:33.963376  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.964394  5150 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:25:33.976418  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.976491  5074 raft_consensus.cc:3060] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:33.970662  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:33.977701  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 83261.5us
I20260709 12:25:33.977664  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:25:33.982252  5223 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35616 (request call id 1391) took 2405 ms (client timeout 1847 ms). Trace:
I20260709 12:25:33.978247  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:33.985183  5150 raft_consensus.cc:2393] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 12: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:33.985625  5142 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:33.989168  5149 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
W20260709 12:25:33.982975  5223 rpcz_store.cc:269] 0709 12:25:31.552494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.563284 (+ 10790us) service_pool.cc:224] Handling call
0709 12:25:33.957494 (+2394210us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b540023b9d4. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a853f2fa 00007fd8ad500b17 00007fd8a6a7b6d4 00007fd8a6a7b662 00007fd8a6a7b591 00007fd8a6a7b50c 00007fd8a6a7a7c2 00007fd8a687648c 00007fd8a68758c6 00007fd8a687526f
0709 12:25:33.957679 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5288220928}
I20260709 12:25:33.992303  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:33.996251  5115 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:33.993266  5146 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:33.951681 10856 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.998557  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:34.015599  5146 raft_consensus.cc:3060] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.016813  5144 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:33.999013  5900 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:33.999728  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:34.018322  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:34.001313  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:34.002488  5190 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:34.025345  5115 leader_election.cc:400] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 18. Current term is 19.
I20260709 12:25:34.025763  5115 leader_election.cc:403] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:25:34.015806  5148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
W20260709 12:25:34.026973  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 356312us
I20260709 12:25:33.950589 10599 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.027113  5148 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:34.028775  5075 raft_consensus.cc:1240] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:34.004736  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:34.029935  5149 raft_consensus.cc:2393] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 19: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:34.030669  5114 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:34.041420  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:34.042940  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:34.043228  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:34.044494  5069 raft_consensus.cc:2410] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:25:34.042421  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77063f082034d8bbc280357b7b66bb7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:34.043048  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:33.992281 10761 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:33.994437 10748 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:34.057216  5074 raft_consensus.cc:2468] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 19.
I20260709 12:25:34.012223 10614 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:33.986459 10728 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:34.032461 10799 raft_consensus.cc:2804] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:34.060411  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bc6f04e62a431ba2f113d08a129898" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:33.957857 10715 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:25:33.992846 10901 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:34.048471 10519 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 18. Current term is 19.
I20260709 12:25:34.026134 10856 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:34.038396 10939 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:25:33.989240 10822 consensus_queue.cc:1059] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:25:34.065970  5947 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:34.082048  5068 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
W20260709 12:25:34.081488  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50926.5us
I20260709 12:25:34.079030  5148 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:34.091288  5145 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:34.061676 10518 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:34.087466 10748 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:34.031905 10747 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:25:34.038730 10599 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:34.058470  5940 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:34.089354 10728 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:34.119377  5114 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:34.108916 10578 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 for earlier term 12. Current term is 13.
I20260709 12:25:34.144986  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:34.142866  5940 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:34.148515  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:34.149511  5220 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 21: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:34.152081  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:34.152407  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:34.129396 10822 raft_consensus.cc:3055] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:34.161794  5223 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 19: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:34.167940  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:34.137887 10810 leader_election.cc:290] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:34.128628 10799 raft_consensus.cc:697] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 19 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:34.098057 10901 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:34.142957 10579 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:34.189708  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:34.196300  5075 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.190261  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:34.163832 10900 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.165951 10822 raft_consensus.cc:740] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:34.211098 10728 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:34.211457  5068 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:34.213040  5072 raft_consensus.cc:1240] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:34.213941 10913 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:34.217144 10901 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:34.218538 10938 log.cc:927] Time spent T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:25:34.224539  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:34.226096  5146 raft_consensus.cc:2468] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:34.194731 10717 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:34.233121  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:34.196890 10873 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 17. Current term is 18.
I20260709 12:25:34.194732 10906 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:34.235502  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:34.236500  5075 raft_consensus.cc:2468] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:34.232838 10904 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:34.235841 10815 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:34.254905  5141 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:34.238315 10905 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:34.254189  5070 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:34.242449 10730 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.260542  5114 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:34.242126 10717 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:34.255520 10799 consensus_queue.cc:237] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:34.271368 10717 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:34.272859 10717 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:34.267490 10822 consensus_queue.cc:260] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:34.274772 10905 raft_consensus.cc:2804] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:34.275420 10905 raft_consensus.cc:697] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:34.260356 10715 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:34.236416 10748 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:34.267681  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 179093us
I20260709 12:25:34.284361  5115 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:34.288302  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:34.276427 10905 consensus_queue.cc:237] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:34.287101 10900 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:34.291105  5115 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:34.288595 10715 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:34.291839 10904 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:34.299446  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:34.278254 10856 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:34.302057  5042 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:34.299400  5900 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:34.303331  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:34.305287  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:34.301098 10519 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:34.308477  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:34.312180  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:34.311800  5900 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:34.313169  5222 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 19: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
W20260709 12:25:34.304289  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 48717.8us
I20260709 12:25:34.316522 10868 consensus_queue.cc:1059] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:34.319851  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:34.321244  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:34.308374 10913 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:34.304152 10810 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:34.325587  5142 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:34.321573  5042 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:34.309954 10819 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:34.331252 10856 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:34.325484  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:34.315251 10801 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:34.311020 10908 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:34.318141 10818 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:25:34.338413 10913 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.346911 10913 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:34.350015 10715 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:34.322027 10518 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:25:34.345281 10940 log.cc:927] Time spent T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:25:34.314774 10697 consensus_queue.cc:1059] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:25:34.361369  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:34.364529  5142 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:34.356375 10761 raft_consensus.cc:2804] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:34.363077 10519 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:34.363564 10900 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:34.394186 10518 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:34.450937 10518 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:34.448441 10900 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:34.437144  5142 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 20.
W20260709 12:25:34.458806  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 120167us
I20260709 12:25:34.458596  5900 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:34.461292  5188 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:34.469127  5188 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:34.436259 10761 raft_consensus.cc:697] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:34.465202 10801 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:34.476776 10801 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 20 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:34.464097 10614 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:34.483623  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 125601us
I20260709 12:25:34.487641  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:34.489642  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: true dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:34.490753  5075 raft_consensus.cc:2393] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 21: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:34.494377  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75210ca16c9d4d0e91d54b6ffbbd75d4" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:34.498412  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:34.500061 10954 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:34.501256 10954 raft_consensus.cc:2804] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:34.502206 10954 raft_consensus.cc:697] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 21 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:34.503087  5222 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:34.504375 10900 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:34.503137 10801 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:34.512662  5898 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
W20260709 12:25:34.520516  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 51503.3us
I20260709 12:25:34.523041  5188 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:34.535533  5898 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:34.525736 10960 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:34.550053 10801 raft_consensus.cc:2749] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:25:34.576484  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 52119.6us
I20260709 12:25:34.541731 10579 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:25:34.598266 10952 log.cc:927] Time spent T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:25:34.599772 10951 log.cc:927] Time spent T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:25:34.600172 10697 consensus_queue.cc:1059] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:34.611159  5075 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
W20260709 12:25:34.621686  5040 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.544970 10906 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:25:34.578603 10833 consensus_queue.cc:1059] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 12:25:34.632171  5898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 96751.7us
I20260709 12:25:34.633144  5041 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:34.634151 10908 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:34.635097  5041 leader_election.cc:304] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:34.578838 10761 consensus_queue.cc:237] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:34.636461 10599 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:34.637022 10717 raft_consensus.cc:2749] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:34.640616  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:34.642143  5146 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:34.610085 10954 consensus_queue.cc:237] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:34.616163 10579 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:34.651651  5074 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:25:34.628115 10769 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:34.659490  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 82099.6us
I20260709 12:25:34.668702 10714 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:34.668754 10579 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:34.653295 10856 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:25:34.675190 10913 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:34.681744 10913 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:34.687650 10957 consensus_queue.cc:1059] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:34.690083  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:34.695039  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:34.695946  5220 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:34.697039  5068 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:25:34.698861  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:34.699990  5071 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:34.706435  5142 raft_consensus.cc:1240] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:34.794881  5220 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 20.
W20260709 12:25:34.801934  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 59888.1us
I20260709 12:25:34.802767 10715 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:34.804152 10872 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:34.804595 10872 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:34.805514 10872 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:34.809741 10872 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:34.811372 10888 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.812074 10888 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:34.803184  5188 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:34.818300  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:34.819164 10714 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:34.819804 10714 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.820173 10714 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:34.664816 10769 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:34.824072 10769 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:34.816056 10888 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:34.796885 10699 raft_consensus.cc:3055] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:34.826186 10699 raft_consensus.cc:740] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:34.827442 10699 consensus_queue.cc:260] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:34.833673  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:34.834878  5220 raft_consensus.cc:2393] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:34.835723  5072 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:34.838021  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:34.819219  5143 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:34.690086 10818 raft_consensus.cc:3055] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:34.847199  5071 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 18: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:34.860256  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:34.707442 10599 consensus_queue.cc:1059] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.143s
I20260709 12:25:34.848623 10831 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.846673 10818 raft_consensus.cc:740] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:34.857239 10699 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:34.865228 10890 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:34.846321 10906 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:25:34.900624 10697 consensus_queue.cc:1059] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:25:34.911774 10697 consensus_queue.cc:1243] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer ceec344dd00f40c18ef58414ce147016 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:25:34.922801  5222 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:34.899037 10890 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.928438 10747 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:25:34.934949  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 127674us
I20260709 12:25:34.889497 10831 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:34.942034 10818 consensus_queue.cc:260] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:34.946570 10749 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 13 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:34.950721 10953 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.578s	user 0.002s	sys 0.003s
I20260709 12:25:34.998922  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:35.000101  5942 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:34.959962  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "874aa8edc94946d69e4a25563a5a0f0f" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:34.974964  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:34.963260 10578 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:34.961300 10890 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:34.980993 10831 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:34.986155 10818 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:34.987403  5947 raft_consensus.cc:1275] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:34.990219  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:34.972321  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:35.006042  5222 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:25:35.014396  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:35.019320  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:35.023521 10775 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:35.030541 10775 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:35.030943 10775 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:35.039577  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:35.038544  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 167274us
I20260709 12:25:35.068567 10964 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:35.069262 10964 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:35.070621 10942 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:35.071337 10942 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:35.071900 10964 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:35.034961 10901 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:35.081676 10942 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:35.016901 10578 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:35.084528 10578 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:35.103631  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:35.106532  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:35.108316  5115 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:35.105116  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:35.111908  5071 raft_consensus.cc:2393] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 15: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:35.116780  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:35.112319 10730 consensus_queue.cc:1048] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:25:35.118211  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:35.126926 10949 raft_consensus.cc:3055] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:35.105288 10748 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:25:35.127599 10949 raft_consensus.cc:740] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:35.128564 10949 consensus_queue.cc:260] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:35.129698  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:35.130306 10948 raft_consensus.cc:3055] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:35.130744 10948 raft_consensus.cc:740] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:35.131304  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:35.131673 10948 consensus_queue.cc:260] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:35.132863 10948 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:35.134395  5222 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:35.137574 10877 raft_consensus.cc:3055] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:35.138114 10877 raft_consensus.cc:740] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:35.139200 10877 consensus_queue.cc:260] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:35.140702 10877 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:35.152000  5898 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:35.158452  5898 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:35.162058  5943 raft_consensus.cc:1275] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:25:35.170570  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 264418us
W20260709 12:25:35.177673  5899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:35.183758  5900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.164424 10578 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:35.167008 10949 raft_consensus.cc:3060] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:35.164835 10856 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:35.162642 10748 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 12:25:35.208141  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 99962.4us
I20260709 12:25:35.209125  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:35.214406 10878 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:35.195271 10714 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:35.204460 10890 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:35.240475 10822 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:35.222880 10730 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:35.247539  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 76017.7us
I20260709 12:25:35.258006 10775 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:35.263949  5069 raft_consensus.cc:1240] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:35.264140  5074 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:35.251619 10804 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.275532 10714 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:35.279057  5149 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:35.286828  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:35.289312  5145 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:35.290030 10775 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:35.291095  5143 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:35.294629 10890 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:35.301064  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:35.280256 10804 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:35.305481 10804 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:35.278200 10819 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
W20260709 12:25:35.309952  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 120440us
W20260709 12:25:35.311676  5188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:35.313719  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 121294us
I20260709 12:25:35.316721  5900 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:35.312170  5947 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
W20260709 12:25:35.313858  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 96161.6us
I20260709 12:25:35.318944  5145 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:25:35.319639  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:35.320430  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:35.321413  5075 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:35.321733  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:35.320438  5939 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:35.334434  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:35.335352  5221 raft_consensus.cc:2393] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:35.338408  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:35.339125  5143 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:35.335592  5115 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:35.346148  5075 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:35.328403 10819 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.367650  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:35.327379 10906 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:35.369947  5146 raft_consensus.cc:3060] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:35.368139  5145 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:35.335739 10579 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:35.382422 10578 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:35.383682  5946 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:35.385672  5190 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:35.392951  5144 raft_consensus.cc:1240] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:35.397735  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 70398.5us
I20260709 12:25:35.397998  5219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:35.399027  5219 raft_consensus.cc:2410] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:25:35.362238 10900 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:35.400795 10900 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:35.400990  5215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:35.401643  5215 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:35.342494 10870 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:25:35.362986 10908 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.410576  5146 raft_consensus.cc:2468] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:35.411085  5947 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:25:35.415504  5041 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:35.401726 10900 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:35.389392 10872 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:35.431432  5190 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:35.405905 10877 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:25:35.432618 10804 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:35.435436 10699 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:35.363224 10747 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:25:35.441979  5944 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:35.379488 10906 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:35.428730  5041 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:25:35.428267 10979 log.cc:927] Time spent T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.052s	user 0.002s	sys 0.001s
I20260709 12:25:35.376534 10819 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:35.447836 10872 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.481628 10872 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:35.489967 10977 log.cc:927] Time spent T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.250s	user 0.003s	sys 0.000s
I20260709 12:25:35.492022  5150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1716) took 1130 ms. Trace:
I20260709 12:25:35.492574  5150 rpcz_store.cc:276] 0709 12:25:34.361239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.361947 (+   708us) service_pool.cc:224] Handling call
0709 12:25:34.362288 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.363746 (+  1458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.363790 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.944053 (+580263us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b2c00193f58. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a6e34a11 00007fd8a6e44372 00007fd8a8576a01 00007fd8a853b43a 00007fd8a853a83e 00007fd8ad4ff419 00007fd8a6a76bd4 00007fd8a6a76b62 00007fd8a6a76a91 00007fd8a6a76a0c 00007fd8a6a75cc2
0709 12:25:34.944663 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.173369 (+228706us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:35.173428 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.173576 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.489133 (+315557us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:35.491687 (+  2554us) raft_consensus.cc:1685] finished
0709 12:25:35.491721 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.491935 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":972,"raft.queue_time_us":1107,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":25478,"spinlock_wait_cycles":88071296,"thread_start_us":228201,"threads_started":1,"wal-append.queue_time_us":294762}
I20260709 12:25:35.456596 10818 raft_consensus.cc:2804] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:35.505895 10818 raft_consensus.cc:697] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:35.507121 10818 consensus_queue.cc:237] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:35.511196 10906 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:35.524057  5147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1714) took 1202 ms. Trace:
I20260709 12:25:35.521126 10872 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:35.467628 10761 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:35.533512 10761 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.534017 10761 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:35.534150  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:35.531937 10819 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:35.446153 10908 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:35.540763 10908 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:35.543264  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:35.549455  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:35.551565  5898 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:35.553391 10748 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:35.554193 10748 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:35.429667 10578 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:25:35.556502  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:35.524655  5147 rpcz_store.cc:276] 0709 12:25:34.321196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.321980 (+   784us) service_pool.cc:224] Handling call
0709 12:25:34.336822 (+ 14842us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.337214 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.337252 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.339143 (+  1891us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.937407 (+598264us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b18002ec0c0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a5430e07 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8a80d520b 00007fd8a80dad23 00007fd8a80db0d1 00007fd8a84f6a0f
0709 12:25:35.247302 (+309895us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:35.248298 (+   996us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.248585 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.248695 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.385196 (+136501us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
0709 12:25:35.523760 (+138564us) raft_consensus.cc:1685] finished
0709 12:25:35.523792 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.523973 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9467,"prepare.queue_time_us":148,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":38,"raft.queue_time_us":2164,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":14208,"spinlock_wait_cycles":46691072,"thread_start_us":309581,"threads_started":1,"wal-append.queue_time_us":992826}
I20260709 12:25:35.563957  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:35.565722 10775 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:35.566270  5071 raft_consensus.cc:1275] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:25:35.566627 10775 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:35.571745 10775 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:35.580026  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:35.580165  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:35.580914  5144 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:35.584944  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:35.602538  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:35.606371  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:35.616964  5070 raft_consensus.cc:1275] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:25:35.619169  5069 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:35.620034  5115 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:25:35.624063  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 47148.4us
I20260709 12:25:35.628382  5143 raft_consensus.cc:1275] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:35.631073  5222 raft_consensus.cc:1275] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:35.645056  5144 raft_consensus.cc:2468] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
W20260709 12:25:35.655395  5900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 274871us
I20260709 12:25:35.555471 10748 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:35.623807 10819 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:35.696022  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 62093.9us
I20260709 12:25:35.646085 10833 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:25:35.649060 10949 consensus_queue.cc:1048] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:25:35.681718 10749 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:35.682173 10769 consensus_queue.cc:1048] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:25:35.681329 10699 consensus_queue.cc:1048] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:25:35.691359 10510 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
W20260709 12:25:35.781365  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 132922us
I20260709 12:25:35.789644  5143 raft_consensus.cc:1240] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:25:35.752210 10749 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:35.776546 10833 consensus_queue.cc:1243] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:25:35.803506  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 160116us
I20260709 12:25:35.778743 10910 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:35.810140  5114 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:35.797188 10749 raft_consensus.cc:3060] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:35.824573 10761 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:35.831960 10987 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: Append to log took a long time: real 0.464s	user 0.000s	sys 0.004s
I20260709 12:25:35.808744 10875 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:35.804945 10943 consensus_queue.cc:1059] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:25:35.834304 10912 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:35.858650  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:35.863394  5072 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:35.866359 10730 raft_consensus.cc:493] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:35.867012 10730 raft_consensus.cc:515] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:35.867583  5070 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:35.857402 10875 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:35.820034 10910 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:35.873412  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:35.859761 10872 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:35.886307  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:35.885321 11006 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:35.902540 10819 consensus_queue.cc:1048] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:35.877044 10801 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 12:25:35.938419 11005 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.896754 11003 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:35.946878 10730 leader_election.cc:290] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:35.897509  5042 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:35.919003 10992 raft_consensus.cc:3055] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:35.951066 10992 raft_consensus.cc:740] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:35.952044 10992 consensus_queue.cc:260] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:35.919319 10910 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:35.877331 10948 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:25:35.932817 10761 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:35.886745 10999 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.944380 11005 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:35.953410 10992 raft_consensus.cc:3060] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:35.965107 10999 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:35.968381 11005 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:35.969663 10999 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:35.972349  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 74909.5us
I20260709 12:25:35.982710  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:35.972975 11005 raft_consensus.cc:2804] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:35.992136  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:35.992594  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:35.993146  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:35.994674  5146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:35.995390  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:35.996795  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:35.976497 10988 log.cc:927] Time spent T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.270s	user 0.000s	sys 0.004s
I20260709 12:25:35.980521 10875 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:35.891856 11006 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:35.986660  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82e74ddaf2154bc1ba1aeb5775d6e514" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:35.992937  5223 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 13.
I20260709 12:25:36.006815  5147 raft_consensus.cc:2393] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 18: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:35.996073  5145 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:36.021234  5115 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:36.030457  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:36.035061 10999 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:36.036949  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48592.8us
I20260709 12:25:36.042429 11005 raft_consensus.cc:697] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:36.046546 11005 consensus_queue.cc:237] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:36.076680  5223 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:36.083552  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:36.084501  5947 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:36.042984 10999 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.092907 11006 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:36.102291  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48787.1us
I20260709 12:25:36.108467  5115 leader_election.cc:304] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:36.114064  5947 raft_consensus.cc:2468] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:36.124259  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:36.127023  5223 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:36.090353 10579 raft_consensus.cc:493] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:36.120990 10769 raft_consensus.cc:2749] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:36.107475 10999 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:36.091533 10943 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:36.137997  5944 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:36.115376 10749 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:36.139523 10943 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:36.151628  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:36.151600  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:36.158001  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:36.164431  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:36.139632 10730 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:25:36.135492 10579 raft_consensus.cc:515] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:36.154160 10943 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:36.169951 10931 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:36.213199  5939 raft_consensus.cc:2468] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:36.186645 10749 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:36.223608  5115 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:36.226930  5073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:36.220966  5114 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:36.213581 11003 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:36.195240 10931 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:36.263581  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:36.275960  5070 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:25:36.278642  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 57813.2us
I20260709 12:25:36.194242 10769 consensus_queue.cc:1059] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:25:36.302670  5073 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:36.264894 10985 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.278795 10912 raft_consensus.cc:2804] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:36.321992 10985 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:36.324971 10579 leader_election.cc:290] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:36.276732 10931 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:36.290531 10878 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:36.287197 10978 log.cc:927] Time spent T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:25:36.352382  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:36.353314  5220 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:36.355253  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:36.356056  5223 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:36.349273  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:36.357663  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 75901.1us
I20260709 12:25:36.365581  5220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:25:36.366973  5220 raft_consensus.cc:606] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:36.322074 11003 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:36.370637 11003 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:36.371580 10999 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:36.376672  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:36.359957 10949 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.388585  5114 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:36.324995 10985 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:36.397585  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 118783us
I20260709 12:25:36.364782 10912 raft_consensus.cc:697] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 18 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:36.404847  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:36.408807  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:36.413780  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725c773e7344020b5752861d00b493e" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:36.415365  5068 raft_consensus.cc:2393] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 18: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:36.420269  5143 raft_consensus.cc:1275] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:36.405164 11006 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:36.411803 10878 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.445509  5189 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:25:36.385480 10818 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:25:36.450882 10715 raft_consensus.cc:2804] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:36.451568 10715 raft_consensus.cc:697] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:36.452531 10715 consensus_queue.cc:237] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:36.436348 10908 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:36.458415 10908 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:36.459908 10912 consensus_queue.cc:237] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:36.460906 10908 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:36.398603 10949 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:36.462286  5216 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:25:36.463193 10949 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:36.464763 10931 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:36.498342  5223 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
I20260709 12:25:36.513852  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:36.514707  5144 raft_consensus.cc:2368] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:25:36.514991  5075 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:25:36.469966 10878 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:36.517158 10999 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:36.519423  5075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:36.520143  5075 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:36.500770 10957 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:25:36.540794 11003 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:36.544469 10833 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:36.547259 10875 raft_consensus.cc:3055] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:36.547736 10875 raft_consensus.cc:740] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:36.548590 10875 consensus_queue.cc:260] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:36.551802 10878 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:36.554183 10875 raft_consensus.cc:3060] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:36.555181  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:36.556454  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:36.568338  5068 raft_consensus.cc:1240] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:36.570427  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:36.571401  5222 raft_consensus.cc:3060] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:36.571791  5042 leader_election.cc:400] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:25:36.572235  5042 leader_election.cc:403] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:36.559554  5069 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:36.584569  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:36.556499  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:36.585290  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:36.586287  5223 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:36.586539  5947 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:36.590758  5075 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
I20260709 12:25:36.592782  5074 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 16: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:36.589872  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:36.595991  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:36.599496  5220 raft_consensus.cc:1240] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:25:36.613957 10888 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:36.614881 10888 consensus_queue.cc:1243] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Peer 9847a7797ddc4df689a1ab0911af8785 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:36.595008 10890 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855) with higher term. Message: Invalid argument: T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:25:36.594715 10578 consensus_queue.cc:1059] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:25:36.588129 11003 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:36.555341 10931 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:36.602072 11010 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:36.635200 11010 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.635625 11010 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:36.636950  5947 pending_rounds.cc:85] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Aborting all ops after (but not including) 8
I20260709 12:25:36.637331  5947 pending_rounds.cc:107] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:25:36.637573  5947 raft_consensus.cc:2889] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:36.635456  5114 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:36.653896  5222 raft_consensus.cc:2468] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 14.
W20260709 12:25:36.657608  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 212173us
W20260709 12:25:36.658396  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 208354us
I20260709 12:25:36.660292  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:36.662643  5221 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:36.656792 10999 raft_consensus.cc:2804] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:36.681655 10999 raft_consensus.cc:697] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:36.682922 10999 consensus_queue.cc:237] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:36.674028 11006 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:36.685384  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:36.687852 11006 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:36.689787 10801 raft_consensus.cc:3055] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:36.690282 10801 raft_consensus.cc:740] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:36.690989 11006 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:36.691228 10801 consensus_queue.cc:260] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:36.692853 10801 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:36.662817 11003 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:36.709005  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7383bc919f9c47378977b98535752581" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:36.709332  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:36.709874  5222 raft_consensus.cc:2368] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:25:36.709905  5221 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 19.
I20260709 12:25:36.722047  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:36.735968  5899 leader_election.cc:304] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:36.706346 11023 consensus_queue.cc:1048] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:25:36.732337 10856 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:36.739646 10833 raft_consensus.cc:2749] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:36.803195  5041 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:25:36.809950  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:36.810761  5074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:36.812884  5074 raft_consensus.cc:2368] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16.
I20260709 12:25:36.815786  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:36.819977  5075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5244acc0775748b7933286d588ec2b56"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:36.821121  5075 raft_consensus.cc:606] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:25:36.821681  5075 raft_consensus.cc:614] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:36.826285  5114 leader_election.cc:400] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16.
I20260709 12:25:36.829522  5075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:36.835415  5114 leader_election.cc:403] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:36.835686  5075 raft_consensus.cc:606] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:36.836434  5075 raft_consensus.cc:614] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:36.838771  5150 raft_consensus.cc:1275] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:25:36.847894  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 138708us
I20260709 12:25:36.851413  5221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:36.852859  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 116770us
I20260709 12:25:36.853331  5221 raft_consensus.cc:2368] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16.
I20260709 12:25:36.862761  5188 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:36.866580 10550 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:36.870635 10875 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:36.871693 10875 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:36.873553 11007 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:36.874910 10997 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:25:36.826395  5041 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: ceec344dd00f40c18ef58414ce147016
I20260709 12:25:36.892364 10819 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.893803 10985 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:36.894596 10985 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:36.895119 10985 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:36.893407 10819 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:36.874825 11007 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:36.909551 10819 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:36.767547 11010 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:36.924930 10862 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.952523 10862 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:36.925134  5075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:36.958024  5075 raft_consensus.cc:606] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:25:36.958636  5075 raft_consensus.cc:614] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:36.965832  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 100279us
W20260709 12:25:36.968770 11020 log.cc:927] Time spent T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.088s	user 0.004s	sys 0.005s
I20260709 12:25:36.919940 11007 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:36.997449 10812 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:36.998320 10812 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:36.998664  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 172576us
I20260709 12:25:37.002568  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:37.004257  5068 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 14: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:36.936383 10992 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 12:25:37.017935  5068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "ceec344dd00f40c18ef58414ce147016"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:37.018913  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:37.067400  5943 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
I20260709 12:25:37.028620 10862 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:37.047047 10812 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:37.055549  5041 leader_election.cc:400] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 13. Current term is 14.
I20260709 12:25:37.057096 10997 raft_consensus.cc:2749] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955) with higher term. Message: Invalid argument: T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16.
I20260709 12:25:37.057875  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:37.059859 11010 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:37.060420  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:37.073043  5076 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 14.
I20260709 12:25:36.926175 10875 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:37.067948  5068 raft_consensus.cc:606] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Received request to transfer leadership to TS ceec344dd00f40c18ef58414ce147016
I20260709 12:25:37.077658  5068 raft_consensus.cc:614] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:37.085919  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:37.088542  5898 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:37.089849 10819 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:37.090479 10819 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:37.091106 10819 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:37.092756 10943 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:37.097010 10943 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.099493  5068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5751113d37b44af87d3d912da388078"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:56034
I20260709 12:25:37.097496 10943 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:37.107982  5068 raft_consensus.cc:606] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:37.120846  5068 raft_consensus.cc:614] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:37.131351  5041 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:37.134419 11005 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:37.134935 11005 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.135306 11005 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:37.139611  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:37.141162  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:37.142196  5147 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 13.
I20260709 12:25:37.158399  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:37.159238  5149 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 13.
I20260709 12:25:37.168202 10984 consensus_queue.cc:1048] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:37.170686  5076 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:37.173323 11010 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:37.173985  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:37.191706  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:37.195070  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:37.202428  5944 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:37.201965 10985 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:37.221002  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:37.224046  5222 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:37.217420 10943 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:37.250901  5943 raft_consensus.cc:2368] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15.
I20260709 12:25:37.252130 10943 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:37.261798  5222 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:37.271005  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:37.278218  5072 raft_consensus.cc:3060] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:37.285953 10697 raft_consensus.cc:993] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: : Instructing follower 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:37.287056 10963 raft_consensus.cc:1081] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Signalling peer 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:37.289014 10985 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:37.315009 10819 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:37.315515  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:37.316406  5216 raft_consensus.cc:2368] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 18. Current term is 19.
I20260709 12:25:37.314772  5072 raft_consensus.cc:2468] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:37.323119  5041 leader_election.cc:400] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 18. Current term is 19.
I20260709 12:25:37.323644  5041 leader_election.cc:403] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:25:37.327235  5115 leader_election.cc:400] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 15. Current term is 16.
I20260709 12:25:37.332242  5115 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:37.333832 10985 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449) with higher term. Message: Invalid argument: T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f for earlier term 18. Current term is 19.
I20260709 12:25:37.341866 11023 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:25:37.350069 10875 raft_consensus.cc:2804] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:37.350816 10875 raft_consensus.cc:697] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 19 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:37.352074 10875 consensus_queue.cc:237] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:37.358364  5943 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:39878
I20260709 12:25:37.359206  5943 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:37.359650  5943 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:37.318914 11005 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:37.362371  5071 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9847a7797ddc4df689a1ab0911af8785 in current term 15: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:37.362568  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:37.363780  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:37.365044 11005 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:37.368716  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:37.369630  5944 raft_consensus.cc:2393] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dc0d06e662d49d88e1b153e09a44e4f in current term 15: Already voted for candidate 9847a7797ddc4df689a1ab0911af8785 in this term.
I20260709 12:25:37.381237  5149 raft_consensus.cc:1275] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:25:37.381657  5149 pending_rounds.cc:85] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: Aborting all ops after (but not including) 8
I20260709 12:25:37.381963  5149 pending_rounds.cc:107] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:25:37.382272  5149 raft_consensus.cc:2889] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:37.394680  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:37.317427 10819 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:37.401126 10913 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:37.448433  5943 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:37.454538  5947 raft_consensus.cc:1240] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:37.463310  5943 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:37.469597  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:37.470427  5142 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:25:37.479676  5222 raft_consensus.cc:1275] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:37.484879  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:37.485076  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a61a09c949a4cc9959bd9cfbe00992a" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:37.489203  5220 raft_consensus.cc:3055] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:37.493093  5220 raft_consensus.cc:740] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:37.497367  5220 consensus_queue.cc:260] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:37.498334  5150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:37.500514  5115 leader_election.cc:400] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15.
I20260709 12:25:37.501187  5115 leader_election.cc:403] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:37.498724  5220 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:37.511690  5220 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:37.513177  5899 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:37.515471 10992 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:37.516149 10992 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:37.522946 11033 consensus_queue.cc:1048] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:37.546802  5142 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 15.
I20260709 12:25:37.548066 10997 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 14. Current term is 15.
I20260709 12:25:37.517272 10992 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:37.569399  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:37.571296 10812 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:37.594828  5149 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:37.607817  5076 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:37.630223  5069 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:37.630388 11008 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:25:37.642575 11010 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:37.643581  5900 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:37.644598 10862 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:25:37.661584  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 69435.8us
I20260709 12:25:37.660647 10862 raft_consensus.cc:697] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:37.684158  5943 raft_consensus.cc:1275] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:37.687813 10761 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:25:37.663009 10912 consensus_queue.cc:1048] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 12:25:37.703029  5222 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:37.693207 10862 consensus_queue.cc:237] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:37.749342  5072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:37.750151  5072 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:37.752107  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 138767us
W20260709 12:25:37.757513  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 48884.5us
I20260709 12:25:37.769150  5076 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Rejecting Update request from peer 9847a7797ddc4df689a1ab0911af8785 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:37.754858 10812 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:37.797695 10812 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:37.806396  5072 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 16.
I20260709 12:25:37.789652 11001 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 12:25:37.868489 10801 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:25:37.869664 11063 consensus_queue.cc:1048] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
W20260709 12:25:37.872341 11055 log.cc:927] Time spent T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:37.874230 10856 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.874861 10856 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:37.869836 10801 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:37.882006 10801 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:37.883098 10801 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:37.891865  5069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:37.901275  5072 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:25:37.885432 10856 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:37.906610  5069 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 19.
I20260709 12:25:37.913089 10801 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:37.920742  5898 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
W20260709 12:25:37.944028  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 85077.8us
I20260709 12:25:37.942277 10862 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:37.951152 10862 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.958151 10862 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:37.969786 10931 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
W20260709 12:25:38.001704 11048 log.cc:927] Time spent T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: Append to log took a long time: real 0.583s	user 0.006s	sys 0.004s
I20260709 12:25:38.006470  5216 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:38.029747 11076 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:38.030472 11076 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:38.032893 10812 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:38.034461 11010 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:38.032900 11076 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:38.061353  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:38.062042  5220 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 16.
W20260709 12:25:38.114037  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 52647.4us
I20260709 12:25:38.116395  5076 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:38.117944  5041 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
W20260709 12:25:38.114979 11075 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:25:38.122242 10875 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:38.130616 10875 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:38.126492 10992 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:38.132750  5222 raft_consensus.cc:1275] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:25:38.133385 10875 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:38.127993  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:38.137765  5142 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 18.
I20260709 12:25:38.197995  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:38.209365  5070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:38.210444  5070 raft_consensus.cc:2393] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 19: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:38.216107  5220 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:38.240239  5939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
W20260709 12:25:38.251616 11060 log.cc:927] Time spent T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:25:38.257457  4898 catalog_manager.cc:5697] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 9 to 10, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 10 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:38.269131  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:38.295109  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 106638us
I20260709 12:25:38.264758 10992 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:25:38.298301 11001 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:38.345255 11001 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:38.345912 11001 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:38.341604 11057 consensus_queue.cc:1048] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:25:38.345521 10862 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:38.408630  5939 raft_consensus.cc:2368] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 19. Current term is 20.
I20260709 12:25:38.415512 10862 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:38.422401 11001 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:38.422734  4898 catalog_manager.cc:5697] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:38.434994  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:38.449689 11001 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:38.463598  4898 catalog_manager.cc:5697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:38.465260  4898 catalog_manager.cc:5697] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:38.469014  4898 catalog_manager.cc:5697] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 5 to 12, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 12 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:38.478319  4898 catalog_manager.cc:5697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 10 to 11, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 11 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:38.488396  5069 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:38.499574  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:38.504087  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 184397us
I20260709 12:25:38.504467  5071 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:38.511966 10579 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:25:38.512394  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:38.552721  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:38.553565  5147 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:25:38.566990  5143 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:25:38.569487  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 43581us
I20260709 12:25:38.575349  5115 leader_election.cc:400] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 19. Current term is 20.
I20260709 12:25:38.575861  5115 leader_election.cc:403] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:38.575443  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:38.586745  5071 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 20.
I20260709 12:25:38.592612  5898 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:38.609666  5190 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:38.606667 10579 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:38.620163 10579 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 20 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:38.619326 11056 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:38.624006  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:38.621447 10579 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:38.626117 11005 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:38.644763 10862 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:38.683314  5147 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 20.
I20260709 12:25:38.684139 10769 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253) with higher term. Message: Invalid argument: T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 for earlier term 19. Current term is 20.
I20260709 12:25:38.750008 10831 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:38.750663 10831 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:38.753037 10831 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:38.754738  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:38.755811  5068 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 20: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:38.769071  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:38.769932  5942 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 20.
I20260709 12:25:38.774343  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:38.775357 10831 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:38.775820 10831 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:38.776175 10831 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:38.789707  4891 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43222 (request call id 145) took 15260 ms (client timeout 15000 ms). Trace:
I20260709 12:25:38.797494 10831 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:38.800671  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:38.802192  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:38.803004  5068 raft_consensus.cc:2393] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 20: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
W20260709 12:25:38.790504  4891 rpcz_store.cc:269] 0709 12:25:23.546185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.620677 (+ 74492us) service_pool.cc:224] Handling call
0709 12:25:38.740776 (+15120099us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:34.394917 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:34.604960 (+210043us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:34.605033 (+    73us) write_op.cc:435] Acquired schema lock
0709 12:25:34.605056 (+    23us) tablet.cc:662] Decoding operations
0709 12:25:34.610561 (+  5505us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:34.610727 (+   166us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:34.610759 (+    32us) tablet.cc:685] Acquiring locks for 27 operations
0709 12:25:34.678723 (+ 67964us) tablet.cc:701] Row locks acquired
0709 12:25:34.678787 (+    64us) write_op.cc:260] PREPARE: finished
0709 12:25:34.679001 (+   214us) write_op.cc:270] Start()
0709 12:25:34.679213 (+   212us) write_op.cc:276] Timestamp: P: 1783599934678917 usec, L: 0
0709 12:25:34.679241 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:25:34.682715 (+  3474us) log.cc:844] Serialized 10544 byte log entry
0709 12:25:35.402610 (+719895us) op_driver.cc:464] REPLICATION: finished
0709 12:25:35.750062 (+347452us) write_op.cc:301] APPLY: starting
0709 12:25:35.750192 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:36.508322 (+758130us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b180043a1a0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a827a0ec 00007fd8a827caf7 00007fd8a827dfb0 00007fd8abf921bb 00007fd8abf92532 00007fd8abfd9b93 00007fd8abe86de3 00007fd8abe866ab 00007fd8abe87e77
0709 12:25:36.632717 (+124395us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:36.632753 (+    36us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:37.229007 (+596254us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:37.229766 (+   759us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=54,mrs_lookups=0
0709 12:25:37.229885 (+   119us) write_op.cc:312] APPLY: finished
0709 12:25:37.230936 (+  1051us) log.cc:844] Serialized 235 byte log entry
0709 12:25:37.446715 (+215779us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:37.469678 (+ 22963us) write_op.cc:454] Released schema lock
0709 12:25:37.469786 (+   108us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:37.470570 (+   784us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:37.470606 (+    36us) write_op.cc:454] Released schema lock
0709 12:25:38.000820 (+530214us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a80d686e 00007fd8a80d5bb6 00007fd8a80d57d8 00007fd8a80df221 00007fd8a80df1d9 00007fd8a80df169
Metrics: {"mutex_wait_us":9918,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":345309,"apply.run_cpu_time_us":77192,"apply.run_wall_time_us":1720437,"cfile_cache_hit":110,"cfile_cache_hit_bytes":116933,"num_ops":27,"prepare.queue_time_us":42875,"prepare.run_cpu_time_us":38045,"prepare.run_wall_time_us":531422,"raft.queue_time_us":439333,"raft.run_cpu_time_us":100563,"raft.run_wall_time_us":310693,"replication_time_us":692146,"spinlock_wait_cycles":51246720,"thread_start_us":952809,"threads_started":6,"wal-append.queue_time_us":456621,"wal-append.run_cpu_time_us":124271,"wal-append.run_wall_time_us":1324548}]]}
I20260709 12:25:38.804757 10831 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:38.813202  5946 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 20.
I20260709 12:25:38.814344  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:38.815681 10831 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:38.816263 10831 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 20 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:38.817235 10831 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:38.912606  4898 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49944 (request call id 80) took 38371 ms (client timeout 15000 ms). Trace:
I20260709 12:25:38.917677  4894 catalog_manager.cc:5697] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 15 to 17, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 17 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:38.921674  4894 catalog_manager.cc:5697] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 6 to 13, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 13 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:38.913106  4898 rpcz_store.cc:269] 0709 12:25:00.541051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.544271 (+  3220us) service_pool.cc:224] Handling call
0709 12:25:38.912502 (+38368231us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:38.599835 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:38.600566 (+   731us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:38.600619 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:25:38.600658 (+    39us) tablet.cc:662] Decoding operations
0709 12:25:38.614378 (+ 13720us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:38.637982 (+ 23604us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:38.638040 (+    58us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:25:38.640350 (+  2310us) tablet.cc:701] Row locks acquired
0709 12:25:38.640406 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:25:38.640599 (+   193us) write_op.cc:270] Start()
0709 12:25:38.640779 (+   180us) write_op.cc:276] Timestamp: P: 1783599938640529 usec, L: 0
0709 12:25:38.640804 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:25:38.641516 (+   712us) log.cc:844] Serialized 2463 byte log entry
0709 12:25:38.757579 (+116063us) op_driver.cc:464] REPLICATION: finished
0709 12:25:38.762343 (+  4764us) write_op.cc:301] APPLY: starting
0709 12:25:38.762866 (+   523us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:38.804421 (+ 41555us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:38.804590 (+   169us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:38.861110 (+ 56520us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:38.861306 (+   196us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 12:25:38.861360 (+    54us) write_op.cc:312] APPLY: finished
0709 12:25:38.861634 (+   274us) log.cc:844] Serialized 64 byte log entry
0709 12:25:38.861827 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:38.861936 (+   109us) write_op.cc:454] Released schema lock
0709 12:25:38.861981 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:38.862401 (+   420us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:38.862430 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1662620,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":4375,"apply.run_cpu_time_us":12148,"apply.run_wall_time_us":100030,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29195,"num_ops":6,"prepare.queue_time_us":60347,"prepare.run_cpu_time_us":4311,"prepare.run_wall_time_us":42055,"replication_time_us":107829,"thread_start_us":53241,"threads_started":2}]]}
I20260709 12:25:38.930677  4894 catalog_manager.cc:5697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 15 to 19, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 19 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:38.949388  4894 catalog_manager.cc:5697] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 8 to 13, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 13 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:38.958154  4894 catalog_manager.cc:5697] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:38.960253  4894 catalog_manager.cc:5697] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 13 to 16, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 16 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:38.974268 10985 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.974905 10985 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:38.977792  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:38.978703  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785" is_pre_election: true
I20260709 12:25:38.979530  5942 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 20.
I20260709 12:25:38.977993 10985 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:38.985428  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:38.991678 10913 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:38.992213 10913 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.992558 10913 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:38.962212  4894 catalog_manager.cc:5697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 6 to 19. New cstate: current_term: 19 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:39.013082  4894 catalog_manager.cc:5697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:39.034725 10913 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:39.059103 10913 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:39.060724  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
I20260709 12:25:39.061419  5942 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:39.062608  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1339c2a0d7f344d09f483e6d965adcc8" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:39.077353  5942 raft_consensus.cc:2468] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 21.
I20260709 12:25:39.079025  5041 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:39.080411 10985 raft_consensus.cc:2804] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:39.081265 10985 raft_consensus.cc:697] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:39.082381 10985 consensus_queue.cc:237] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:39.098109  5145 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:25:39.100092 10862 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:39.118757  4896 catalog_manager.cc:5697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 9 to 12, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 12 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:39.128497  4894 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43234 (request call id 128) took 16201 ms (client timeout 15000 ms). Trace:
I20260709 12:25:39.136119  5076 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Rejecting Update request from peer ceec344dd00f40c18ef58414ce147016 for earlier term 16. Current term is 17. Ops: []
W20260709 12:25:39.130221  4894 rpcz_store.cc:269] 0709 12:25:22.927183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.927528 (+   345us) service_pool.cc:224] Handling call
0709 12:25:39.128389 (+16200861us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:39.020440 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:39.020940 (+   500us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:39.020978 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:25:39.020995 (+    17us) tablet.cc:662] Decoding operations
0709 12:25:39.021858 (+   863us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:39.021912 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:39.021940 (+    28us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:25:39.022736 (+   796us) tablet.cc:701] Row locks acquired
0709 12:25:39.022757 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:25:39.022914 (+   157us) write_op.cc:270] Start()
0709 12:25:39.023010 (+    96us) write_op.cc:276] Timestamp: P: 1783599939022863 usec, L: 0
0709 12:25:39.023024 (+    14us) op_driver.cc:348] REPLICATION: starting
0709 12:25:39.024753 (+  1729us) log.cc:844] Serialized 3229 byte log entry
0709 12:25:39.029425 (+  4672us) op_driver.cc:464] REPLICATION: finished
0709 12:25:39.032124 (+  2699us) write_op.cc:301] APPLY: starting
0709 12:25:39.032214 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:39.049160 (+ 16946us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:39.049260 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:39.065085 (+ 15825us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:39.065308 (+   223us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 12:25:39.065361 (+    53us) write_op.cc:312] APPLY: finished
0709 12:25:39.065725 (+   364us) log.cc:844] Serialized 80 byte log entry
0709 12:25:39.065914 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.066065 (+   151us) write_op.cc:454] Released schema lock
0709 12:25:39.066122 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:39.066565 (+   443us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.066599 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":349,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":2438,"apply.run_cpu_time_us":11858,"apply.run_wall_time_us":34395,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33424,"num_ops":8,"prepare.queue_time_us":110,"prepare.run_cpu_time_us":3606,"prepare.run_wall_time_us":26833,"replication_time_us":6248}]]}
I20260709 12:25:39.138535 11091 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:39.140007 11091 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:39.140416 11091 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:39.141398 11091 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:39.142964 11091 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:39.150755  5076 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:25:39.152884 10579 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:39.185743  4898 catalog_manager.cc:5697] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 10 to 18, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 18 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.188519  4898 catalog_manager.cc:5697] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 20 to 21, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 21 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:39.192466  4898 catalog_manager.cc:5697] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 17 to 19, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 19 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:39.193418  4898 catalog_manager.cc:5697] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 14 to 19, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 19 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.195065  4898 catalog_manager.cc:5697] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:39.196347  4898 catalog_manager.cc:5697] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:39.197554  4898 catalog_manager.cc:5697] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 14 to 17, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 17 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.208069  5071 raft_consensus.cc:1240] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:39.208037  4898 catalog_manager.cc:5697] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 15 to 16, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 16 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:39.205109  4896 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43226 (request call id 131) took 59266 ms (client timeout 15000 ms). Trace:
I20260709 12:25:39.210050  4898 catalog_manager.cc:5697] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.212057  4898 catalog_manager.cc:5697] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 12 to 14, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 14 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:39.214419 11071 consensus_queue.cc:1059] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:39.215252 11071 consensus_queue.cc:1243] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer 6dc0d06e662d49d88e1b153e09a44e4f log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:25:39.215875 10699 raft_consensus.cc:3055] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:39.216323 10699 raft_consensus.cc:740] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:39.217300 10699 consensus_queue.cc:260] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:39.222795 10699 raft_consensus.cc:3060] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:39.214874  4898 catalog_manager.cc:5697] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 17 to 18, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 18 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:39.234763  4898 catalog_manager.cc:5697] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 13 to 16, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 16 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:39.244477  4898 catalog_manager.cc:5697] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 10 to 16, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 16 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
W20260709 12:25:39.211284  4896 rpcz_store.cc:269] 0709 12:24:39.938491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.942031 (+  3540us) service_pool.cc:224] Handling call
0709 12:25:39.204999 (+59262968us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:39.138333 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:39.138914 (+   581us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:39.138953 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:25:39.138971 (+    18us) tablet.cc:662] Decoding operations
0709 12:25:39.139334 (+   363us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:39.139401 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:39.139427 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:39.139634 (+   207us) tablet.cc:701] Row locks acquired
0709 12:25:39.139655 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:25:39.139812 (+   157us) write_op.cc:270] Start()
0709 12:25:39.139939 (+   127us) write_op.cc:276] Timestamp: P: 1783599939139761 usec, L: 0
0709 12:25:39.139958 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:25:39.140566 (+   608us) log.cc:844] Serialized 537 byte log entry
0709 12:25:39.142324 (+  1758us) op_driver.cc:464] REPLICATION: finished
0709 12:25:39.142641 (+   317us) write_op.cc:301] APPLY: starting
0709 12:25:39.142728 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:39.143619 (+   891us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:39.143725 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:39.144772 (+  1047us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:39.144972 (+   200us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:25:39.145030 (+    58us) write_op.cc:312] APPLY: finished
0709 12:25:39.145267 (+   237us) log.cc:844] Serialized 24 byte log entry
0709 12:25:39.145477 (+   210us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.145673 (+   196us) write_op.cc:454] Released schema lock
0709 12:25:39.145723 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:39.146185 (+   462us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.146217 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":355955,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":115,"apply.run_cpu_time_us":3471,"apply.run_wall_time_us":3469,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4178,"num_ops":1,"prepare.queue_time_us":156,"prepare.run_cpu_time_us":2580,"prepare.run_wall_time_us":2579,"replication_time_us":2207}]]}
I20260709 12:25:39.246518  4898 catalog_manager.cc:5697] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 14 to 16, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 16 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:39.250437  4898 catalog_manager.cc:5697] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 12 to 13, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 13 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:39.256729  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:39.266659  4898 catalog_manager.cc:5697] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 20 to 21, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 21 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:39.268486  4898 catalog_manager.cc:5697] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 15 to 18, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 18 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:39.273670  4898 catalog_manager.cc:5697] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 8 to 16. New cstate: current_term: 16 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:39.278263  4898 catalog_manager.cc:5697] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 10 to 15, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 15 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:39.292620  4898 catalog_manager.cc:5697] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 13 to 20, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 20 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:39.294528  4898 catalog_manager.cc:5697] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 16 to 17, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 17 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.296548  4898 catalog_manager.cc:5697] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 8 to 17, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 17 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.298352  4898 catalog_manager.cc:5697] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:39.299789  4898 catalog_manager.cc:5697] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 13 to 14, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 14 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.301584  4898 catalog_manager.cc:5697] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 14 to 16, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 16 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.303164  4898 catalog_manager.cc:5697] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 12 to 16, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 16 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.304672  4898 catalog_manager.cc:5697] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 12 to 13, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 13 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:39.306056  4898 catalog_manager.cc:5697] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 9 to 14, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 14 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } }
I20260709 12:25:39.307385  4898 catalog_manager.cc:5697] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:39.308621  4898 catalog_manager.cc:5697] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 4 to 13, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 13 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.371734  4898 catalog_manager.cc:5697] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 12 to 19, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 19 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:39.392021  4898 catalog_manager.cc:5697] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:39.427872  5222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:25:39.428658  5222 raft_consensus.cc:606] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 21 LEADER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:39.437419  5220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85f48767a7ee419197aff24610838e25"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:25:39.438109  5220 raft_consensus.cc:606] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:39.438495  5220 raft_consensus.cc:614] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:39.450124  5220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8bbb28bc9f5410d95b20827beeb87cc"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:25:39.450722  5220 raft_consensus.cc:606] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:39.451076  5220 raft_consensus.cc:614] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:39.458336  5220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:25:39.459010  5220 raft_consensus.cc:606] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Received request to transfer leadership to TS 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:39.461964  5220 raft_consensus.cc:614] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:39.464623  5222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9"
dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
mode: GRACEFUL
new_leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:57004
I20260709 12:25:39.465408  5222 raft_consensus.cc:606] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Received request to transfer leadership to TS 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:39.467010  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:39.468945  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:39.469863  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:39.547648 11071 raft_consensus.cc:993] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: : Instructing follower 6dc0d06e662d49d88e1b153e09a44e4f to start an election
I20260709 12:25:39.548280 11071 raft_consensus.cc:1081] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Signalling peer 6dc0d06e662d49d88e1b153e09a44e4f to start an election
I20260709 12:25:39.550220  5076 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9"
dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
 from {username='slave'} at 127.0.0.1:58050
I20260709 12:25:39.550812  5076 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:39.551160  5076 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:39.597452  5222 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:25:39.597985  5222 pending_rounds.cc:85] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 13
I20260709 12:25:39.598205  5222 pending_rounds.cc:107] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:25:39.598472  5222 raft_consensus.cc:2889] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:39.600732 10913 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:39.607335  5076 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:39.616528  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:39.617139  5220 raft_consensus.cc:3055] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:39.617439  5220 raft_consensus.cc:740] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:39.617559  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:39.618351  5147 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:39.624316  5946 raft_consensus.cc:1275] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:25:39.626044 10985 consensus_queue.cc:1048] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:39.618274  5220 consensus_queue.cc:260] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:39.629793  5220 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:39.649016  5076 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:39.654811  5220 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:39.661386  5041 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:39.662303 10913 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:39.662807 10913 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:39.663754 10913 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:39.660019  5147 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:39.756590  4898 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43222 (request call id 146) took 1014 ms. Trace:
I20260709 12:25:39.762602  4890 catalog_manager.cc:5697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 12 to 21, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 21 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.757274  4898 rpcz_store.cc:276] 0709 12:25:38.742517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.942821 (+200304us) service_pool.cc:224] Handling call
0709 12:25:39.756479 (+813658us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:39.419904 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:39.420431 (+   527us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:39.420466 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:25:39.420478 (+    12us) tablet.cc:662] Decoding operations
0709 12:25:39.422665 (+  2187us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:39.422869 (+   204us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:39.422901 (+    32us) tablet.cc:685] Acquiring locks for 32 operations
0709 12:25:39.425195 (+  2294us) tablet.cc:701] Row locks acquired
0709 12:25:39.425217 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:25:39.425392 (+   175us) write_op.cc:270] Start()
0709 12:25:39.425513 (+   121us) write_op.cc:276] Timestamp: P: 1783599939425337 usec, L: 0
0709 12:25:39.425579 (+    66us) op_driver.cc:348] REPLICATION: starting
0709 12:25:39.426556 (+   977us) log.cc:844] Serialized 12473 byte log entry
0709 12:25:39.441263 (+ 14707us) op_driver.cc:464] REPLICATION: finished
0709 12:25:39.442465 (+  1202us) write_op.cc:301] APPLY: starting
0709 12:25:39.442548 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:39.525386 (+ 82838us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:39.529850 (+  4464us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:39.687342 (+157492us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:39.688023 (+   681us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=64,key_file_lookups=64,delta_file_lookups=64,mrs_lookups=0
0709 12:25:39.688094 (+    71us) write_op.cc:312] APPLY: finished
0709 12:25:39.690011 (+  1917us) log.cc:844] Serialized 275 byte log entry
0709 12:25:39.690214 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.690544 (+   330us) write_op.cc:454] Released schema lock
0709 12:25:39.690637 (+    93us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:39.691127 (+   490us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.691164 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16064,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":946,"apply.run_cpu_time_us":39740,"apply.run_wall_time_us":248589,"cfile_cache_hit":128,"cfile_cache_hit_bytes":133696,"num_ops":32,"prepare.queue_time_us":150,"prepare.run_cpu_time_us":6971,"prepare.run_wall_time_us":44790,"replication_time_us":15516}]]}
I20260709 12:25:39.770373  4890 catalog_manager.cc:5697] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 8 to 15, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 15 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:39.775673  4890 catalog_manager.cc:5697] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 14 to 20, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 20 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:39.790046  4890 catalog_manager.cc:5697] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 13 to 18, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 18 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:39.791922  4890 catalog_manager.cc:5697] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 14 to 19, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 19 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.793705  4890 catalog_manager.cc:5697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.795451  4890 catalog_manager.cc:5697] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 16 to 18, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 18 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:39.797464  4890 catalog_manager.cc:5697] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 7 to 20, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 20 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.799643  4890 catalog_manager.cc:5697] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.811336 11105 raft_consensus.cc:993] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: : Instructing follower 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:39.813020 11105 raft_consensus.cc:1081] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 21 LEADER]: Signalling peer 9847a7797ddc4df689a1ab0911af8785 to start an election
I20260709 12:25:39.818174  5942 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138"
dest_uuid: "9847a7797ddc4df689a1ab0911af8785"
 from {username='slave'} at 127.0.0.1:39878
I20260709 12:25:39.819027  5942 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:39.819439  5942 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:39.811676  4890 catalog_manager.cc:5697] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:39.824470  4890 catalog_manager.cc:5697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:39.826391  4890 catalog_manager.cc:5697] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:39.827734  4890 catalog_manager.cc:5697] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 6 to 19. New cstate: current_term: 19 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } }
I20260709 12:25:39.835642  5942 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:39.839856  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:39.840605  5143 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:39.845903  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:39.846741  5220 raft_consensus.cc:3055] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:39.847050  5220 raft_consensus.cc:740] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:39.847883  5220 consensus_queue.cc:260] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:39.849215  5220 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:39.867189  5143 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 22.
I20260709 12:25:39.867897  5942 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 22 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:39.875892  5220 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9847a7797ddc4df689a1ab0911af8785 in term 22.
I20260709 12:25:39.877717  5899 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785; no voters: 
I20260709 12:25:39.884501 10579 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:39.885138 10579 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 22 LEADER]: Becoming Leader. State: Replica: 9847a7797ddc4df689a1ab0911af8785, State: Running, Role: LEADER
I20260709 12:25:39.887079 10579 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:39.943321  4898 catalog_manager.cc:5697] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 17 to 18, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 18 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:39.945379  4898 catalog_manager.cc:5697] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.946902  4898 catalog_manager.cc:5697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 13 to 20, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 20 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } }
I20260709 12:25:40.127777  4891 catalog_manager.cc:5697] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 10 to 16, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 16 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } }
I20260709 12:25:40.131449  4890 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49944 (request call id 82) took 1509 ms. Trace:
I20260709 12:25:40.144158  5220 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:40.141412  4890 rpcz_store.cc:276] 0709 12:25:38.622063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.622337 (+   274us) service_pool.cc:224] Handling call
0709 12:25:39.914533 (+1292196us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.914571 (+    38us) write_op.cc:454] Released schema lock
0709 12:25:40.131334 (+216763us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:39.854823 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:39.855379 (+   556us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:39.855414 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:25:39.855429 (+    15us) tablet.cc:662] Decoding operations
0709 12:25:39.856576 (+  1147us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:39.856635 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:39.856665 (+    30us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:25:39.857891 (+  1226us) tablet.cc:701] Row locks acquired
0709 12:25:39.857914 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:25:39.858103 (+   189us) write_op.cc:270] Start()
0709 12:25:39.858224 (+   121us) write_op.cc:276] Timestamp: P: 1783599939858047 usec, L: 0
0709 12:25:39.858247 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:25:39.859029 (+   782us) log.cc:844] Serialized 5154 byte log entry
0709 12:25:39.862196 (+  3167us) op_driver.cc:464] REPLICATION: finished
0709 12:25:39.862628 (+   432us) write_op.cc:301] APPLY: starting
0709 12:25:39.862733 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:39.873944 (+ 11211us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:39.874099 (+   155us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:39.891155 (+ 17056us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:39.891442 (+   287us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 12:25:39.891495 (+    53us) write_op.cc:312] APPLY: finished
0709 12:25:39.891883 (+   388us) log.cc:844] Serialized 120 byte log entry
0709 12:25:39.892816 (+   933us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.893029 (+   213us) write_op.cc:454] Released schema lock
0709 12:25:39.893095 (+    66us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":52988,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":137,"apply.run_cpu_time_us":16588,"apply.run_wall_time_us":30826,"cfile_cache_hit":52,"cfile_cache_hit_bytes":54314,"num_ops":13,"prepare.queue_time_us":127,"prepare.run_cpu_time_us":4542,"prepare.run_wall_time_us":7842,"replication_time_us":3823}]]}
I20260709 12:25:40.158396 10985 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:40.165303  5147 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:40.168195 10913 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:40.140558  4893 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43226 (request call id 134) took 3176 ms. Trace:
I20260709 12:25:40.170177  4893 rpcz_store.cc:276] 0709 12:25:36.964641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.964990 (+   349us) service_pool.cc:224] Handling call
0709 12:25:40.140265 (+3175275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23366,"reported_tablets":44}
I20260709 12:25:40.238616  4890 catalog_manager.cc:5697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 reported cstate change: term changed from 21 to 22, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65). New cstate: current_term: 22 leader_uuid: "9847a7797ddc4df689a1ab0911af8785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:40.347021  5143 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:40.353328 10862 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:40.412402  4891 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43234 (request call id 129) took 1752 ms. Trace:
I20260709 12:25:40.415623  4891 rpcz_store.cc:276] 0709 12:25:38.660066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.805123 (+145057us) service_pool.cc:224] Handling call
0709 12:25:40.412308 (+1607185us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:40.172219 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:40.172810 (+   591us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:40.172850 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:25:40.172862 (+    12us) tablet.cc:662] Decoding operations
0709 12:25:40.173219 (+   357us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:40.173285 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:40.173313 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:40.173520 (+   207us) tablet.cc:701] Row locks acquired
0709 12:25:40.179310 (+  5790us) write_op.cc:260] PREPARE: finished
0709 12:25:40.179487 (+   177us) write_op.cc:270] Start()
0709 12:25:40.179647 (+   160us) write_op.cc:276] Timestamp: P: 1783599940179436 usec, L: 0
0709 12:25:40.179668 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:25:40.180388 (+   720us) log.cc:844] Serialized 537 byte log entry
0709 12:25:40.183142 (+  2754us) op_driver.cc:464] REPLICATION: finished
0709 12:25:40.183594 (+   452us) write_op.cc:301] APPLY: starting
0709 12:25:40.183695 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:40.184631 (+   936us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:40.184746 (+   115us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:40.185798 (+  1052us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:40.185999 (+   201us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:25:40.186051 (+    52us) write_op.cc:312] APPLY: finished
0709 12:25:40.186262 (+   211us) log.cc:844] Serialized 24 byte log entry
0709 12:25:40.186455 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:40.186556 (+   101us) write_op.cc:454] Released schema lock
0709 12:25:40.186605 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:40.186989 (+   384us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:40.187023 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11629,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":248,"apply.run_cpu_time_us":3361,"apply.run_wall_time_us":3360,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4178,"num_ops":1,"prepare.queue_time_us":549,"prepare.run_cpu_time_us":2741,"prepare.run_wall_time_us":8472,"replication_time_us":3325}]]}
I20260709 12:25:40.469242  5222 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Refusing update from remote peer 9847a7797ddc4df689a1ab0911af8785: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:40.471231 11065 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:40.528378 10699 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:40.529052 10699 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:40.533149  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:40.533310  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:40.533944  5143 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:40.534397  5076 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 17: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:40.535944  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:40.539237 11072 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:40.539717 11072 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:40.540086 11072 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:40.533375 10699 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:40.559334  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:40.604158 11072 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:40.620246  4897 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49944 (request call id 79) took 63548 ms (client timeout 15000 ms). Trace:
W20260709 12:25:40.620798  4897 rpcz_store.cc:269] 0709 12:24:37.072101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.128599 (+ 56498us) service_pool.cc:224] Handling call
0709 12:25:40.620012 (+63491413us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1099648,"reported_tablets":45}
I20260709 12:25:40.623499  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:40.624534  5076 raft_consensus.cc:2393] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 17: Already voted for candidate 6dc0d06e662d49d88e1b153e09a44e4f in this term.
I20260709 12:25:40.626215  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:40.636449  5143 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:40.626264 11072 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:40.644160  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:40.645066 11072 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:40.645982 11072 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:40.647058 11072 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:493: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 4
I20260709 12:25:40.688751  1517 tablet_server.cc:179] TabletServer@127.1.123.65:37253 shutting down...
W20260709 12:25:40.736444  5115 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
I20260709 12:25:40.759348  4898 catalog_manager.cc:5697] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 16 to 17, leader changed from ceec344dd00f40c18ef58414ce147016 (127.1.123.67) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 17 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:40.774037 10819 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.775031 10819 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:40.786684 10819 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:40.792699  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 56281us
I20260709 12:25:40.797111  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:40.800884  5145 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 17.
I20260709 12:25:40.811756  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:40.816115  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 77991.9us
I20260709 12:25:40.823624 10985 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:40.837759 10985 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:40.856618  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44914.4us
I20260709 12:25:40.846340 10985 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:40.875111  5115 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:40.907884 10985 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:40.927281  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:40.941464  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:40.942490  5147 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:40.945974 10985 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:40.940480  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:40.941170  5115 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:40.957197  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
W20260709 12:25:40.959930  5115 consensus_peers.cc:597] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:40.962432  5041 consensus_peers.cc:597] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:40.963033  5041 consensus_peers.cc:597] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:40.977955  5147 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 18.
I20260709 12:25:40.981045  5042 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac
W20260709 12:25:40.986339  5041 consensus_peers.cc:597] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:40.988252  5115 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:40.983719 10985 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:25:40.990797  5041 consensus_peers.cc:597] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.015086  5041 consensus_peers.cc:597] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.045027  5041 consensus_peers.cc:597] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.048692  5899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 64690.5us
W20260709 12:25:41.054718  5041 consensus_peers.cc:597] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.068581  5189 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.077234  5115 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:41.074172 10985 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
I20260709 12:25:41.084578 10985 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:41.097479  5041 consensus_peers.cc:597] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:41.099208  5147 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 17. Current term is 18. Ops: []
W20260709 12:25:41.098137  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.101471  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.104853  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.107630  5189 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.112830  5115 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:41.105890 10699 consensus_queue.cc:1059] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:25:41.119451 10699 raft_consensus.cc:3055] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:41.119956 10699 raft_consensus.cc:740] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:41.120786 10699 consensus_queue.cc:260] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:41.122279 10699 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:41.142098  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.143165  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.161680  5189 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.164798  5041 consensus_peers.cc:597] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:41.188040  5076 raft_consensus.cc:1240] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Rejecting Update request from peer 90e5220a62ed42fe83c82d0302d18fac for earlier term 17. Current term is 18. Ops: []
W20260709 12:25:41.197300  5189 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.198910  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 81895.3us
W20260709 12:25:41.216903  4892 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49944 (request call id 81) took 17945 ms (client timeout 15000 ms). Trace:
W20260709 12:25:41.217562  4892 rpcz_store.cc:269] 0709 12:25:23.237916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.238559 (+   643us) service_pool.cc:224] Handling call
0709 12:25:41.183330 (+17944771us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33356,"reported_tablets":45}
W20260709 12:25:41.203114  5189 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.227020  5189 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.230074  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.235695  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:41.252427  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:25:41.261341  5115 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:41.262303  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.263324  1517 raft_consensus.cc:2243] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Raft consensus shutting down.
W20260709 12:25:41.264189  5041 consensus_peers.cc:597] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:41.264497  1517 raft_consensus.cc:2272] T c5751113d37b44af87d3d912da388078 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:41.269200  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:41.273092  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.274217  1517 raft_consensus.cc:2243] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.282790  1517 raft_consensus.cc:2272] T 874aa8edc94946d69e4a25563a5a0f0f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.292979  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.294031  1517 raft_consensus.cc:2243] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.295274  1517 raft_consensus.cc:2272] T c0a39e3f43214d8ea4ff72401ffdbdcb P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.300690  4892 catalog_manager.cc:5697] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 17 to 18, leader changed from 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 18 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:41.305156  4895 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43226 (request call id 133) took 21545 ms (client timeout 15000 ms). Trace:
W20260709 12:25:41.316339  4895 rpcz_store.cc:269] 0709 12:25:19.759688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.760231 (+   543us) service_pool.cc:224] Handling call
0709 12:25:41.304928 (+21544697us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":496049,"reported_tablets":44}
I20260709 12:25:41.323943  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.325062  1517 raft_consensus.cc:2243] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.326301  1517 raft_consensus.cc:2272] T 9d055165bf9e4bfdb042cd7bd63ec993 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.329767  1517 tablet_replica.cc:333] stopping tablet replica
W20260709 12:25:41.341982  4899 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43226 (request call id 132) took 45204 ms (client timeout 15000 ms). Trace:
W20260709 12:25:41.342519  4899 rpcz_store.cc:269] 0709 12:24:56.137672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.138091 (+   419us) service_pool.cc:224] Handling call
0709 12:25:41.341745 (+45203654us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1432360,"reported_tablets":44}
I20260709 12:25:41.410722  1517 raft_consensus.cc:2243] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.411672  1517 raft_consensus.cc:2272] T f6a42608cf874aa6943aaf8e349714b9 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.429797  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.430819  1517 raft_consensus.cc:2243] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.431841  1517 raft_consensus.cc:2272] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 9847a7797ddc4df689a1ab0911af8785 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.444128  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.448865  1517 raft_consensus.cc:2243] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.450201  1517 raft_consensus.cc:2272] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.468750  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.470654  1517 raft_consensus.cc:2243] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.472180  1517 raft_consensus.cc:2272] T 5e15328865c9479dab64cbc3fa161e86 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.494827  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.495870  1517 raft_consensus.cc:2243] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.498603  1517 raft_consensus.cc:2272] T 77f30d7c9f3244999f248bc1aed226c7 P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.514281  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.525916  1517 raft_consensus.cc:2243] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.526712  1517 raft_consensus.cc:2272] T 59bc6f04e62a431ba2f113d08a129898 P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.529897  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.542930  1517 raft_consensus.cc:2243] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.543881  1517 raft_consensus.cc:2272] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.554303  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.557338  1517 raft_consensus.cc:2243] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.566280  1517 raft_consensus.cc:2272] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.576776  5147 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:41.585366  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.586467  1517 raft_consensus.cc:2243] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.587334  1517 raft_consensus.cc:2272] T 159b5709d37c488d935b1fe85d29c5bc P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.590785  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.591778  1517 raft_consensus.cc:2243] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.592778  1517 raft_consensus.cc:2272] T 85f48767a7ee419197aff24610838e25 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.596374  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.597340  1517 raft_consensus.cc:2243] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.598253  1517 raft_consensus.cc:2272] T e77063f082034d8bbc280357b7b66bb7 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.601605  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.602463  1517 raft_consensus.cc:2243] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.603237  1517 raft_consensus.cc:2272] T 9ee83a16af0a40998fc58f3e882c6080 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.604892 10819 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:41.620560  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.621675  1517 raft_consensus.cc:2243] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.637691  1517 raft_consensus.cc:2272] T a6f50ca0c2db44278637fcdffffc03a2 P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.669971  5222 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:25:41.670604  5222 pending_rounds.cc:85] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Aborting all ops after (but not including) 10
I20260709 12:25:41.671856  5222 pending_rounds.cc:107] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:25:41.672245  5222 raft_consensus.cc:2889] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:41.674969 10913 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:41.680449 10856 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:41.681167 10856 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:41.693562  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:41.695217  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:41.696578  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.710361  1517 raft_consensus.cc:2243] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.685518 10856 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:41.718106  1517 raft_consensus.cc:2272] T 2725c773e7344020b5752861d00b493e P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.726408  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.725272 10856 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:41.731513  1517 raft_consensus.cc:2243] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.732471  1517 raft_consensus.cc:2272] T 5deada35125f4c008544b2ff246b039c P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.741312  5898 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:41.742451 10579 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:41.732384 10856 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:41.752463 10856 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:41.754977  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:41.759123 10579 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:41.759737 10579 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:41.764026 10579 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:41.776664  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:41.749918  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.778770  1517 raft_consensus.cc:2243] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.780014  1517 raft_consensus.cc:2272] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.783306  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.784196  1517 raft_consensus.cc:2243] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.784842  1517 raft_consensus.cc:2272] T 1339c2a0d7f344d09f483e6d965adcc8 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.786834  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719a453defab43d4a1232c664dbaae9c" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:41.788923  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15dbeb6b54ba4f0c80adc43d8f184cb5" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:41.789328  5898 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:41.790378 11141 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:41.791606  5898 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:41.792593 11141 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:41.815919  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.822404  1517 raft_consensus.cc:2243] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.825407  1517 raft_consensus.cc:2272] T 94a5838186ec4283b67fe9aca66d2a22 P 9847a7797ddc4df689a1ab0911af8785 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.838593  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.848276  1517 raft_consensus.cc:2243] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.855348  1517 raft_consensus.cc:2272] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.866787  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.867707  1517 raft_consensus.cc:2243] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.869829  1517 raft_consensus.cc:2272] T 805537066aa5440cbb0608aa77daafcf P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.876824  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.893121 11141 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:41.894243 11141 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:41.894695  1517 raft_consensus.cc:2243] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.895347  1517 raft_consensus.cc:2272] T 719a453defab43d4a1232c664dbaae9c P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.896705 11141 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:41.899009  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:41.904946  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4935fc8f35664324a253df9804c8074d" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:41.906786  5898 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:41.908241 11141 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:41.925822  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.926937  1517 raft_consensus.cc:2243] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.928068  1517 raft_consensus.cc:2272] T 5de3ceedceac4e0ca06de7a806ec05e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.931351  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.938251  1517 raft_consensus.cc:2243] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.942405  1517 raft_consensus.cc:2272] T 4da8de53d0db43f9a92906a98dfda98f P 9847a7797ddc4df689a1ab0911af8785 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.952363  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.962236  1517 raft_consensus.cc:2243] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.963120  1517 raft_consensus.cc:2272] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.969702 11141 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:41.970502 11141 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:41.972473  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:41.973404  1517 raft_consensus.cc:2243] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.974577  1517 raft_consensus.cc:2272] T 35db327bbc4d4777969286cb1174a85d P 9847a7797ddc4df689a1ab0911af8785 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.976294 11141 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:41.977478 10856 raft_consensus.cc:493] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:41.978123 10856 raft_consensus.cc:515] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:41.989883 11141 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:41.990962 11141 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:42.010100 11141 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:42.011492 10579 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:42.014964 10579 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:42.017067 10579 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:42.011308  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:42.011544 10856 leader_election.cc:290] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:42.010242  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.013029  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:42.013974  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.019809  1517 raft_consensus.cc:2243] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.020541  1517 raft_consensus.cc:2272] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:42.010223  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.033933  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:42.035892  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516325f133e4ea8beacc1968f96ee9b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.036275  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.041209  5898 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:42.041443  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:42.042855 10856 raft_consensus.cc:493] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:42.043524 10856 raft_consensus.cc:515] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.044945  5900 leader_election.cc:304] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.045859 11141 raft_consensus.cc:2749] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:42.046312  5900 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.047138 11141 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:42.047329 10856 leader_election.cc:290] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:42.050483 11141 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:42.051045 11141 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:42.054541 10856 raft_consensus.cc:493] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:42.055285 10856 raft_consensus.cc:515] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:42.043368 10579 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:42.059517  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.060534 10579 raft_consensus.cc:493] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:42.060731 10856 leader_election.cc:290] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:42.061091 10579 raft_consensus.cc:515] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.062307  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.063185 10579 leader_election.cc:290] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:42.058298 11141 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:42.065050  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.066882  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003133c544c3449a827e3758faa02f02" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.067471  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.068292  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:42.069191  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15944aba8caf4c1497501efd45dc0430" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:42.076169 11144 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:42.076929 11144 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:42.084851 11144 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:42.081815  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf661d015f0e4fc19e57069da58329e3" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.089123  5900 leader_election.cc:304] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.080467  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.090770 11145 raft_consensus.cc:2749] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:42.092338  5898 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.093899  5898 leader_election.cc:304] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.095080  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:42.091796  1517 raft_consensus.cc:2243] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.097522 10856 raft_consensus.cc:2749] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:42.097880 11145 raft_consensus.cc:2706] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Received pre-election callback for term 19 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:25:42.098304 10856 raft_consensus.cc:493] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:42.099053 10856 raft_consensus.cc:515] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:42.101984 10856 leader_election.cc:290] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:42.103341  5898 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:42.105110  5900 leader_election.cc:304] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.106492 10856 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:42.107146 11144 raft_consensus.cc:2749] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:42.134647  1517 raft_consensus.cc:2272] T 09749d11015f4695b5c630c6e5dcbcb0 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:42.271924  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 165330us
I20260709 12:25:42.266659 11144 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:42.274238 11144 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.267702 11145 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:42.278049 11145 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.281669 11145 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:42.277405 11144 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:42.272974  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.289388  1517 raft_consensus.cc:2243] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.290465  1517 raft_consensus.cc:2272] T 2b6b7243efc34b21882fab443d3a3720 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:42.306610  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.309888  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:42.312572  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:42.315618  1517 raft_consensus.cc:2243] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.340514  1517 raft_consensus.cc:2272] T c1c232e09b414797838d5e57d028ca97 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:42.348394  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 100115us
I20260709 12:25:42.349794  5144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e6663f57ec46609e0512d07df6f705" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.350042  5143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d065b39ba44699c339e80bcc02744" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.365486  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.366647  5900 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.370133  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.381845  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb1b479abeca4781986d8821b704c31f" candidate_uuid: "9847a7797ddc4df689a1ab0911af8785" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.386716  5899 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac
I20260709 12:25:42.378322 11144 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:42.390662  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.399811 11145 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:42.406533  1517 raft_consensus.cc:2243] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:25:42.440802  1517 raft_consensus.cc:2272] T 0141a059bf6949c49e5e0091604786e6 P 9847a7797ddc4df689a1ab0911af8785 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:42.456084  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 91161.5us
I20260709 12:25:42.460160  5900 leader_election.cc:304] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.463020  5900 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9847a7797ddc4df689a1ab0911af8785; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.467911 10856 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:42.467489 11145 raft_consensus.cc:2749] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:42.561286  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 74480.6us
I20260709 12:25:42.561661 11154 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.562479 11154 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:42.565073 11154 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:42.568250 11148 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.610052 11148 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.612717 11148 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:42.572651  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 106043us
I20260709 12:25:42.578593  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:42.582263  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.615247  1517 raft_consensus.cc:2243] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.616093  1517 raft_consensus.cc:2272] T 552d065b39ba44699c339e80bcc02744 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:42.585754  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 120379us
I20260709 12:25:42.587438 11160 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.621886  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.622303 11149 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.622092 11160 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.624626  5141 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 20.
I20260709 12:25:42.625046 11160 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.625897 11149 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.628412  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.628609 11149 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.641067  1517 raft_consensus.cc:2243] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.646060  1517 raft_consensus.cc:2272] T cf661d015f0e4fc19e57069da58329e3 P 9847a7797ddc4df689a1ab0911af8785 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:42.653117  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.657231  1517 raft_consensus.cc:2243] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.657823  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:42.663436  5189 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.665251  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.658703  5222 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 20.
I20260709 12:25:42.671501  1517 raft_consensus.cc:2272] T 4935fc8f35664324a253df9804c8074d P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:42.674918  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:42.664374  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:42.669057 11157 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.676216 11160 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:42.677347 11157 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.678018 11160 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.678560 11160 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:42.679872 11157 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.686269  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:42.688643 11148 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:42.689190 11148 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.689579 11148 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:42.695354  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.697096 11157 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.698446 11159 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.699157 11159 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:42.701746 11159 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.703210 11149 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:42.706826  5041 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.697954 11157 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.696465  1517 raft_consensus.cc:2243] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.710369 11157 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.711306 11149 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.711979 11149 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.710391  1517 raft_consensus.cc:2272] T 003133c544c3449a827e3758faa02f02 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:42.714964  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.717121  5222 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 14.
W20260709 12:25:42.717854  5189 leader_election.cc:336] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.719213  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.726429  1517 raft_consensus.cc:2243] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.719434 11149 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.720788  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.727242  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:42.728788 11159 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:42.729310 11159 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.729780 11159 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:42.727303  1517 raft_consensus.cc:2272] T eb1b479abeca4781986d8821b704c31f P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:42.736999  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.738058  1517 raft_consensus.cc:2243] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.738783  1517 raft_consensus.cc:2272] T 01e6663f57ec46609e0512d07df6f705 P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:42.745620  5189 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.746997  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.746909 11160 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.758181  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.759230  1517 raft_consensus.cc:2243] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.754132 11148 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.757081  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.758026 11149 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:42.762295  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.769501  5222 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:42.759996  1517 raft_consensus.cc:2272] T bfdb4c20c33743d79d57552db35c484b P 9847a7797ddc4df689a1ab0911af8785 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:42.762831  5189 leader_election.cc:336] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.769091 11149 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.772794 11149 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.759950 11160 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.767870 11159 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:42.779536 11148 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.779186 11149 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.779881  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.781620  1517 raft_consensus.cc:2243] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.782416  1517 raft_consensus.cc:2272] T 15944aba8caf4c1497501efd45dc0430 P 9847a7797ddc4df689a1ab0911af8785 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:42.786394  5041 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.787038  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.789988  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5751113d37b44af87d3d912da388078" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:42.790740  5222 raft_consensus.cc:3060] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:42.792953  1517 raft_consensus.cc:2243] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.793982  1517 raft_consensus.cc:2272] T 4516325f133e4ea8beacc1968f96ee9b P 9847a7797ddc4df689a1ab0911af8785 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:42.787132 11159 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.798064  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:42.799329  1517 raft_consensus.cc:2243] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:42.800654  1517 raft_consensus.cc:2272] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 9847a7797ddc4df689a1ab0911af8785 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:42.849767  1517 tablet_replica.cc:333] stopping tablet replica
W20260709 12:25:42.849879  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 44800.3us
I20260709 12:25:42.850767  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:42.850814  1517 raft_consensus.cc:2243] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:42.852270  1517 raft_consensus.cc:2272] T 63938e2a81834411b8018a2eb6c6818f P 9847a7797ddc4df689a1ab0911af8785 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:42.853901 11126 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:42.854267  5190 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.854521  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:42.855366  5223 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ceec344dd00f40c18ef58414ce147016 in current term 21: Already voted for candidate 90e5220a62ed42fe83c82d0302d18fac in this term.
I20260709 12:25:42.854413 11126 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.855635 11148 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:42.855971 11126 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:42.863420  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:42.864709  5149 raft_consensus.cc:2393] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 90e5220a62ed42fe83c82d0302d18fac in current term 21: Already voted for candidate ceec344dd00f40c18ef58414ce147016 in this term.
I20260709 12:25:42.865396  5222 raft_consensus.cc:2468] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc0d06e662d49d88e1b153e09a44e4f in term 15.
I20260709 12:25:42.865830  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.866577  5147 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 22.
W20260709 12:25:42.873395  5115 leader_election.cc:336] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:25:42.873742  5189 leader_election.cc:336] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.873914  5041 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:42.875445 11161 raft_consensus.cc:2804] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:42.876040 11161 raft_consensus.cc:697] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Becoming Leader. State: Replica: 6dc0d06e662d49d88e1b153e09a44e4f, State: Running, Role: LEADER
W20260709 12:25:42.876307  5115 leader_election.cc:336] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.876994 11161 consensus_queue.cc:237] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:42.879885  5115 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:42.881435 11152 raft_consensus.cc:2749] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:42.884140 11159 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.884863 11159 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:42.862556 11157 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.885859 11157 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.889074 11159 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:42.876159  5189 leader_election.cc:336] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.895666  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:42.897730 11146 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:42.897302  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:42.899923  5041 leader_election.cc:336] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.901141 11146 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:42.901599 11146 raft_consensus.cc:493] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.901978 11146 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:42.902362  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.903896  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:42.904724 11156 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:42.900970  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.887989 11157 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.919157 11126 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.923138 11155 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.923834 11155 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.926301 11155 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.927536 11126 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:42.928870 11146 raft_consensus.cc:515] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.932360 11146 leader_election.cc:290] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:42.943785  5115 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:25:42.944845  5115 leader_election.cc:336] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.948071  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:42.952527 11146 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.953339 11146 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:42.956406 11146 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:42.956537  5189 leader_election.cc:336] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:42.957672 11126 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:42.962203  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:42.962987  5222 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 21.
I20260709 12:25:42.972553  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d055165bf9e4bfdb042cd7bd63ec993" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:42.973258  5223 raft_consensus.cc:3060] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:42.976466  5149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3665d1ab30ab4c8d9b7304c26a0cd138" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:42.977303  5149 raft_consensus.cc:3060] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:42.977975  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:42.978703  5147 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 19.
I20260709 12:25:42.982540  5223 raft_consensus.cc:2468] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 20.
I20260709 12:25:42.989236  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:42.988438  5149 raft_consensus.cc:2468] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 23.
I20260709 12:25:42.991725 11126 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:42.992223 11126 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:42.992652 11126 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:42.997754  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:42.999742 11149 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:43.000175 11149 raft_consensus.cc:493] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.000555 11149 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:43.004669 11126 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:43.008235  5115 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:43.009611 11152 raft_consensus.cc:2804] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:43.010178 11152 raft_consensus.cc:697] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 20 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:43.011145 11152 consensus_queue.cc:237] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:43.021653  5115 leader_election.cc:336] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:43.022169  5115 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:43.024178 11126 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:43.030447 11155 raft_consensus.cc:2749] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:43.033937  5190 leader_election.cc:304] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:43.033815  4899 catalog_manager.cc:5697] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f reported cstate change: term changed from 14 to 15, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66). New cstate: current_term: 15 leader_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:43.036873 11146 raft_consensus.cc:2804] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:43.039307 11149 raft_consensus.cc:515] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:43.041617 11146 raft_consensus.cc:697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 23 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:43.050180  4892 catalog_manager.cc:5697] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 19 to 20, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 20 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:43.051810 11148 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.052434 11148 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:43.056025  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:43.056726  5068 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 14.
I20260709 12:25:43.058140  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:43.058957 11154 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:43.059357 11154 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.059818 11154 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:43.062438  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6732ed2583444ab53a0bd4a87ff1d0" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:43.063263  5222 raft_consensus.cc:3060] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:43.059280  5189 leader_election.cc:336] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:43.075671  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e15328865c9479dab64cbc3fa161e86" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:43.076333  5142 raft_consensus.cc:3060] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:43.078135  5115 leader_election.cc:336] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:25:43.080881  5115 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:43.058964 11148 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:43.087648  5222 raft_consensus.cc:2468] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 22.
I20260709 12:25:43.043083 11146 consensus_queue.cc:237] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:43.096447 11128 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.098892  5142 raft_consensus.cc:2468] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 20.
I20260709 12:25:43.101176  5115 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:43.101203  5190 leader_election.cc:304] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:43.102169 11126 raft_consensus.cc:2804] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:25:43.102229  5115 leader_election.cc:336] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:43.102860 11126 raft_consensus.cc:697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 22 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:43.097239 11128 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:43.103842 11126 consensus_queue.cc:237] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:43.105958 11128 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:43.106448 11148 raft_consensus.cc:2804] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:43.107158 11148 raft_consensus.cc:697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 20 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:43.110493 11148 consensus_queue.cc:237] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:43.109088  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:43.087353 11161 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.114212 11161 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:43.130327 11161 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:43.130986 11154 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:43.133647  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:43.139948 11161 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.140630 11161 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:43.141644 11149 leader_election.cc:290] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 election: Requested vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:43.147821 11161 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:43.148325  5189 leader_election.cc:336] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:43.150954  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:43.146831  5041 leader_election.cc:336] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:43.154639  5041 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:43.155890  5189 leader_election.cc:336] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:25:43.158591  5189 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:43.159118  5189 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:43.161393 11161 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:25:43.165733  5041 consensus_peers.cc:597] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.166827 11159 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.167603 11135 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:43.167469 11159 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:43.169873 11159 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:43.171718  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:43.172322 11154 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:43.169934  5041 leader_election.cc:336] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:43.173838  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:43.175022  5189 leader_election.cc:336] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:43.173995  5041 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:43.175972 11152 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.176173 11159 raft_consensus.cc:2749] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:43.176659 11152 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:43.175087  5068 raft_consensus.cc:3060] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:43.188684  5041 connection.cc:573] client connection to 127.1.123.65:37253 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:25:43.190793  5041 leader_election.cc:336] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 12:25:43.191264  5041 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:43.193183 11161 raft_consensus.cc:2749] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:43.193894 11152 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:43.195730  4895 catalog_manager.cc:5697] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 22 to 23, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 23 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:43.205502  5068 raft_consensus.cc:2468] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 15.
I20260709 12:25:43.219807 11126 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.220546 11126 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:43.222304  5188 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:43.216652 11152 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.223819 11154 raft_consensus.cc:2804] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:43.224068 11152 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:43.226313 11152 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:43.228826 11126 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:43.230804  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:43.231453  5076 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:43.232926  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:43.224505 11154 raft_consensus.cc:697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
W20260709 12:25:43.234074  5189 connection.cc:573] client connection to 127.1.123.65:37253 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 12:25:43.237442 11154 consensus_queue.cc:237] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:43.240470  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:43.241464  5068 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 20.
I20260709 12:25:43.239503 11126 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:43.243692 11126 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.247749  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:43.246729 11126 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:43.249994 11152 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:43.257416  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:43.258425  5223 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:43.256644 11152 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.278950 11152 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:43.307255 11152 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:43.311678 11152 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:43.311884  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:43.312999  5068 raft_consensus.cc:3060] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:43.320222  5068 raft_consensus.cc:2468] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 21.
W20260709 12:25:43.334060  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.335054 11135 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.335605 11135 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:43.337690  5223 raft_consensus.cc:1275] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Refusing update from remote peer 6dc0d06e662d49d88e1b153e09a44e4f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:25:43.337759 11135 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:43.339676  5068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:43.340323  5068 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:43.341718  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:43.343436  5114 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:43.344916 11155 raft_consensus.cc:2804] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:43.345341 11146 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:43.346184 11155 raft_consensus.cc:697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 21 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:43.355010 11155 consensus_queue.cc:237] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:43.362684 11156 consensus_queue.cc:1048] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:43.361367  4895 catalog_manager.cc:5697] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 21 to 22, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 22 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:43.354291 11146 raft_consensus.cc:493] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.369115 11146 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:43.373236  5189 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.377328  5189 leader_election.cc:336] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:43.381991  5041 consensus_peers.cc:597] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:43.384202  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 110129us
I20260709 12:25:43.388105  4892 catalog_manager.cc:5697] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 19 to 20, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 20 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:43.391290 11126 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:43.405653  5041 consensus_peers.cc:597] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.407400  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:43.408059  4892 catalog_manager.cc:5697] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 14 to 15, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 15 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:43.414674 11126 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:43.420893  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:43.421870  5071 raft_consensus.cc:3060] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:43.412428 11157 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:43.428345 11157 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.428788 11157 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:43.439592 11146 raft_consensus.cc:515] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:43.451231  5071 raft_consensus.cc:2468] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:43.453246  5114 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016; no voters: 
W20260709 12:25:43.465102  5041 consensus_peers.cc:597] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.456931  5115 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:43.466398  5115 leader_election.cc:336] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:43.466908 11160 raft_consensus.cc:2804] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:43.467662 11160 raft_consensus.cc:697] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
W20260709 12:25:43.473248  5189 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.473832  5189 leader_election.cc:336] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:43.468786 11160 consensus_queue.cc:237] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:43.475854  5115 leader_election.cc:336] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:43.482056  4895 catalog_manager.cc:5697] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 20 to 21, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 21 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:43.499132  5041 consensus_peers.cc:597] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.500415 11157 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:43.502697 11157 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:43.514024 11146 leader_election.cc:290] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:43.513713  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.521428  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da8de53d0db43f9a92906a98dfda98f" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
I20260709 12:25:43.523787  5071 raft_consensus.cc:3060] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:43.534502  5041 consensus_peers.cc:597] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.544203 11159 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.544879 11159 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:43.547364 11159 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:43.557468  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:43.558388  5041 leader_election.cc:336] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:43.560284  5071 raft_consensus.cc:2468] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
I20260709 12:25:43.561641  5188 leader_election.cc:304] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:43.562354  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.563045 11146 raft_consensus.cc:2804] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:43.564607 11146 raft_consensus.cc:697] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:43.566201 11146 consensus_queue.cc:237] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:43.568459  5041 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:43.582883 11159 raft_consensus.cc:2749] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:25:43.619692  5189 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.636797  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.646411  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35db327bbc4d4777969286cb1174a85d" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:43.645994  4895 catalog_manager.cc:5697] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 16 to 17, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 17 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:43.647080  5223 raft_consensus.cc:3060] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:43.655525  5041 consensus_peers.cc:597] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.661244  5223 raft_consensus.cc:2468] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 20.
I20260709 12:25:43.664012  4892 catalog_manager.cc:5697] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 15 to 16, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 16 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:43.677292  5189 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:43.707492  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:43.710423  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.719861  5041 consensus_peers.cc:597] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.737342  5141 raft_consensus.cc:1275] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 23 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:25:43.744675  5141 raft_consensus.cc:1275] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 12:25:43.757834  5041 consensus_peers.cc:597] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.763875  5115 leader_election.cc:336] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:43.765069  5115 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.765655  5115 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.769874  5115 leader_election.cc:336] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:43.792128  5222 raft_consensus.cc:1275] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:43.798309  5222 raft_consensus.cc:1275] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:25:43.801715  5189 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:43.804147 11146 consensus_queue.cc:1048] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:43.816293  5115 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:43.817737 11157 raft_consensus.cc:2804] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:43.818297 11157 raft_consensus.cc:697] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 20 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:43.819507  5071 raft_consensus.cc:1275] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:43.819284 11157 consensus_queue.cc:237] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:43.823009 11149 consensus_queue.cc:1048] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:43.828946 11146 consensus_queue.cc:1048] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:43.839010 11160 consensus_queue.cc:1048] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:43.837029 11155 consensus_queue.cc:1048] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:25:43.857136  5115 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.857770  5115 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:43.858553  5115 consensus_peers.cc:597] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.859069  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.873117  4895 catalog_manager.cc:5697] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 19 to 20, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 20 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:43.898837 11146 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.899641 11146 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:43.911008 11135 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.911727 11135 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:43.931859 11135 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:43.937867  5071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" is_pre_election: true
I20260709 12:25:43.938700  5071 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 21.
I20260709 12:25:43.941210  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 
I20260709 12:25:43.943643  5071 raft_consensus.cc:1275] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:25:43.952505 11157 consensus_queue.cc:1048] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:43.966455  5076 raft_consensus.cc:1275] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:43.968999 11157 consensus_queue.cc:1048] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:43.972177 11135 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:43.972741 11135 raft_consensus.cc:493] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:43.973114 11135 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:43.981411  5115 consensus_peers.cc:597] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:43.982334  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:43.982429  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:43.985877 11146 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:43.996197  5041 consensus_peers.cc:597] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.997136  5041 consensus_peers.cc:597] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.982916  5115 consensus_peers.cc:597] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:44.018513  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 77390.7us
W20260709 12:25:44.066871  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:44.106463  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 72065.3us
W20260709 12:25:44.147305  5115 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:44.147823  5115 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:44.153378 11135 raft_consensus.cc:515] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:44.176692 11135 leader_election.cc:290] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 election: Requested vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:44.178884 11187 log.cc:927] Time spent T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: Append to log took a long time: real 0.113s	user 0.000s	sys 0.003s
W20260709 12:25:44.234191  5189 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:44.234838  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 99984.8us
I20260709 12:25:44.249243  5071 raft_consensus.cc:1275] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:25:44.253643  5115 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:44.254066  5076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0141a059bf6949c49e5e0091604786e6" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6dc0d06e662d49d88e1b153e09a44e4f"
W20260709 12:25:44.282240  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 128840us
I20260709 12:25:44.282670  5076 raft_consensus.cc:3060] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:44.260066  1517 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 5970)
W20260709 12:25:44.278723  5189 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:44.306219  5189 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:44.306680  5189 leader_election.cc:336] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:44.307101  5189 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:44.312729 11135 consensus_queue.cc:1048] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:44.315380 11149 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:44.316207  5189 leader_election.cc:336] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:44.320132  5115 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:44.347023  5115 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:44.347899  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:44.348569  5189 leader_election.cc:336] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:44.364696 11157 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
W20260709 12:25:44.365713  5115 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:44.365459 11157 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:44.366356  5076 raft_consensus.cc:2468] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 22.
I20260709 12:25:44.368615 11157 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:44.380463  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:44.381213  5220 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:44.388263  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:44.394237 11157 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:44.394765 11157 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9847a7797ddc4df689a1ab0911af8785)
I20260709 12:25:44.395184 11157 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:44.398895  5115 leader_election.cc:336] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:44.403519  5189 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:44.413499 11157 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:44.425300  5223 raft_consensus.cc:1275] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:25:44.427690 11205 consensus_queue.cc:1048] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:44.439031  5188 leader_election.cc:304] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f, 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:44.440415 11149 raft_consensus.cc:2804] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:44.441476 11149 raft_consensus.cc:697] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 22 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:44.443215 11149 consensus_queue.cc:237] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:44.447636 11157 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:44.449090  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb4fbb07495443bac93cfbcfa13bbe2" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:44.449941  5222 raft_consensus.cc:3060] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:44.454898  5115 leader_election.cc:336] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:44.481899  5222 raft_consensus.cc:2468] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:44.483618  4895 catalog_manager.cc:5697] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 21 to 22, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 22 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:44.488021  5115 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:44.492584 11200 raft_consensus.cc:2804] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:44.493234 11200 raft_consensus.cc:697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 19 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:44.494529 11200 consensus_queue.cc:237] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:44.594376 11200 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.595059 11200 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:44.598938 11200 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:44.604408  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:44.605388  5222 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 21.
I20260709 12:25:44.635872  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:44.644323 11200 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:44.644883 11200 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.645232 11200 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:44.654281  5115 leader_election.cc:336] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:44.687719 11200 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:44.705727 11200 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 election: Requested vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:44.711580  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a39e3f43214d8ea4ff72401ffdbdcb" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:44.712644  5222 raft_consensus.cc:3060] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:44.725524  4895 catalog_manager.cc:5697] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 18 to 19, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 19 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:44.730423  5222 raft_consensus.cc:2468] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 22.
W20260709 12:25:44.739945  5115 leader_election.cc:336] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:44.753228  5115 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:44.759423 11200 raft_consensus.cc:2804] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:44.760064 11200 raft_consensus.cc:697] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 22 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:44.761030 11200 consensus_queue.cc:237] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:25:44.803177  4895 catalog_manager.cc:5697] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 20 to 22, leader changed from 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 22 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:44.948321  5068 raft_consensus.cc:1275] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 22 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:25:44.952260 11193 consensus_queue.cc:1048] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:44.963486  5222 raft_consensus.cc:1275] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:25:44.979895 11200 consensus_queue.cc:1048] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:25:45.046876  5189 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:45.087258  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:45.149915  5222 raft_consensus.cc:1275] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:25:45.153044 11200 consensus_queue.cc:1048] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:45.286481  1517 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 5970)
W20260709 12:25:45.346415  5115 consensus_peers.cc:597] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:45.533439  5041 consensus_peers.cc:597] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:45.648613  5041 consensus_peers.cc:597] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:45.680159  5041 consensus_peers.cc:597] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:45.721900  5115 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:45.746524  5041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111) [suppressed 395 similar messages]
W20260709 12:25:45.805748  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:45.821735  5189 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:45.862464  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:45.896212  5115 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:45.938484  5041 consensus_peers.cc:597] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:45.997576  5189 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:46.031474  5041 consensus_peers.cc:597] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.074565  5041 consensus_peers.cc:597] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.133282  5115 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.140856  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.145017  5041 consensus_peers.cc:597] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.163456  5189 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.189234  5115 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.194067  5189 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.198062  5115 consensus_peers.cc:597] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.217901  5041 consensus_peers.cc:597] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.245368  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.245692  5115 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:46.245452  5041 consensus_peers.cc:597] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.266347  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.285532  5189 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.287912  1517 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 5970)
W20260709 12:25:46.293915  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.374001  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:46.379926  5041 consensus_peers.cc:597] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.382923  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.419940  5115 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.434602  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.459204  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.502987  5189 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:46.578303  5115 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:46.585314  5115 consensus_peers.cc:597] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:46.624116  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 51918.1us
W20260709 12:25:46.629614  5189 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.647944  5115 consensus_peers.cc:597] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:46.743515  5041 consensus_peers.cc:597] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.864563  5189 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:46.882807  5115 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:46.920840  5189 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:46.925447  5115 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:47.288756  1517 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 5970)
W20260709 12:25:47.594518  5115 consensus_peers.cc:597] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:47.691500  5189 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:47.810679  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:47.875756  5041 consensus_peers.cc:597] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:47.904247  5041 consensus_peers.cc:597] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:47.980203  5189 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.110458  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.249671  5041 consensus_peers.cc:597] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:48.290352  1517 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 5970)
W20260709 12:25:48.292131  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.315201  5041 consensus_peers.cc:597] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.350275  5041 consensus_peers.cc:597] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.405737  5041 consensus_peers.cc:597] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.454043  5189 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:48.469887  5115 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:48.470798  5115 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.471354  5115 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.579458  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.581162  5041 consensus_peers.cc:597] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.611001  5041 consensus_peers.cc:597] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.611994  5189 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.641911  5115 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.675239  5115 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.723810  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.744508  5115 consensus_peers.cc:597] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.750126  5189 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.818572  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.855513  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.874187  5041 consensus_peers.cc:597] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.880802  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.898118  5189 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.926528  5041 consensus_peers.cc:597] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.936362  5189 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.955720  5115 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.969663  5115 consensus_peers.cc:597] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:48.991860  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.994760  5041 consensus_peers.cc:597] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:48.999984  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:49.046589  5189 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:49.106330  5115 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:49.136348  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:49.217438  5115 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:49.219686  5115 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:49.291693  1517 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 5970)
W20260709 12:25:49.301738  5115 consensus_peers.cc:597] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:49.321978  5189 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:49.356236  5189 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:49.959354  5115 consensus_peers.cc:597] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:49.992640  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:50.248860  5189 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:50.275559  5041 consensus_peers.cc:597] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:50.292351  1517 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 5970)
W20260709 12:25:50.397255  5041 consensus_peers.cc:597] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:50.544160  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:50.547909  5189 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:50.649778  5115 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:50.706079  5041 consensus_peers.cc:597] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:50.739333  5041 consensus_peers.cc:597] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:50.746554  5115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111) [suppressed 453 similar messages]
W20260709 12:25:50.755895  5041 consensus_peers.cc:597] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:50.781235  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:50.799096  5041 consensus_peers.cc:597] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:50.889006  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:50.958931  5115 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.034516  5115 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.037853  5041 consensus_peers.cc:597] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.066941  5041 consensus_peers.cc:597] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.152209  5115 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.160277  5189 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.179941  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.197429  5115 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.201623  5115 consensus_peers.cc:597] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:51.223160  5189 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.293027  1517 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 5970)
W20260709 12:25:51.297376  5115 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.315373  5189 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:51.317564  5115 consensus_peers.cc:597] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.343176  5041 consensus_peers.cc:597] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.344570  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.407636  5189 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.423890  5189 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:51.431550  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.525521  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.526229  5041 consensus_peers.cc:597] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.599128  5041 consensus_peers.cc:597] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.619498  5189 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.629079  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:51.853335  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:52.004931  5115 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:52.032903  5189 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:52.085021  5189 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:52.105962  5115 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:52.106645  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:52.263546  5115 consensus_peers.cc:597] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:52.267819  5115 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:25:52.362870  1517 tablet_server.cc:196] TabletServer@127.1.123.65:37253 shutdown complete.
W20260709 12:25:52.642184  5189 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:52.652052  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:52.869894  5041 consensus_peers.cc:597] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:52.872723  5041 consensus_peers.cc:597] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:52.878352  5115 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:52.905347  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:52.952950  5115 consensus_peers.cc:597] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:25:52.958123  1517 tablet_server.cc:179] TabletServer@127.1.123.66:0 shutting down...
W20260709 12:25:52.990718  5188 consensus_peers.cc:597] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:52.992776  5114 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.029683  5114 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.049206  5114 consensus_peers.cc:597] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.068123  5114 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.080766  5189 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:53.092423  5114 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.098763  5114 consensus_peers.cc:597] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.102257  5114 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.106431  5041 consensus_peers.cc:597] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:53.123817  5188 consensus_peers.cc:597] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.165993  5188 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.176744  5188 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.184345  5041 consensus_peers.cc:597] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:53.201042  5114 consensus_peers.cc:597] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.203846  5188 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.224579  5114 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.228204  5188 consensus_peers.cc:597] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.235762  5188 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.256397  5188 consensus_peers.cc:597] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.257835  5188 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.308881  5114 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.332387  5188 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.346864  5041 consensus_peers.cc:597] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:53.352065  5188 consensus_peers.cc:597] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.366194  5041 consensus_peers.cc:597] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:53.387315  5188 consensus_peers.cc:597] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.388079  5188 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:53.401114  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:25:53.397171  5188 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:53.411705  5041 consensus_peers.cc:597] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:53.420801  1517 tablet_replica.cc:333] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:53.426765  1517 raft_consensus.cc:2243] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:53.428047  1517 raft_consensus.cc:2272] T 4da8de53d0db43f9a92906a98dfda98f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:53.446986  5114 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:53.462836  1517 tablet_replica.cc:333] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
W20260709 12:25:53.461624  5188 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:53.468104  1517 raft_consensus.cc:2243] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:53.469064  1517 raft_consensus.cc:2272] T 4516325f133e4ea8beacc1968f96ee9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:53.490836  1517 tablet_replica.cc:333] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:53.495196  1517 raft_consensus.cc:2243] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 LEADER]: Raft consensus shutting down.
W20260709 12:25:53.500294  5188 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:53.500401  1517 raft_consensus.cc:2272] T 01e6663f57ec46609e0512d07df6f705 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:53.505784  1517 tablet_replica.cc:333] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:53.509233  1517 raft_consensus.cc:2243] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:53.512570  1517 raft_consensus.cc:2272] T 0141a059bf6949c49e5e0091604786e6 P 6dc0d06e662d49d88e1b153e09a44e4f [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:53.518718  5115 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:53.526262  5188 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:53.528606  1517 tablet_replica.cc:333] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:53.530154  1517 raft_consensus.cc:2243] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 12:25:53.547982  5189 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:53.570899  5115 consensus_peers.cc:597] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:25:53.573153  1517 raft_consensus.cc:2272] T 6a61a09c949a4cc9959bd9cfbe00992a P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:53.576963  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:53.618346  5115 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:53.622964  1517 tablet_replica.cc:333] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:53.636318  1517 raft_consensus.cc:2243] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 LEADER]: Raft consensus shutting down.
W20260709 12:25:53.641921  5041 consensus_peers.cc:597] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:53.654286  1517 raft_consensus.cc:2272] T 1339c2a0d7f344d09f483e6d965adcc8 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:53.659241  1517 tablet_replica.cc:333] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:53.661176  1517 raft_consensus.cc:2243] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:25:53.675251  1517 raft_consensus.cc:2272] T c5751113d37b44af87d3d912da388078 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:53.690258  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:53.714915  1517 tablet_replica.cc:333] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
W20260709 12:25:53.727481  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:53.732482  1517 raft_consensus.cc:2243] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 12:25:53.733305  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:53.758291  1517 raft_consensus.cc:2272] T 552d065b39ba44699c339e80bcc02744 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:53.768473  1517 tablet_replica.cc:333] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:53.771173  1517 raft_consensus.cc:2243] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:25:53.775518  1517 raft_consensus.cc:2272] T 5b780c93046447618cab07ac8b85cdfd P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:53.804905  1517 tablet_replica.cc:333] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:53.806351  1517 raft_consensus.cc:2243] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:53.808918  1517 raft_consensus.cc:2272] T a8bbb28bc9f5410d95b20827beeb87cc P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:53.821040  1517 tablet_replica.cc:333] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:53.824517  1517 raft_consensus.cc:2243] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:53.831950  1517 raft_consensus.cc:2272] T c86395378750473e80c76902ad29fbe7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:53.834584  5041 consensus_peers.cc:597] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:53.860291  1517 tablet_replica.cc:333] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:53.865060  1517 raft_consensus.cc:2243] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:53.867708  1517 raft_consensus.cc:2272] T 15944aba8caf4c1497501efd45dc0430 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:53.899706  5188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 48519.4us
I20260709 12:25:53.906903 11245 raft_consensus.cc:493] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:53.907853 11245 raft_consensus.cc:515] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:53.922043 11236 raft_consensus.cc:493] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:53.922927 11236 raft_consensus.cc:515] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:53.927724 11245 leader_election.cc:290] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:53.880257  1517 tablet_replica.cc:333] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:53.925460 11236 leader_election.cc:290] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:53.933672  1517 raft_consensus.cc:2243] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 LEADER]: Raft consensus shutting down.
W20260709 12:25:53.940028  5041 leader_election.cc:336] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:53.942875  1517 raft_consensus.cc:2272] T f6a42608cf874aa6943aaf8e349714b9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:53.975940 11245 raft_consensus.cc:493] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:53.977026 11245 raft_consensus.cc:515] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:53.996234 11245 leader_election.cc:290] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:54.009454 11245 raft_consensus.cc:493] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:54.010229 11245 raft_consensus.cc:515] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:54.013839 11245 leader_election.cc:290] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:54.015246  5041 leader_election.cc:336] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:54.016389  1517 tablet_replica.cc:333] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.018252  1517 raft_consensus.cc:2243] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:54.019135  1517 raft_consensus.cc:2272] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.027946  1517 tablet_replica.cc:333] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.029963  1517 raft_consensus.cc:2243] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:54.032121  1517 raft_consensus.cc:2272] T 7383bc919f9c47378977b98535752581 P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:54.029973  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 72330.9us
W20260709 12:25:54.034735  5189 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:54.048234  5115 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:54.067371  5041 leader_election.cc:336] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:54.094244  1517 tablet_replica.cc:333] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.112435  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:54.118053 11245 raft_consensus.cc:493] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.118928 11245 raft_consensus.cc:515] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:54.119673  1517 raft_consensus.cc:2243] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:54.120792 11247 raft_consensus.cc:493] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.121564 11247 raft_consensus.cc:515] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:54.130549 11247 leader_election.cc:290] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:54.124230 11245 leader_election.cc:290] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:25:54.121168  5189 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:54.146150  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:54.148376  1517 raft_consensus.cc:2272] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.159860 11245 raft_consensus.cc:493] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.160625 11245 raft_consensus.cc:515] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:54.164286 11245 leader_election.cc:290] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:54.209810 11247 raft_consensus.cc:493] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.210659 11247 raft_consensus.cc:515] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:54.214445 11247 leader_election.cc:290] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:54.217448  1517 tablet_replica.cc:333] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.220896 11247 raft_consensus.cc:493] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.226591 11247 raft_consensus.cc:515] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:54.229439 11247 leader_election.cc:290] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:54.231487  1517 raft_consensus.cc:2243] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Raft consensus shutting down.
W20260709 12:25:54.219316  5189 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:54.235905  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 50458.6us
I20260709 12:25:54.244884  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "805537066aa5440cbb0608aa77daafcf" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:54.245704  1517 raft_consensus.cc:2272] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.253923  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63938e2a81834411b8018a2eb6c6818f" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:54.242996  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63912da780fe41a2887dde6d2051ac8b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:54.246798  5189 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:54.240814  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfc1cfb224a4fd989b97af5dcf7da23" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:54.276314  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:54.278014  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94a5838186ec4283b67fe9aca66d2a22" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:54.279984  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa8f9fb1ec8411aa7e23f87b827f019" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:54.283962  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:54.290427  1517 tablet_replica.cc:333] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.291672  1517 raft_consensus.cc:2243] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:54.292901  1517 raft_consensus.cc:2272] T db646a2919184c1682011b96d25d0cc9 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.300571 11247 raft_consensus.cc:493] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.301569 11245 raft_consensus.cc:493] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.303493 11245 raft_consensus.cc:515] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:54.301680 11247 raft_consensus.cc:515] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:54.308254 11247 leader_election.cc:290] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:54.311444  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acca9bc33b6d43519a79138d66901f9b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:54.312974  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be74bdf01fe4190992575e84da14996" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:54.314028  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:54.233402  5041 leader_election.cc:336] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:54.317870  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 84493.3us
W20260709 12:25:54.319334  5041 leader_election.cc:336] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:54.326892 11245 leader_election.cc:290] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:54.344619 11247 raft_consensus.cc:493] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:54.346071 11247 raft_consensus.cc:515] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:54.327904  5115 consensus_peers.cc:597] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:54.350163  5115 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:54.370324 11236 raft_consensus.cc:493] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.371695 11236 raft_consensus.cc:515] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:54.375074 11236 leader_election.cc:290] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:54.376643 11247 leader_election.cc:290] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:54.381407 11236 raft_consensus.cc:493] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.383431 11245 raft_consensus.cc:493] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.387848  5220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:54.387692  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c91c376e83f484baca078b37cbf00db" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:54.395143  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b6b7243efc34b21882fab443d3a3720" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:54.396559  1517 tablet_replica.cc:333] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.397957  1517 raft_consensus.cc:2243] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:25:54.396013 11236 raft_consensus.cc:515] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:54.416275 11236 leader_election.cc:290] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:54.422976  1517 raft_consensus.cc:2272] T 719a453defab43d4a1232c664dbaae9c P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.396281 11245 raft_consensus.cc:515] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:25:54.424079  5189 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:25:54.441387 11245 leader_election.cc:290] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:54.424504  5041 leader_election.cc:304] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:54.442440  1517 tablet_replica.cc:333] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.448138 11247 raft_consensus.cc:493] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.453147 11247 raft_consensus.cc:515] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:54.458403 11247 leader_election.cc:290] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:54.449499  5042 leader_election.cc:304] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:54.464401  5042 leader_election.cc:304] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:54.452244  1517 raft_consensus.cc:2243] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 12:25:54.468437  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:54.470240  5042 leader_election.cc:304] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:54.470896  5041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 46502.6us
I20260709 12:25:54.472157  5041 leader_election.cc:304] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:54.475194 11236 raft_consensus.cc:2749] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:54.475673 11250 raft_consensus.cc:2749] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:54.476332 11236 raft_consensus.cc:2749] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:54.479878  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:54.480881  5189 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:54.470069 11247 raft_consensus.cc:2749] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:54.487874  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:54.494163  1517 raft_consensus.cc:2272] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.497802 11251 raft_consensus.cc:2749] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:54.508896  1517 tablet_replica.cc:333] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.527349  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a1d0ab099f4c37a8f1fbe48688ac24" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:54.529856  5042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 59720.8us
I20260709 12:25:54.531025  5042 leader_election.cc:304] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:54.531155  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159b5709d37c488d935b1fe85d29c5bc" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:54.533447  1517 raft_consensus.cc:2243] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:54.534314 11251 raft_consensus.cc:2749] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:54.536834  1517 raft_consensus.cc:2272] T 805537066aa5440cbb0608aa77daafcf P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.537932  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfdb4c20c33743d79d57552db35c484b" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:54.542165  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c232e09b414797838d5e57d028ca97" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:54.538159 11251 raft_consensus.cc:493] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:25:54.545775 11251 raft_consensus.cc:515] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:54.548947 11251 leader_election.cc:290] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:54.557192 11253 raft_consensus.cc:493] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90e5220a62ed42fe83c82d0302d18fac)
I20260709 12:25:54.564433 11253 raft_consensus.cc:515] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:54.546010  1517 tablet_replica.cc:333] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.567801  1517 raft_consensus.cc:2243] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:54.571061  1517 raft_consensus.cc:2272] T 7dfc1cfb224a4fd989b97af5dcf7da23 P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.573884  5042 leader_election.cc:304] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:54.565637  5041 leader_election.cc:336] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:54.559909  5115 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:54.575991  5041 leader_election.cc:304] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:54.577565 11251 raft_consensus.cc:2749] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:54.578926  5041 leader_election.cc:336] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:54.568310 11253 leader_election.cc:290] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:25:54.581691 11251 raft_consensus.cc:2749] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:54.583442  5042 leader_election.cc:304] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:54.594974  5042 leader_election.cc:304] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:54.595734 11245 raft_consensus.cc:2749] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:54.597973 11245 raft_consensus.cc:2749] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.600487  1517 tablet_replica.cc:333] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.602296  1517 raft_consensus.cc:2243] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:54.604269  1517 raft_consensus.cc:2272] T 6c91c376e83f484baca078b37cbf00db P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.606123  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f50ca0c2db44278637fcdffffc03a2" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
W20260709 12:25:54.614298  5041 leader_election.cc:336] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:54.630960  5041 leader_election.cc:304] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:54.632880 11251 raft_consensus.cc:2749] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.633421  1517 tablet_replica.cc:333] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.638434  1517 raft_consensus.cc:2243] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:25:54.640098  5041 consensus_peers.cc:597] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:54.642565  5041 leader_election.cc:336] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:54.643225  5041 leader_election.cc:304] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:54.644295 11251 raft_consensus.cc:2706] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Received pre-election callback for term 22 while not running: Illegal state: RaftConsensus is not running: State = Stopping
W20260709 12:25:54.645700  5041 leader_election.cc:336] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:54.646458  5041 leader_election.cc:304] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:54.647600  1517 raft_consensus.cc:2272] T 159b5709d37c488d935b1fe85d29c5bc P 6dc0d06e662d49d88e1b153e09a44e4f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.658012 11251 raft_consensus.cc:2749] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.663419  1517 tablet_replica.cc:333] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
W20260709 12:25:54.664558  5041 leader_election.cc:336] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:54.666402  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5244acc0775748b7933286d588ec2b56" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:54.668426  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de3ceedceac4e0ca06de7a806ec05e3" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:54.665725  5041 leader_election.cc:304] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:54.670706  5145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09749d11015f4695b5c630c6e5dcbcb0" candidate_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:54.671731 11251 raft_consensus.cc:2749] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:54.687901  5115 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:25:54.696623  1517 raft_consensus.cc:2243] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:54.695640 11242 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:54.699790  1517 raft_consensus.cc:2272] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.699774 11242 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:54.705761  5041 leader_election.cc:336] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:54.706571  5041 leader_election.cc:336] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:54.707476  5041 leader_election.cc:304] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:54.711634  5042 leader_election.cc:304] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:54.721571 11242 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:54.713742 11245 raft_consensus.cc:2749] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:54.724666  1517 tablet_replica.cc:333] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
W20260709 12:25:54.730922  5189 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:54.729676  5042 leader_election.cc:304] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:54.732323 11251 raft_consensus.cc:2749] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:54.740149  1517 raft_consensus.cc:2243] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:54.744102  1517 raft_consensus.cc:2272] T 5244acc0775748b7933286d588ec2b56 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.749024  1517 tablet_replica.cc:333] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
W20260709 12:25:54.749737  5042 raft_consensus.cc:2686] T 5de3ceedceac4e0ca06de7a806ec05e3 P 6dc0d06e662d49d88e1b153e09a44e4f: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:25:54.750303  1517 raft_consensus.cc:2243] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:54.751525  1517 raft_consensus.cc:2272] T a6f50ca0c2db44278637fcdffffc03a2 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.751538  5042 leader_election.cc:304] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dc0d06e662d49d88e1b153e09a44e4f; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:54.756582 11251 raft_consensus.cc:2749] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:54.772091  1517 tablet_replica.cc:333] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.774051  1517 raft_consensus.cc:2243] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Raft consensus shutting down.
W20260709 12:25:54.780524  5188 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:54.781476  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:54.783483 11242 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.786512  1517 raft_consensus.cc:2272] T e77063f082034d8bbc280357b7b66bb7 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.806591 11254 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:54.807457 11254 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:54.813066  1517 tablet_replica.cc:333] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.816639  1517 raft_consensus.cc:2243] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:25:54.818989  1517 raft_consensus.cc:2272] T 9ee83a16af0a40998fc58f3e882c6080 P 6dc0d06e662d49d88e1b153e09a44e4f [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:54.819130  5188 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:54.822100 11254 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:54.838802  1517 tablet_replica.cc:333] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.840377  1517 raft_consensus.cc:2243] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:54.841565  1517 raft_consensus.cc:2272] T acca9bc33b6d43519a79138d66901f9b P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.869329  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:54.872318  1517 tablet_replica.cc:333] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.873697  1517 raft_consensus.cc:2243] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:54.875206  1517 raft_consensus.cc:2272] T 09749d11015f4695b5c630c6e5dcbcb0 P 6dc0d06e662d49d88e1b153e09a44e4f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.883291  1517 tablet_replica.cc:333] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.884661  1517 raft_consensus.cc:2243] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:25:54.888500  5190 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:54.890389 11254 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:54.890789  5115 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:54.892055  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:54.895166  5189 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:25:54.907668  1517 raft_consensus.cc:2272] T 58bebed3b53f4f178dae221a0daae185 P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.920224 11243 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:54.920941 11243 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:54.924117 11243 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:54.929316  5114 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:54.946755  1517 tablet_replica.cc:333] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.959751  1517 raft_consensus.cc:2243] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 LEADER]: Raft consensus shutting down.
W20260709 12:25:54.964221  5115 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:54.964923  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:54.966687  1517 raft_consensus.cc:2272] T 874aa8edc94946d69e4a25563a5a0f0f P 6dc0d06e662d49d88e1b153e09a44e4f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:54.968376 11233 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.985001  1517 tablet_replica.cc:333] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:54.986244  1517 raft_consensus.cc:2243] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Raft consensus shutting down.
W20260709 12:25:54.986269  5189 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:25:54.998356  1517 raft_consensus.cc:2272] T 94a5838186ec4283b67fe9aca66d2a22 P 6dc0d06e662d49d88e1b153e09a44e4f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:55.002835  1517 tablet_replica.cc:333] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:55.010663  1517 raft_consensus.cc:2243] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:55.016970  1517 raft_consensus.cc:2272] T 7be74bdf01fe4190992575e84da14996 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:55.034691  1517 tablet_replica.cc:333] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:55.035766  1517 raft_consensus.cc:2243] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:55.036962  1517 raft_consensus.cc:2272] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 6dc0d06e662d49d88e1b153e09a44e4f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:55.041213  1517 tablet_replica.cc:333] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:55.042954  1517 raft_consensus.cc:2243] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:55.058288  1517 raft_consensus.cc:2272] T 63912da780fe41a2887dde6d2051ac8b P 6dc0d06e662d49d88e1b153e09a44e4f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:55.063666  1517 tablet_replica.cc:333] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:55.064306 11233 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.064771  1517 raft_consensus.cc:2243] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:55.065084 11233 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:55.066313  1517 raft_consensus.cc:2272] T c1c232e09b414797838d5e57d028ca97 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:55.070842 11242 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.071861 11233 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:55.071647 11242 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:55.074131  5114 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:55.074357 11242 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:55.079142 11238 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
W20260709 12:25:55.079981  5188 leader_election.cc:336] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:55.079912 11238 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:55.080632  5189 leader_election.cc:336] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:55.081274  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:55.082240 11242 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:55.083395 11238 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:55.090902  5188 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:25:55.093447  5115 consensus_peers.cc:597] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:25:55.095863  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:55.070931  1517 tablet_replica.cc:333] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:55.098907  5216 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 15.
I20260709 12:25:55.099598  1517 raft_consensus.cc:2243] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:55.100749  1517 raft_consensus.cc:2272] T 2b6b7243efc34b21882fab443d3a3720 P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:55.109711  1517 tablet_replica.cc:333] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:55.112819  5142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:55.113281  1517 raft_consensus.cc:2243] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:25:55.126297  1517 raft_consensus.cc:2272] T 4935fc8f35664324a253df9804c8074d P 6dc0d06e662d49d88e1b153e09a44e4f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:55.126281  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:55.127560 11233 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:55.128546 11233 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.129323 11233 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.133380  1517 tablet_replica.cc:333] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:55.134536  1517 raft_consensus.cc:2243] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:25:55.135918  1517 raft_consensus.cc:2272] T 59bc6f04e62a431ba2f113d08a129898 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:55.143153  1517 tablet_replica.cc:333] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:55.158051  1517 raft_consensus.cc:2243] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:25:55.159791  1517 raft_consensus.cc:2272] T e4be6683658549c99c727b84f9cd9774 P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:55.163899  1517 tablet_replica.cc:333] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
W20260709 12:25:55.169713  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 43560.3us
I20260709 12:25:55.174474  1517 raft_consensus.cc:2243] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:55.174780  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:55.175873  1517 raft_consensus.cc:2272] T 63938e2a81834411b8018a2eb6c6818f P 6dc0d06e662d49d88e1b153e09a44e4f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:55.177017 11242 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:55.181308  1517 tablet_replica.cc:333] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f: stopping tablet replica
I20260709 12:25:55.182935  1517 raft_consensus.cc:2243] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:55.190565  1517 raft_consensus.cc:2272] T bfdb4c20c33743d79d57552db35c484b P 6dc0d06e662d49d88e1b153e09a44e4f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:55.195263 11233 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:55.216193 11233 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:55.227192  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b780c93046447618cab07ac8b85cdfd" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:55.227939  5216 raft_consensus.cc:3060] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:55.229944  5114 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:25:55.235929  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:55.241112  5115 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:55.323514  5114 consensus_peers.cc:597] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:55.316790  5216 raft_consensus.cc:2468] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 16.
I20260709 12:25:55.327406  5115 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:55.328848 11233 raft_consensus.cc:2804] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:55.329798 11233 raft_consensus.cc:697] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:55.331123 11233 consensus_queue.cc:237] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:55.364962  5188 consensus_peers.cc:597] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:55.382598  4899 catalog_manager.cc:5697] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 15 to 16, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 16 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:55.412169  5114 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:55.430642 11255 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.432906 11255 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:55.435889  5042 connection.cc:466] server connection from 127.0.0.1:58036 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58036 (request call id 1943) could send its response
I20260709 12:25:55.460934 11255 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:55.463035  5114 connection.cc:573] client connection to 127.1.123.66:33955 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 12:25:55.500401  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:55.501726  5222 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:55.536057  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 
I20260709 12:25:55.537490 11259 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:55.538018 11259 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.538446 11259 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:55.550750  5114 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:25:55.554270  5114 consensus_peers.cc:597] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:55.554795 11259 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:55.558249 11259 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:55.559640  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:55.563045  5114 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:55.570813  5216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86395378750473e80c76902ad29fbe7" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:55.571796  5216 raft_consensus.cc:3060] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:55.608903  5216 raft_consensus.cc:2468] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
I20260709 12:25:55.611254  5115 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:55.612617 11263 raft_consensus.cc:2804] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:55.613252 11263 raft_consensus.cc:697] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 19 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:55.614619 11263 consensus_queue.cc:237] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:55.621523 11256 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.622742 11256 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:55.629132  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:55.639554 11256 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:25:55.642117 11254 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.643055 11254 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:55.645476 11254 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:55.649392  5188 leader_election.cc:336] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:55.650069  5188 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:25:55.652140 11238 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:55.657109  5189 leader_election.cc:336] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:55.669795  5188 leader_election.cc:336] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:55.670553  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:55.671676 11238 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:55.683929  4899 catalog_manager.cc:5697] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 18 to 19, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 19 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:55.701398 11263 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.702201 11263 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:55.723631  5188 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:55.729894  5114 leader_election.cc:336] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:25:55.731006  5114 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:55.733214 11263 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:55.735291  5188 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:55.738440  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:55.739308  5222 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
W20260709 12:25:55.740868  5115 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:55.744050  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:55.744999 11259 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:25:55.745158  5188 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:55.745640 11259 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.746143 11259 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:55.747917  5114 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:55.759049  5188 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111) [suppressed 569 similar messages]
I20260709 12:25:55.761349 11263 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.762110 11263 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:55.763077  5188 consensus_peers.cc:597] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:55.765480 11263 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:55.766325 11263 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.766937 11263 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:55.767236  5188 consensus_peers.cc:597] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:55.767720 11259 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:55.769114 11263 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:55.770627 11243 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:55.774040 11259 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
I20260709 12:25:55.780778  5222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db646a2919184c1682011b96d25d0cc9" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:55.791061  5222 raft_consensus.cc:3060] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:55.790841 11243 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:55.793354 11243 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:55.806407  5115 leader_election.cc:336] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:55.808898  5115 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:55.809968  5188 consensus_peers.cc:597] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:55.811024  5114 leader_election.cc:336] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:55.812083  5222 raft_consensus.cc:2468] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 19.
W20260709 12:25:55.816996  5115 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:55.817930  5114 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:55.818863 11233 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:55.820124  5114 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:55.820613  5114 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:55.829802 11233 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:25:55.831889  5114 leader_election.cc:336] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:25:55.843083  5188 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:55.845659  5114 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:55.846235  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:55.847434 11263 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:55.846035  5115 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:55.851166 11263 raft_consensus.cc:2804] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:25:55.854079  5188 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:55.864526 11263 raft_consensus.cc:697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 19 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:55.865996 11263 consensus_queue.cc:237] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:55.870142  5114 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:55.870981  5114 consensus_peers.cc:597] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:55.871496  5114 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:55.881007  5188 consensus_peers.cc:597] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:55.890333  5114 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:55.895911  5114 consensus_peers.cc:597] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:55.906639  4899 catalog_manager.cc:5697] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 18 to 19, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 19 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:55.937788  5189 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:25:55.974493  5223 raft_consensus.cc:1275] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:25:55.981745  5188 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:55.983989  5114 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:55.996059  5189 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:25:55.996371 11243 consensus_queue.cc:1048] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
W20260709 12:25:56.030552  5115 consensus_peers.cc:597] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:25:56.035351 11242 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:56.036041 11242 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:56.039640 11242 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:56.034318  5114 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:56.047071  5188 leader_election.cc:336] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:25:56.060729  5188 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:56.100598  5188 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:56.115767  5188 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:56.117353  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:56.138851  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 74092us
W20260709 12:25:56.146198  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 54545.4us
W20260709 12:25:56.172722  5114 consensus_peers.cc:597] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:56.193285  5141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
I20260709 12:25:56.194268  5141 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 16.
I20260709 12:25:56.198041 11242 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:56.199046 11242 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:56.202059 11242 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:56.218724  5188 leader_election.cc:336] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:56.219635  5223 raft_consensus.cc:1275] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:25:56.232270  5190 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:56.238334 11238 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:56.239523 11238 raft_consensus.cc:493] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:56.240160 11238 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:56.284981 11243 consensus_queue.cc:1048] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:56.307329  5188 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:56.314368 11238 raft_consensus.cc:515] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:56.316906 11238 leader_election.cc:290] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:56.332238  5188 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:56.334106  5188 consensus_peers.cc:597] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:56.334157  5189 leader_election.cc:336] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:56.335783  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:56.336139 11252 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:56.337213 11252 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:56.339221  5188 leader_election.cc:336] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:56.340924 11252 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:56.343036 11268 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:56.349810 11252 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:56.350852 11252 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:56.353777 11252 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:56.358875  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bebed3b53f4f178dae221a0daae185" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016"
I20260709 12:25:56.359974  5147 raft_consensus.cc:3060] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:56.363248  5115 consensus_peers.cc:597] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:56.375561  5115 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:56.376389  5115 leader_election.cc:336] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:56.398038  5115 leader_election.cc:336] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:56.394368  5147 raft_consensus.cc:2468] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e5220a62ed42fe83c82d0302d18fac in term 17.
W20260709 12:25:56.404810  5114 consensus_peers.cc:597] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:56.405606  5114 leader_election.cc:336] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:56.407254  5190 leader_election.cc:304] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:56.419013 11270 raft_consensus.cc:2804] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:56.420077 11270 raft_consensus.cc:697] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Becoming Leader. State: Replica: 90e5220a62ed42fe83c82d0302d18fac, State: Running, Role: LEADER
I20260709 12:25:56.421681 11270 consensus_queue.cc:237] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:56.420346  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:56.430652 11255 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:56.432197  5114 leader_election.cc:336] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:56.433046  5114 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:56.435513 11238 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:56.436638 11255 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:56.436550 11238 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:56.441322 11238 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:25:56.451036  5188 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:56.453629  5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "90e5220a62ed42fe83c82d0302d18fac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ceec344dd00f40c18ef58414ce147016" is_pre_election: true
W20260709 12:25:56.455284  5189 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:56.460472  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:25:56.476035  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:25:56.477691 11271 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:56.486338 11255 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:56.487131 11255 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:56.489835 11255 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:56.499619  5216 raft_consensus.cc:1275] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:25:56.500415  5114 leader_election.cc:336] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:25:56.502403  1517 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 5097)
I20260709 12:25:56.506282  4899 catalog_manager.cc:5697] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac reported cstate change: term changed from 16 to 17, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68). New cstate: current_term: 17 leader_uuid: "90e5220a62ed42fe83c82d0302d18fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:56.520514  5189 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:56.534775  5115 leader_election.cc:336] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:56.537765  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:56.538960 11255 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:56.545683 11255 consensus_queue.cc:1048] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:56.563872  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:56.584437  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:56.618320  5189 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:56.631932  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:25:56.690400 11252 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:56.691200 11252 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:56.695704 11252 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:56.703567  5114 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:25:56.700749  5115 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:25:56.708482  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac" is_pre_election: true
I20260709 12:25:56.709308  5223 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 17.
I20260709 12:25:56.716995  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
W20260709 12:25:56.726647  5189 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:56.753430 11252 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:56.761951 11255 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:56.766048 11255 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:56.769219 11255 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:56.756316 11252 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:56.794376 11252 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:56.802609  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 85759.4us
W20260709 12:25:56.818212  5114 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:25:56.822431  5189 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:25:56.838328 11238 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:56.839102 11238 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:56.844027 11238 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:56.859989  5189 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:56.872583  5188 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:56.873277  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:56.874523 11238 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:56.899987 11252 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:25:56.929441 11252 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Requested vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), 90e5220a62ed42fe83c82d0302d18fac (127.1.123.68:43449)
W20260709 12:25:56.903988  5115 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:56.931242  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:56.933341  5115 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:25:56.934082 11252 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:56.936035  5114 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:56.945730  5223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4be6683658549c99c727b84f9cd9774" candidate_uuid: "ceec344dd00f40c18ef58414ce147016" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90e5220a62ed42fe83c82d0302d18fac"
I20260709 12:25:56.946799  5223 raft_consensus.cc:3060] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:56.958405  5115 consensus_peers.cc:597] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:25:56.961400  5223 raft_consensus.cc:2468] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceec344dd00f40c18ef58414ce147016 in term 18.
I20260709 12:25:56.968899  5115 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac, ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f
I20260709 12:25:56.970167 11252 raft_consensus.cc:2804] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:56.970716 11252 raft_consensus.cc:697] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 18 LEADER]: Becoming Leader. State: Replica: ceec344dd00f40c18ef58414ce147016, State: Running, Role: LEADER
I20260709 12:25:56.971689 11252 consensus_queue.cc:237] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:25:56.986815  5115 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:56.987977  5115 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:25:57.026243  4899 catalog_manager.cc:5697] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 reported cstate change: term changed from 17 to 18, leader changed from 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66) to ceec344dd00f40c18ef58414ce147016 (127.1.123.67). New cstate: current_term: 18 leader_uuid: "ceec344dd00f40c18ef58414ce147016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:57.047714  5188 consensus_peers.cc:597] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:57.050030  5141 raft_consensus.cc:1275] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Refusing update from remote peer 90e5220a62ed42fe83c82d0302d18fac: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:57.052881 11238 consensus_queue.cc:1048] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:25:57.096954  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:57.128218  5115 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:57.130863 11270 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:57.131762 11270 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:57.153318 11270 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:57.161043  5188 leader_election.cc:336] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:25:57.176093  5189 leader_election.cc:336] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:57.176903  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:57.182032 11238 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:57.182168 11284 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:57.183079 11284 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:57.192780  5188 leader_election.cc:336] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:57.196692 11284 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:57.226630  5189 leader_election.cc:336] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:57.227363  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:57.245045 11284 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:57.249089  5189 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:57.363677  5114 consensus_peers.cc:597] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:57.380964  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:57.387926  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:57.418888  5189 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:57.426702  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:57.456079  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:57.503386  1517 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 5097)
I20260709 12:25:57.509672  5223 raft_consensus.cc:1275] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Refusing update from remote peer ceec344dd00f40c18ef58414ce147016: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:25:57.518762  5115 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:57.525138 11263 consensus_queue.cc:1048] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:57.720504  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:25:57.736572 11289 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:57.737970 11289 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:57.764792 11289 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:57.775322  5115 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:57.784389  5114 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:57.794137  5114 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:57.796170 11289 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:57.811146  5115 consensus_peers.cc:597] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:57.850385  5114 consensus_peers.cc:597] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:57.921056  5115 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:57.922511  5114 consensus_peers.cc:597] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:57.940028  5189 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:57.979264  5188 consensus_peers.cc:597] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:25:58.021791 11289 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:58.023000 11289 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:58.029443 11263 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:58.030285 11263 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:25:58.044363  5114 leader_election.cc:336] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:25:58.047008  5115 leader_election.cc:336] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:58.047479  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:58.047896 11289 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:58.033329 11263 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:58.048265 11270 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:58.049343 11270 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:58.049834 11233 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:58.049775  5114 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:25:58.054014  5115 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:58.056227  5115 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:58.061878  5189 leader_election.cc:336] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:58.075541 11270 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:25:58.080325 11263 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:25:58.086710  5188 leader_election.cc:336] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:58.087404  5188 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:58.090317 11284 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:58.126654 11263 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:58.127816 11263 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:58.134799 11263 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:58.141342  5115 leader_election.cc:336] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:58.143751  5114 leader_election.cc:336] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:58.144515  5114 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:58.145406 11263 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:58.148005  5188 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.176927  5188 consensus_peers.cc:597] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.254726  5188 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.309371  5114 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.310097  5114 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.318678  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:25:58.334496 11263 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:58.335413 11263 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:58.348873 11263 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:58.364692  5114 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.372291  5115 leader_election.cc:336] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:58.378635  5188 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.378703  5114 leader_election.cc:336] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:58.381789  5114 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:58.389261 11252 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:58.399708 11252 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:58.401651 11252 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:58.405047 11252 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:58.415701  5114 consensus_peers.cc:597] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:58.417129  5188 consensus_peers.cc:597] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.422596  5188 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.437217  5188 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.451231  5115 leader_election.cc:336] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:58.452214  5114 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.454838  5114 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.456439  5189 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:58.468776  5114 leader_election.cc:336] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:58.484208  5114 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:25:58.483417  5188 consensus_peers.cc:597] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:25:58.492987 11289 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:25:58.495087  5114 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.504292  1517 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 5097)
W20260709 12:25:58.538936  5188 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.544608  5188 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.558184  5114 consensus_peers.cc:597] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.562809  5189 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:58.597884  5188 consensus_peers.cc:597] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.606927  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:25:58.611423 11289 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:58.613090 11289 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:58.624235 11289 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:58.625728  5188 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.626652  5114 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:25:58.628538  5189 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:58.631681  5115 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:58.640446  5115 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:58.641013  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:58.642628 11289 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:58.663518  5188 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.673687  5114 consensus_peers.cc:597] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:58.677973  5115 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:58.691846  5115 consensus_peers.cc:597] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:58.694142  5188 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:25:58.750787 11284 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:58.754186 11284 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:25:58.772624  5188 leader_election.cc:336] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:58.776144 11284 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:58.780073  5114 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.798126  5114 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.802332  5189 leader_election.cc:336] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:25:58.803480  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:58.806922 11270 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:58.850306  5188 consensus_peers.cc:597] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.924397  5188 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:58.941298  5189 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:25:58.949414 11284 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:58.951746 11284 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:25:58.955873 11284 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:58.968556  5189 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:25:58.972427  5188 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:58.974493  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:25:58.976555 11294 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:59.053007  5114 consensus_peers.cc:597] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:59.103340  5189 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.138723  5115 consensus_peers.cc:597] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:59.158038  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.159845  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.185356  5189 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.237118  5189 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:59.326576  5115 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.339795  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.377959  5115 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.496187  5115 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.580490  5188 consensus_peers.cc:597] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:59.648382  5115 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.649147  5115 consensus_peers.cc:597] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.689921  5114 consensus_peers.cc:597] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:59.695668  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.741645  5115 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:59.809437  5189 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:25:59.889086 11289 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:59.889811 11289 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:59.892321 11289 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:25:59.934123  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:59.938052  5114 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:25:59.978845 11289 raft_consensus.cc:493] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:25:59.980192 11289 raft_consensus.cc:515] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:25:59.983294 11289 leader_election.cc:290] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:00.037158  5114 leader_election.cc:336] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:26:00.006994  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:00.075347  5189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 68488.7us
W20260709 12:26:00.087914  5114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:33955, user_credentials={real_user=slave}} blocked reactor thread for 50794.4us
W20260709 12:26:00.098354  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:00.115674  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.68:43449, user_credentials={real_user=slave}} blocked reactor thread for 43722.1us
W20260709 12:26:00.130174  5189 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:00.164137  5115 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:00.175088  5115 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:00.189016 11252 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:00.224354 11284 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:00.230252 11284 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:26:00.234181  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:26:00.278590 11284 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:26:00.333418 11270 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:00.335176 11270 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:26:00.338594 11270 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:00.343614  5115 leader_election.cc:336] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:00.374397  5115 leader_election.cc:304] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:26:00.369601  5188 leader_election.cc:336] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:00.388914 11252 raft_consensus.cc:2749] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:00.396160  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 52584.3us
W20260709 12:26:00.407112  5189 leader_election.cc:336] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:00.430496  5189 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:00.440042 11284 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:00.472177  5188 leader_election.cc:336] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:26:00.473412  5114 consensus_peers.cc:597] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:00.499699  5189 leader_election.cc:336] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:00.513015  5189 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:00.529848 11270 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:00.540697  5114 consensus_peers.cc:597] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:00.542922  5188 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:00.620538  5189 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:00.626925  5188 consensus_peers.cc:597] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:00.712365  5188 consensus_peers.cc:597] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:00.713248 11252 raft_consensus.cc:493] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:00.735239 11252 raft_consensus.cc:515] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:26:00.738631 11252 leader_election.cc:290] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:00.751344  5114 leader_election.cc:336] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:26:00.752609  5188 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:00.767658 11252 raft_consensus.cc:493] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:00.768846 11252 raft_consensus.cc:515] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:26:00.769212  5189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111) [suppressed 685 similar messages]
W20260709 12:26:00.779115  5114 leader_election.cc:336] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:00.781004 11252 leader_election.cc:290] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:00.797142  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:00.803100  5114 consensus_peers.cc:597] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:00.812507  5114 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:00.826117  5188 consensus_peers.cc:597] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:00.850009  5188 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:00.870170  5114 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:00.873311  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:00.932183  5188 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:00.944901  5189 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:26:00.985158 11301 raft_consensus.cc:493] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:00.986058 11301 raft_consensus.cc:515] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:26:00.992115 11301 leader_election.cc:290] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:01.007258  5115 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:01.016739  5188 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.019767  5114 consensus_peers.cc:597] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.050006  5188 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:01.064460 11301 raft_consensus.cc:493] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:01.066146 11301 raft_consensus.cc:515] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
W20260709 12:26:01.069046  5188 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:01.070554 11301 leader_election.cc:290] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:01.062114  5114 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.081014  5114 consensus_peers.cc:597] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.086169  5114 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:26:01.145174  5115 leader_election.cc:336] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:26:01.146759  5188 consensus_peers.cc:597] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.156930  5188 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:01.156600  5115 leader_election.cc:304] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:26:01.159476  5114 leader_election.cc:336] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:01.167619 11302 raft_consensus.cc:2749] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:01.168596  5115 leader_election.cc:336] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:01.175351  5115 leader_election.cc:304] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:26:01.205924  5189 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:01.225893  5188 consensus_peers.cc:597] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:01.231248 11302 raft_consensus.cc:2749] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:01.235051  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 66437.5us
W20260709 12:26:01.265913  5188 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.271083  5114 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.305080  5189 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:01.343001  5188 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.359265  5188 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.386147  5114 consensus_peers.cc:597] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:01.396080  5188 consensus_peers.cc:597] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.459090  5114 consensus_peers.cc:597] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.462909  5189 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:01.465806  5115 consensus_peers.cc:597] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:01.470897  5115 leader_election.cc:336] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:01.474674  5115 leader_election.cc:304] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:01.490924 11301 raft_consensus.cc:2749] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:01.503808  5114 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.617743  5115 leader_election.cc:336] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:26:01.621376  5189 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:26:01.621299  5115 leader_election.cc:304] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:01.626775 11302 raft_consensus.cc:2749] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:01.628954  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:01.633443  5115 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:01.695570  5189 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:26:01.729367  1517 tablet_server.cc:196] TabletServer@127.1.123.66:0 shutdown complete.
W20260709 12:26:01.766307  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:01.784897  5114 consensus_peers.cc:597] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:01.804556  5115 consensus_peers.cc:597] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:01.941449  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:01.943902  5115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:37253, user_credentials={real_user=slave}} blocked reactor thread for 75377.7us
I20260709 12:26:01.952477 11298 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:01.953310 11298 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:26:01.964454 11298 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:02.005363  5189 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:26:02.012480  5188 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:26:02.008926  5115 consensus_peers.cc:597] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:02.020860  5115 consensus_peers.cc:597] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:26:02.032358  5188 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:02.047338 11298 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:02.052284  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 44977.4us
W20260709 12:26:02.088781  5189 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:02.099363  5190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:46855, user_credentials={real_user=slave}} blocked reactor thread for 45135.8us
I20260709 12:26:02.110003 11301 raft_consensus.cc:493] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:02.110903 11301 raft_consensus.cc:515] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
W20260709 12:26:02.122291  5114 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:02.123930 11301 leader_election.cc:290] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:26:02.130829 11307 raft_consensus.cc:493] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:02.131670 11307 raft_consensus.cc:515] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:26:02.136008 11307 leader_election.cc:290] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:02.147825  5188 consensus_peers.cc:597] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:02.159749  5189 leader_election.cc:336] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:26:02.163527  5188 leader_election.cc:336] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:02.164126  5188 leader_election.cc:304] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:02.165119 11307 raft_consensus.cc:2749] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:02.182996  5115 leader_election.cc:336] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:02.195348  1517 tablet_server.cc:179] TabletServer@127.1.123.67:0 shutting down...
W20260709 12:26:02.199093  5114 consensus_peers.cc:597] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:02.200114  5115 leader_election.cc:304] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:02.202865 11310 raft_consensus.cc:2749] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:02.253175  5115 consensus_peers.cc:597] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:02.295892  5189 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:02.298460  5190 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:02.315265  5190 consensus_peers.cc:597] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:02.316432  5190 consensus_peers.cc:597] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:02.365639  5190 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:02.381615  5190 consensus_peers.cc:597] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:02.386106  5190 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:02.395272  5115 consensus_peers.cc:597] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:02.395363  5190 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:02.398222  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:02.444989  5190 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:02.474431  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:26:02.500784  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:02.511090  1517 tablet_replica.cc:333] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.513195  1517 raft_consensus.cc:2243] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:02.514489  1517 raft_consensus.cc:2272] T 3665d1ab30ab4c8d9b7304c26a0cd138 P ceec344dd00f40c18ef58414ce147016 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.521826  1517 tablet_replica.cc:333] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
W20260709 12:26:02.525767  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:26:02.552006  1517 raft_consensus.cc:2243] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:02.553350  1517 raft_consensus.cc:2272] T 01e6663f57ec46609e0512d07df6f705 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.559973  1517 tablet_replica.cc:333] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.562300  1517 raft_consensus.cc:2243] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Raft consensus shutting down.
W20260709 12:26:02.565362  5114 consensus_peers.cc:597] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:02.567842  1517 raft_consensus.cc:2272] T 6c91c376e83f484baca078b37cbf00db P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:02.580811  5115 consensus_peers.cc:597] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:26:02.590282  1517 tablet_replica.cc:333] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.591830  1517 raft_consensus.cc:2243] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 18 LEADER]: Raft consensus shutting down.
W20260709 12:26:02.592412  5188 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:02.593089  5189 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:26:02.614114  1517 raft_consensus.cc:2272] T 552d065b39ba44699c339e80bcc02744 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:02.615728  5190 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:02.619011  1517 tablet_replica.cc:333] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.625634  1517 raft_consensus.cc:2243] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Raft consensus shutting down.
W20260709 12:26:02.628170  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:26:02.629715  1517 raft_consensus.cc:2272] T 5b780c93046447618cab07ac8b85cdfd P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:02.651381  5190 consensus_peers.cc:597] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:02.651317  1517 tablet_replica.cc:333] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.653311  1517 raft_consensus.cc:2243] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:02.657151  1517 raft_consensus.cc:2272] T cf661d015f0e4fc19e57069da58329e3 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:02.658183  5190 consensus_peers.cc:597] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:02.661597  1517 tablet_replica.cc:333] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.670588  1517 raft_consensus.cc:2243] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:26:02.673445  1517 raft_consensus.cc:2272] T a8bbb28bc9f5410d95b20827beeb87cc P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.678331  1517 tablet_replica.cc:333] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
W20260709 12:26:02.683784  5115 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:26:02.690346  1517 raft_consensus.cc:2243] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:26:02.692598  1517 raft_consensus.cc:2272] T c86395378750473e80c76902ad29fbe7 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.698850  1517 tablet_replica.cc:333] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.700063  1517 raft_consensus.cc:2243] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:26:02.701735  1517 raft_consensus.cc:2272] T 15944aba8caf4c1497501efd45dc0430 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.706439  1517 tablet_replica.cc:333] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.707736  1517 raft_consensus.cc:2243] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:02.709156  1517 raft_consensus.cc:2272] T 003133c544c3449a827e3758faa02f02 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.735841  1517 tablet_replica.cc:333] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.739715  1517 raft_consensus.cc:2243] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:02.741185  1517 raft_consensus.cc:2272] T 4935fc8f35664324a253df9804c8074d P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.747008  1517 tablet_replica.cc:333] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.748891  1517 raft_consensus.cc:2243] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:26:02.750759  1517 raft_consensus.cc:2272] T 77f30d7c9f3244999f248bc1aed226c7 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:02.751274  5190 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:02.760668  5115 consensus_peers.cc:597] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:26:02.762501  1517 tablet_replica.cc:333] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.788919  1517 raft_consensus.cc:2243] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:02.767572 11310 raft_consensus.cc:493] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:02.789876 11310 raft_consensus.cc:515] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } }
I20260709 12:26:02.796903 11310 leader_election.cc:290] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:02.799001  5114 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:26:02.802336  5190 consensus_peers.cc:597] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:02.803646  1517 raft_consensus.cc:2272] T 59bc6f04e62a431ba2f113d08a129898 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.809656  1517 tablet_replica.cc:333] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.811381  1517 raft_consensus.cc:2243] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:26:02.813264  1517 raft_consensus.cc:2272] T 6a61a09c949a4cc9959bd9cfbe00992a P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:02.815042  5190 consensus_peers.cc:597] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:02.818174  1517 tablet_replica.cc:333] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.819670  1517 raft_consensus.cc:2243] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:02.820916  1517 raft_consensus.cc:2272] T 82e74ddaf2154bc1ba1aeb5775d6e514 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:02.827414  5115 leader_election.cc:336] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:02.828312  5115 leader_election.cc:304] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceec344dd00f40c18ef58414ce147016; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:02.830327 11310 raft_consensus.cc:2749] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:02.859670  1517 tablet_replica.cc:333] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.863142  1517 raft_consensus.cc:2243] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 14 LEADER]: Raft consensus shutting down.
W20260709 12:26:02.874423  5188 consensus_peers.cc:597] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:02.878665  1517 raft_consensus.cc:2272] T 7383bc919f9c47378977b98535752581 P ceec344dd00f40c18ef58414ce147016 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.886046  1517 tablet_replica.cc:333] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
W20260709 12:26:02.895821  5115 consensus_peers.cc:597] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:26:02.896479  1517 raft_consensus.cc:2243] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:26:02.899298  1517 raft_consensus.cc:2272] T 63938e2a81834411b8018a2eb6c6818f P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.911669  1517 tablet_replica.cc:333] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.913945  1517 raft_consensus.cc:2243] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:26:02.919039  1517 raft_consensus.cc:2272] T e4be6683658549c99c727b84f9cd9774 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.926258  1517 tablet_replica.cc:333] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
W20260709 12:26:02.935209  5188 consensus_peers.cc:597] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:02.936545  1517 raft_consensus.cc:2243] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:26:02.938489  1517 raft_consensus.cc:2272] T 4c6732ed2583444ab53a0bd4a87ff1d0 P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:02.961133  5115 consensus_peers.cc:597] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:26:02.973007  1517 tablet_replica.cc:333] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:02.983345  1517 raft_consensus.cc:2243] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:02.990694  1517 raft_consensus.cc:2272] T 3a7f46906b2d49fe83fd7c7729fc9d53 P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:02.995427  1517 tablet_replica.cc:333] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.013414  1517 raft_consensus.cc:2243] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.049500  1517 raft_consensus.cc:2272] T 2725c773e7344020b5752861d00b493e P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.055402  1517 tablet_replica.cc:333] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.062628  1517 raft_consensus.cc:2243] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.064527  1517 raft_consensus.cc:2272] T 5deada35125f4c008544b2ff246b039c P ceec344dd00f40c18ef58414ce147016 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.068732  1517 tablet_replica.cc:333] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.070098  1517 raft_consensus.cc:2243] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.073058  1517 raft_consensus.cc:2272] T 7dfc1cfb224a4fd989b97af5dcf7da23 P ceec344dd00f40c18ef58414ce147016 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.077936  1517 tablet_replica.cc:333] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.079634  1517 raft_consensus.cc:2243] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.081818  1517 raft_consensus.cc:2272] T eb1b479abeca4781986d8821b704c31f P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.088884  1517 tablet_replica.cc:333] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.090412  1517 raft_consensus.cc:2243] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.092775  1517 raft_consensus.cc:2272] T 719a453defab43d4a1232c664dbaae9c P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.097900  1517 tablet_replica.cc:333] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.099649  1517 raft_consensus.cc:2243] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.101960  1517 raft_consensus.cc:2272] T 5de3ceedceac4e0ca06de7a806ec05e3 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:03.111429  5115 consensus_peers.cc:597] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:26:03.113948  1517 tablet_replica.cc:333] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.115403  1517 raft_consensus.cc:2243] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.118180  1517 raft_consensus.cc:2272] T 805537066aa5440cbb0608aa77daafcf P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.123016  1517 tablet_replica.cc:333] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.124226  1517 raft_consensus.cc:2243] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.125625  1517 raft_consensus.cc:2272] T e77063f082034d8bbc280357b7b66bb7 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.130628  1517 tablet_replica.cc:333] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.132206  1517 raft_consensus.cc:2243] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.135488  1517 raft_consensus.cc:2272] T 85f48767a7ee419197aff24610838e25 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:03.140295  5188 consensus_peers.cc:597] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:03.148782  1517 tablet_replica.cc:333] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.150169  1517 raft_consensus.cc:2243] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.151367  1517 raft_consensus.cc:2272] T 5244acc0775748b7933286d588ec2b56 P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.157656  1517 tablet_replica.cc:333] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.159158  1517 raft_consensus.cc:2243] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.161037  1517 raft_consensus.cc:2272] T acca9bc33b6d43519a79138d66901f9b P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.196110  1517 tablet_replica.cc:333] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
W20260709 12:26:03.206944  5189 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:26:03.198092  1517 raft_consensus.cc:2243] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.216053  1517 raft_consensus.cc:2272] T 58bebed3b53f4f178dae221a0daae185 P ceec344dd00f40c18ef58414ce147016 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.232705  1517 tablet_replica.cc:333] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.233930  1517 raft_consensus.cc:2243] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.235745  1517 raft_consensus.cc:2272] T 874aa8edc94946d69e4a25563a5a0f0f P ceec344dd00f40c18ef58414ce147016 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.241443  1517 tablet_replica.cc:333] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.242689  1517 raft_consensus.cc:2243] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.244976  1517 raft_consensus.cc:2272] T c0a39e3f43214d8ea4ff72401ffdbdcb P ceec344dd00f40c18ef58414ce147016 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:03.249128  5114 consensus_peers.cc:597] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:03.251593  1517 tablet_replica.cc:333] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.258719  1517 raft_consensus.cc:2243] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.261694  1517 raft_consensus.cc:2272] T 09749d11015f4695b5c630c6e5dcbcb0 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.271270  1517 tablet_replica.cc:333] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.272409  1517 raft_consensus.cc:2243] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.274322  1517 raft_consensus.cc:2272] T db646a2919184c1682011b96d25d0cc9 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.281677  1517 tablet_replica.cc:333] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.283136  1517 raft_consensus.cc:2243] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.289930  1517 raft_consensus.cc:2272] T 15dbeb6b54ba4f0c80adc43d8f184cb5 P ceec344dd00f40c18ef58414ce147016 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.296226  1517 tablet_replica.cc:333] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.297883  1517 raft_consensus.cc:2243] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.300093  1517 raft_consensus.cc:2272] T 35db327bbc4d4777969286cb1174a85d P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.305840  1517 tablet_replica.cc:333] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.307116  1517 raft_consensus.cc:2243] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.308481  1517 raft_consensus.cc:2272] T 7be74bdf01fe4190992575e84da14996 P ceec344dd00f40c18ef58414ce147016 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.333613  1517 tablet_replica.cc:333] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.335510  1517 raft_consensus.cc:2243] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.337064  1517 raft_consensus.cc:2272] T 5e15328865c9479dab64cbc3fa161e86 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.355026  1517 tablet_replica.cc:333] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.356053  1517 raft_consensus.cc:2243] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.360183  1517 raft_consensus.cc:2272] T 9d055165bf9e4bfdb042cd7bd63ec993 P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.365374  1517 tablet_replica.cc:333] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.375310  1517 raft_consensus.cc:2243] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.377023  1517 raft_consensus.cc:2272] T 4516325f133e4ea8beacc1968f96ee9b P ceec344dd00f40c18ef58414ce147016 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:03.378041  5189 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:26:03.387349  1517 tablet_replica.cc:333] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.389170  1517 raft_consensus.cc:2243] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:26:03.391626  1517 raft_consensus.cc:2272] T 4eb4fbb07495443bac93cfbcfa13bbe2 P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:03.396488  1517 tablet_replica.cc:333] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016: stopping tablet replica
I20260709 12:26:03.397795  1517 raft_consensus.cc:2243] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:03.399387  1517 raft_consensus.cc:2272] T 63912da780fe41a2887dde6d2051ac8b P ceec344dd00f40c18ef58414ce147016 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:03.420459  5188 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.441381  5188 consensus_peers.cc:597] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.508150  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:03.527935  5188 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.567234  5188 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.572201  5188 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.588143  5188 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.641310  5189 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:03.705158  5188 consensus_peers.cc:597] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.719182  5188 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.733796  5188 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:03.765894 11308 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:03.766793 11308 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:26:03.771598 11308 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
I20260709 12:26:03.774279 11308 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:03.775252 11308 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:26:03.772900  5188 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.784946  5189 leader_election.cc:336] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:26:03.786367  5188 leader_election.cc:336] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:03.787060  5188 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:03.790776 11308 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:26:03.791515 11284 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:03.798318  5190 leader_election.cc:336] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:03.808636  5188 consensus_peers.cc:597] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.809330  5188 leader_election.cc:336] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:03.810335  5188 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:03.812139 11284 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:03.814302  5188 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.855515  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:26:03.955473 11308 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:03.956362 11308 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:26:03.959558 11308 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:03.976612  5188 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:03.980274  5188 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:26:03.986690  5189 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:03.995308  5190 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:04.001202  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:04.009130 11308 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:04.013407  5189 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:26:04.019901 11308 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:04.021013 11308 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:26:04.024411 11308 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:04.060281  5190 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:04.072453  5188 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:04.078815  5188 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:04.090857 11284 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:04.099387 11308 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:04.100172 11308 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:26:04.106755 11308 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:26:04.109602 11284 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:04.135020 11284 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:26:04.138877 11284 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
I20260709 12:26:04.158270 11308 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:04.160246 11308 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:26:04.159323  5189 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:26:04.173836 11308 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:26:04.204052 11314 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
W20260709 12:26:04.205992  5188 leader_election.cc:336] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:04.211727 11308 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:04.220023 11308 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:26:04.207587 11314 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:26:04.236451 11314 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:26:04.225785 11284 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:04.237246 11284 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:26:04.240337 11284 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:04.233072  5190 leader_election.cc:336] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:04.251839 11308 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:04.259271  5189 leader_election.cc:336] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:04.261277  5189 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:04.263849 11308 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:26:04.259629  5190 leader_election.cc:336] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:04.265204  5190 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:04.268358 11308 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:04.272717  5189 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:04.275110  5189 leader_election.cc:336] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:26:04.285344  5188 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:26:04.293294  5188 leader_election.cc:336] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:26:04.301594  5189 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:04.306711  5189 leader_election.cc:336] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:26:04.309522  5190 leader_election.cc:336] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:04.311717  5190 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:04.314746 11314 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:04.341754 11308 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:04.344810 11308 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:26:04.349129 11308 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:26:04.350345  5190 leader_election.cc:336] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:04.363724 11314 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:04.367503  5190 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:04.367720 11314 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:26:04.368659 11284 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:26:04.368683  5190 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:04.369674  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:26:04.370699  5190 leader_election.cc:336] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:04.371069  5190 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:04.373878 11306 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:04.374063 11314 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:26:04.375566 11314 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:04.365983  5189 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:04.387866  5189 leader_election.cc:336] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:26:04.406345  5190 leader_election.cc:336] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:04.406900  5190 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:26:04.408125  5190 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:04.408358 11308 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:04.410585 11308 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:04.411276 11308 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:26:04.420013  5190 leader_election.cc:336] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:04.420812  5189 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:04.421428  5189 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:04.424335 11315 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:04.415179 11308 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:26:04.435320  5189 leader_election.cc:336] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:04.436357  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:04.437438 11308 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:04.492388 11315 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:04.493057 11315 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:26:04.496024 11315 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:26:04.507402  5190 leader_election.cc:336] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:04.511097  5189 leader_election.cc:336] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:04.511889  5189 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:04.513283 11315 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:04.525767 11315 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:04.527014 11315 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:26:04.542053  5190 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:04.542764 11315 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:26:04.544327  5189 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:04.544879  5189 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:04.545965 11315 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:04.548830  5189 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:26:04.599406 11308 raft_consensus.cc:493] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:04.600484 11308 raft_consensus.cc:515] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:26:04.608471 11308 leader_election.cc:290] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:04.626508  5189 leader_election.cc:336] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:26:04.632340  5188 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:04.634887  5188 consensus_peers.cc:597] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:04.635532  5188 leader_election.cc:336] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:04.638187  5188 leader_election.cc:304] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:04.639173 11308 raft_consensus.cc:2749] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:04.772114  1517 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 5171)
I20260709 12:26:04.998561 11314 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:05.000381 11314 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:26:05.010900 11314 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:05.034001  5188 leader_election.cc:336] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:05.049144 11319 raft_consensus.cc:493] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:05.049839 11319 raft_consensus.cc:515] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:26:05.052937 11319 leader_election.cc:290] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:05.064616  5190 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.064702  5188 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:26:05.065515  5189 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:05.065924  5190 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.066323  5189 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:05.067446  5190 consensus_peers.cc:597] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.071992  5189 leader_election.cc:336] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:05.074430  5189 leader_election.cc:304] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
W20260709 12:26:05.082526  5190 consensus_peers.cc:597] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.083412  5190 consensus_peers.cc:597] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.083884  5190 consensus_peers.cc:597] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.084389  5190 leader_election.cc:336] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:05.084977 11319 raft_consensus.cc:2749] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:05.086275  5189 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:05.087745  5189 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:05.088344  5189 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:26:05.095256  5190 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:05.099267 11317 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:05.114352  5190 consensus_peers.cc:597] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.115553  5190 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.126299  5190 consensus_peers.cc:597] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.126957  5190 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.127470  5190 consensus_peers.cc:597] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.402057  5190 consensus_peers.cc:597] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.441184  5188 consensus_peers.cc:597] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:05.531224  5190 consensus_peers.cc:597] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:05.601161 11319 raft_consensus.cc:493] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:05.601984 11319 raft_consensus.cc:515] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:26:05.606089 11319 leader_election.cc:290] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:05.617187  5189 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:05.620445  5190 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:05.624439  5188 leader_election.cc:336] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:05.626231  5188 leader_election.cc:304] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:05.627452 11319 raft_consensus.cc:2749] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:05.639582  5188 consensus_peers.cc:597] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:05.643347  5189 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:05.659708  5190 consensus_peers.cc:597] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac -> Peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Couldn't send request to peer ceec344dd00f40c18ef58414ce147016. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:05.672324  4930 debug-util.cc:398] Leaking SignalData structure 0x7b0800174c20 after lost signal to thread 1520
W20260709 12:26:05.674391  4930 debug-util.cc:398] Leaking SignalData structure 0x7b08000572a0 after lost signal to thread 4933
W20260709 12:26:05.676477  4930 debug-util.cc:398] Leaking SignalData structure 0x7b08000e3800 after lost signal to thread 5248
I20260709 12:26:05.757181 11317 raft_consensus.cc:493] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:05.758536 11317 raft_consensus.cc:515] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:26:05.772413  5190 leader_election.cc:336] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:05.774019 11317 leader_election.cc:290] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:05.776249  5188 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111) [suppressed 635 similar messages]
W20260709 12:26:05.782084  5188 leader_election.cc:336] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:05.782860  5188 leader_election.cc:304] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:05.783888 11319 raft_consensus.cc:2749] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.808990 11319 raft_consensus.cc:493] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:05.809995 11319 raft_consensus.cc:515] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:26:05.826740  5190 leader_election.cc:336] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:05.828743  5189 leader_election.cc:336] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:05.829486  5189 leader_election.cc:304] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:05.830633 11317 raft_consensus.cc:2749] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:05.831835 11319 leader_election.cc:290] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:26:05.874174 11321 raft_consensus.cc:493] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:05.875310 11321 raft_consensus.cc:515] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:26:05.912627 11321 leader_election.cc:290] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:26:05.916388  5190 leader_election.cc:336] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:05.937577  5189 leader_election.cc:336] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:05.939401  5189 leader_election.cc:304] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:05.941278 11319 raft_consensus.cc:2749] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:05.995225 11319 raft_consensus.cc:493] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:05.995981 11319 raft_consensus.cc:515] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:26:05.999457 11319 leader_election.cc:290] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:26:06.030133  5188 consensus_peers.cc:597] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:06.036860  5190 leader_election.cc:336] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:06.037978 11321 raft_consensus.cc:493] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:06.053440 11321 raft_consensus.cc:515] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:26:06.057883 11321 leader_election.cc:290] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:06.073498  5189 leader_election.cc:336] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:26:06.080585  5188 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:06.078250  5189 leader_election.cc:304] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:06.089618 11319 raft_consensus.cc:493] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:06.090615 11319 raft_consensus.cc:515] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:26:06.093954 11319 leader_election.cc:290] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:06.098678  5189 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:26:06.099401 11319 raft_consensus.cc:2749] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:06.110682  5188 consensus_peers.cc:597] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:06.126144  5190 leader_election.cc:336] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:06.135607  5190 leader_election.cc:304] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
W20260709 12:26:06.133594  5188 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:06.140408 11319 raft_consensus.cc:2749] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:06.157202 11319 raft_consensus.cc:493] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:06.158023 11319 raft_consensus.cc:515] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
W20260709 12:26:06.164100  5190 leader_election.cc:336] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:06.165007  5190 leader_election.cc:304] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:06.166510 11317 raft_consensus.cc:2749] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:06.168911 11319 leader_election.cc:290] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:06.177127  5188 consensus_peers.cc:597] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:06.188076  5190 leader_election.cc:336] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:06.188048  5188 consensus_peers.cc:597] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:06.192814  5188 leader_election.cc:336] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:06.196398  5188 leader_election.cc:304] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:06.201118 11319 raft_consensus.cc:2749] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:06.235916  5188 consensus_peers.cc:597] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:06.244683  5188 consensus_peers.cc:597] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:06.245761  5188 consensus_peers.cc:597] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:06.257863  5189 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:06.277998  5189 consensus_peers.cc:597] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:06.289134  5188 consensus_peers.cc:597] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:06.309948 11319 raft_consensus.cc:493] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:06.310900 11321 raft_consensus.cc:493] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:06.311666 11319 raft_consensus.cc:515] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:26:06.312345 11321 raft_consensus.cc:515] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:26:06.315527 11321 leader_election.cc:290] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:06.319371  5188 consensus_peers.cc:597] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:06.323966 11319 leader_election.cc:290] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
I20260709 12:26:06.329221 11319 raft_consensus.cc:493] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:06.329972 11319 raft_consensus.cc:515] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
W20260709 12:26:06.331655  5190 leader_election.cc:336] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:06.332566  5190 leader_election.cc:336] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:06.333182 11319 leader_election.cc:290] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:26:06.338366  5188 leader_election.cc:336] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:06.339459  5188 leader_election.cc:304] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:06.341452 11319 raft_consensus.cc:2749] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:06.342316  5189 leader_election.cc:336] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:06.344420  5189 leader_election.cc:304] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
W20260709 12:26:06.343751  5190 leader_election.cc:336] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:06.346558 11319 raft_consensus.cc:2749] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:06.356601  5189 leader_election.cc:336] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:06.357954  5189 leader_election.cc:304] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:06.359340 11319 raft_consensus.cc:2749] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:06.417419  5188 consensus_peers.cc:597] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:06.426208  5188 consensus_peers.cc:597] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:06.429864 11321 raft_consensus.cc:493] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:06.430601 11321 raft_consensus.cc:515] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:26:06.440670 11321 leader_election.cc:290] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:26:06.444185  5189 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
W20260709 12:26:06.449769  5190 leader_election.cc:336] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
I20260709 12:26:06.450605  5190 leader_election.cc:304] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:06.451543 11321 raft_consensus.cc:2749] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:06.477823  5189 consensus_peers.cc:597] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:26:06.488462  1517 tablet_server.cc:196] TabletServer@127.1.123.67:0 shutdown complete.
W20260709 12:26:06.515960  5188 consensus_peers.cc:597] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:06.555094  5189 consensus_peers.cc:597] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:06.596589  5188 consensus_peers.cc:597] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:06.615711 11319 raft_consensus.cc:493] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:06.616557 11319 raft_consensus.cc:515] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } } peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } }
I20260709 12:26:06.620076 11319 leader_election.cc:290] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955), ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855)
W20260709 12:26:06.628000  5190 leader_election.cc:336] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:06.628696  5188 leader_election.cc:336] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
I20260709 12:26:06.629297  5188 leader_election.cc:304] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:06.631455 11319 raft_consensus.cc:2749] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:06.634789 11321 raft_consensus.cc:493] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:06.635892 11321 raft_consensus.cc:515] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:26:06.642345 11321 leader_election.cc:290] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:26:06.649474  5190 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:06.650101  5189 leader_election.cc:336] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:06.650552  5189 leader_election.cc:304] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:06.651291 11319 raft_consensus.cc:2749] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:06.656694 11319 raft_consensus.cc:493] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6dc0d06e662d49d88e1b153e09a44e4f)
I20260709 12:26:06.657449 11319 raft_consensus.cc:515] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } } peers { permanent_uuid: "6dc0d06e662d49d88e1b153e09a44e4f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 33955 } }
I20260709 12:26:06.660460 11319 leader_election.cc:290] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253), 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955)
W20260709 12:26:06.665670  5188 leader_election.cc:336] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111)
W20260709 12:26:06.668864  5189 leader_election.cc:336] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:06.669566  5189 leader_election.cc:304] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 6dc0d06e662d49d88e1b153e09a44e4f, 9847a7797ddc4df689a1ab0911af8785
I20260709 12:26:06.671133 11321 raft_consensus.cc:2749] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:06.673872  1517 tablet_server.cc:179] TabletServer@127.1.123.68:0 shutting down...
W20260709 12:26:06.735503  5188 consensus_peers.cc:597] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:06.742605 11319 raft_consensus.cc:493] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ceec344dd00f40c18ef58414ce147016)
I20260709 12:26:06.744148 11319 raft_consensus.cc:515] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceec344dd00f40c18ef58414ce147016" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 46855 } } peers { permanent_uuid: "90e5220a62ed42fe83c82d0302d18fac" member_type: VOTER last_known_addr { host: "127.1.123.68" port: 43449 } } peers { permanent_uuid: "9847a7797ddc4df689a1ab0911af8785" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 37253 } }
I20260709 12:26:06.749398 11319 leader_election.cc:290] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855), 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253)
W20260709 12:26:06.757218  5189 consensus_peers.cc:597] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:06.774585  5190 leader_election.cc:336] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ceec344dd00f40c18ef58414ce147016 (127.1.123.67:46855): Network error: Client connection negotiation failed: client connection to 127.1.123.67:46855: connect: Connection refused (error 111)
W20260709 12:26:06.778700  5189 leader_election.cc:336] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111)
I20260709 12:26:06.779510  5189 leader_election.cc:304] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90e5220a62ed42fe83c82d0302d18fac; no voters: 9847a7797ddc4df689a1ab0911af8785, ceec344dd00f40c18ef58414ce147016
I20260709 12:26:06.780350 11319 raft_consensus.cc:2749] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:06.827572  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:06.829082  1517 tablet_replica.cc:333] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.830418  1517 raft_consensus.cc:2243] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.832158  1517 raft_consensus.cc:2272] T 4da8de53d0db43f9a92906a98dfda98f P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:06.834493  5189 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 9847a7797ddc4df689a1ab0911af8785 (127.1.123.65:37253): Couldn't send request to peer 9847a7797ddc4df689a1ab0911af8785. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:37253: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:26:06.836331  1517 tablet_replica.cc:333] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.837623  1517 raft_consensus.cc:2243] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.839171  1517 raft_consensus.cc:2272] T 3665d1ab30ab4c8d9b7304c26a0cd138 P 90e5220a62ed42fe83c82d0302d18fac [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.843174  1517 tablet_replica.cc:333] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.844414  1517 raft_consensus.cc:2243] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.846410  1517 raft_consensus.cc:2272] T 0141a059bf6949c49e5e0091604786e6 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.851500  1517 tablet_replica.cc:333] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.852983  1517 raft_consensus.cc:2243] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.854202  1517 raft_consensus.cc:2272] T 5b780c93046447618cab07ac8b85cdfd P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.859356  1517 tablet_replica.cc:333] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.860531  1517 raft_consensus.cc:2243] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.861289  1517 raft_consensus.cc:2272] T c5751113d37b44af87d3d912da388078 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.865273  1517 tablet_replica.cc:333] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.866824  1517 raft_consensus.cc:2243] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.868453  1517 raft_consensus.cc:2272] T 2725c773e7344020b5752861d00b493e P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.872954  1517 tablet_replica.cc:333] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.874115  1517 raft_consensus.cc:2243] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.875650  1517 raft_consensus.cc:2272] T 5deada35125f4c008544b2ff246b039c P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.880376  1517 tablet_replica.cc:333] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.882172  1517 raft_consensus.cc:2243] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.884254  1517 raft_consensus.cc:2272] T cf661d015f0e4fc19e57069da58329e3 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.889261  1517 tablet_replica.cc:333] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.890614  1517 raft_consensus.cc:2243] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.891892  1517 raft_consensus.cc:2272] T 1339c2a0d7f344d09f483e6d965adcc8 P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.896576  1517 tablet_replica.cc:333] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.898172  1517 raft_consensus.cc:2243] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.899868  1517 raft_consensus.cc:2272] T 003133c544c3449a827e3758faa02f02 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.903674  1517 tablet_replica.cc:333] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.905117  1517 raft_consensus.cc:2243] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.905841  1517 raft_consensus.cc:2272] T a8bbb28bc9f5410d95b20827beeb87cc P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.909905  1517 tablet_replica.cc:333] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.910972  1517 raft_consensus.cc:2243] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.911636  1517 raft_consensus.cc:2272] T c86395378750473e80c76902ad29fbe7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.914533  1517 tablet_replica.cc:333] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.915828  1517 raft_consensus.cc:2243] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.916833  1517 raft_consensus.cc:2272] T 6a61a09c949a4cc9959bd9cfbe00992a P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.921197  1517 tablet_replica.cc:333] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.922268  1517 raft_consensus.cc:2243] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.923285  1517 raft_consensus.cc:2272] T 82e74ddaf2154bc1ba1aeb5775d6e514 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.927924  1517 tablet_replica.cc:333] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.929656  1517 raft_consensus.cc:2243] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.931140  1517 raft_consensus.cc:2272] T 77f30d7c9f3244999f248bc1aed226c7 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.935490  1517 tablet_replica.cc:333] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.936467  1517 raft_consensus.cc:2243] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.938925  1517 raft_consensus.cc:2272] T 75210ca16c9d4d0e91d54b6ffbbd75d4 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.942696  1517 tablet_replica.cc:333] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.944222  1517 raft_consensus.cc:2243] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.944823  1517 raft_consensus.cc:2272] T 7383bc919f9c47378977b98535752581 P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.947503  1517 tablet_replica.cc:333] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.948249  1517 raft_consensus.cc:2243] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.949092  1517 raft_consensus.cc:2272] T e4be6683658549c99c727b84f9cd9774 P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.954056  1517 tablet_replica.cc:333] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.954985  1517 raft_consensus.cc:2243] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.956161  1517 raft_consensus.cc:2272] T 2aa8f9fb1ec8411aa7e23f87b827f019 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.959411  1517 tablet_replica.cc:333] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.960464  1517 raft_consensus.cc:2243] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.961782  1517 raft_consensus.cc:2272] T 7be74bdf01fe4190992575e84da14996 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.967769  1517 tablet_replica.cc:333] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.968673  1517 raft_consensus.cc:2243] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.970227  1517 raft_consensus.cc:2272] T 94a5838186ec4283b67fe9aca66d2a22 P 90e5220a62ed42fe83c82d0302d18fac [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.973348  1517 tablet_replica.cc:333] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.974058  1517 raft_consensus.cc:2243] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.975098  1517 raft_consensus.cc:2272] T 3a7f46906b2d49fe83fd7c7729fc9d53 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.977988  1517 tablet_replica.cc:333] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.978888  1517 raft_consensus.cc:2243] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.980106  1517 raft_consensus.cc:2272] T 4c6732ed2583444ab53a0bd4a87ff1d0 P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.986471  1517 tablet_replica.cc:333] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.987510  1517 raft_consensus.cc:2243] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.989310  1517 raft_consensus.cc:2272] T eb1b479abeca4781986d8821b704c31f P 90e5220a62ed42fe83c82d0302d18fac [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.992178  1517 tablet_replica.cc:333] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:06.993125  1517 raft_consensus.cc:2243] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.993870  1517 raft_consensus.cc:2272] T db646a2919184c1682011b96d25d0cc9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.999020  1517 tablet_replica.cc:333] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.001673  1517 raft_consensus.cc:2243] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:07.003520  1517 raft_consensus.cc:2272] T 35db327bbc4d4777969286cb1174a85d P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.008318  1517 tablet_replica.cc:333] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.009873  1517 raft_consensus.cc:2243] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:26:07.011407  1517 raft_consensus.cc:2272] T 58bebed3b53f4f178dae221a0daae185 P 90e5220a62ed42fe83c82d0302d18fac [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.015353  1517 tablet_replica.cc:333] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.017053  1517 raft_consensus.cc:2243] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:07.018420  1517 raft_consensus.cc:2272] T 4eb4fbb07495443bac93cfbcfa13bbe2 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.023914  1517 tablet_replica.cc:333] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.025758  1517 raft_consensus.cc:2243] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:07.026582  1517 raft_consensus.cc:2272] T 9d055165bf9e4bfdb042cd7bd63ec993 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.030041  1517 tablet_replica.cc:333] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.030740  1517 raft_consensus.cc:2243] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:26:07.031697  1517 raft_consensus.cc:2272] T c1c232e09b414797838d5e57d028ca97 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.034904  1517 tablet_replica.cc:333] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.035573  1517 raft_consensus.cc:2243] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:26:07.036803  1517 raft_consensus.cc:2272] T 2b6b7243efc34b21882fab443d3a3720 P 90e5220a62ed42fe83c82d0302d18fac [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.039885  1517 tablet_replica.cc:333] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.041177  1517 raft_consensus.cc:2243] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:07.042716  1517 raft_consensus.cc:2272] T 85f48767a7ee419197aff24610838e25 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.049247  1517 tablet_replica.cc:333] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.050908  1517 raft_consensus.cc:2243] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:07.052650  1517 raft_consensus.cc:2272] T 9ee83a16af0a40998fc58f3e882c6080 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.057710  1517 tablet_replica.cc:333] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.058586  1517 raft_consensus.cc:2243] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Raft consensus shutting down.
W20260709 12:26:07.059496  5188 consensus_peers.cc:597] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac -> Peer 6dc0d06e662d49d88e1b153e09a44e4f (127.1.123.66:33955): Couldn't send request to peer 6dc0d06e662d49d88e1b153e09a44e4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:33955: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:26:07.060003  1517 raft_consensus.cc:2272] T 5244acc0775748b7933286d588ec2b56 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.063930  1517 tablet_replica.cc:333] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.064626  1517 raft_consensus.cc:2243] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:26:07.065662  1517 raft_consensus.cc:2272] T a6f50ca0c2db44278637fcdffffc03a2 P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.068182  1517 tablet_replica.cc:333] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.069200  1517 raft_consensus.cc:2243] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:26:07.070796  1517 raft_consensus.cc:2272] T acca9bc33b6d43519a79138d66901f9b P 90e5220a62ed42fe83c82d0302d18fac [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.074832  1517 tablet_replica.cc:333] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.076015  1517 raft_consensus.cc:2243] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:26:07.077399  1517 raft_consensus.cc:2272] T 63912da780fe41a2887dde6d2051ac8b P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.080039  1517 tablet_replica.cc:333] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.080941  1517 raft_consensus.cc:2243] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:26:07.082149  1517 raft_consensus.cc:2272] T 6c91c376e83f484baca078b37cbf00db P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.087596  1517 tablet_replica.cc:333] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.088936  1517 raft_consensus.cc:2243] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:26:07.090631  1517 raft_consensus.cc:2272] T 30a1d0ab099f4c37a8f1fbe48688ac24 P 90e5220a62ed42fe83c82d0302d18fac [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.094626  1517 tablet_replica.cc:333] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.095247  1517 raft_consensus.cc:2243] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:26:07.096215  1517 raft_consensus.cc:2272] T 159b5709d37c488d935b1fe85d29c5bc P 90e5220a62ed42fe83c82d0302d18fac [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.099448  1517 tablet_replica.cc:333] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.100033  1517 raft_consensus.cc:2243] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:26:07.101050  1517 raft_consensus.cc:2272] T bfdb4c20c33743d79d57552db35c484b P 90e5220a62ed42fe83c82d0302d18fac [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.105118  1517 tablet_replica.cc:333] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.105825  1517 raft_consensus.cc:2243] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:07.106482  1517 raft_consensus.cc:2272] T f6a42608cf874aa6943aaf8e349714b9 P 90e5220a62ed42fe83c82d0302d18fac [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.112169  1517 tablet_replica.cc:333] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.112742  1517 raft_consensus.cc:2243] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:07.113386  1517 raft_consensus.cc:2272] T c0a39e3f43214d8ea4ff72401ffdbdcb P 90e5220a62ed42fe83c82d0302d18fac [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:07.116253  1517 tablet_replica.cc:333] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac: stopping tablet replica
I20260709 12:26:07.116868  1517 raft_consensus.cc:2243] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:26:07.118184  1517 raft_consensus.cc:2272] T 5e15328865c9479dab64cbc3fa161e86 P 90e5220a62ed42fe83c82d0302d18fac [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:08.751335  1517 tablet_server.cc:196] TabletServer@127.1.123.68:0 shutdown complete.
I20260709 12:26:08.876744  1517 master.cc:562] Master@127.1.123.126:38285 shutting down...
I20260709 12:26:08.956051  1517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:08.957275  1517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:08.957810  1517 tablet_replica.cc:333] T 00000000000000000000000000000000 P ce6fc7638c8549faa2535fb5774c32c1: stopping tablet replica
I20260709 12:26:09.165191  1517 master.cc:584] Master@127.1.123.126:38285 shutdown complete.
I20260709 12:26:09.252825  1517 test_util.cc:182] -----------------------------------------------
I20260709 12:26:09.253077  1517 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599626433458-1517-0
[  FAILED  ] LeaderRebalancerTest.RestartTserver (198339 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:26:09.259496  1517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.123.126:42475
I20260709 12:26:09.261150  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:09.273573 11326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:09.273799  1517 server_base.cc:1061] running on GCE node
W20260709 12:26:09.274030 11323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:09.274030 11324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:09.279076  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:09.279489  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:09.279680  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599969279644 us; error 0 us; skew 500 ppm
I20260709 12:26:09.280576  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:09.284420  1517 webserver.cc:533] Webserver started at http://127.1.123.126:39279/ using document root <none> and password file <none>
I20260709 12:26:09.285126  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:09.285378  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:09.285730  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:09.287314  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/master-0-root/instance:
uuid: "4efc77e575344a799d4727926b1f8c3d"
format_stamp: "Formatted at 2026-07-09 12:26:09 on dist-test-slave-08l4"
I20260709 12:26:09.293316  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:26:09.299160 11331 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:09.300449  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:26:09.300927  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/master-0-root
uuid: "4efc77e575344a799d4727926b1f8c3d"
format_stamp: "Formatted at 2026-07-09 12:26:09 on dist-test-slave-08l4"
I20260709 12:26:09.301303  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:09.324680  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:09.326400  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:09.387145  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.126:42475
I20260709 12:26:09.387282 11382 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.126:42475 every 8 connection(s)
I20260709 12:26:09.392638 11383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:09.407606 11383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d: Bootstrap starting.
I20260709 12:26:09.413401 11383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:09.419059 11383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d: No bootstrap required, opened a new log
I20260709 12:26:09.422013 11383 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4efc77e575344a799d4727926b1f8c3d" member_type: VOTER }
I20260709 12:26:09.422503 11383 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:09.422801 11383 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4efc77e575344a799d4727926b1f8c3d, State: Initialized, Role: FOLLOWER
I20260709 12:26:09.424028 11383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4efc77e575344a799d4727926b1f8c3d" member_type: VOTER }
I20260709 12:26:09.425192 11383 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:26:09.425483 11383 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:26:09.425887 11383 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:09.431768 11383 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4efc77e575344a799d4727926b1f8c3d" member_type: VOTER }
I20260709 12:26:09.432436 11383 leader_election.cc:304] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4efc77e575344a799d4727926b1f8c3d; no voters: 
I20260709 12:26:09.433928 11383 leader_election.cc:290] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:26:09.434459 11386 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:09.436559 11386 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [term 1 LEADER]: Becoming Leader. State: Replica: 4efc77e575344a799d4727926b1f8c3d, State: Running, Role: LEADER
I20260709 12:26:09.437580 11386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4efc77e575344a799d4727926b1f8c3d" member_type: VOTER }
I20260709 12:26:09.438163 11383 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:26:09.450241 11388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4efc77e575344a799d4727926b1f8c3d. Latest consensus state: current_term: 1 leader_uuid: "4efc77e575344a799d4727926b1f8c3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4efc77e575344a799d4727926b1f8c3d" member_type: VOTER } }
I20260709 12:26:09.450241 11387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4efc77e575344a799d4727926b1f8c3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4efc77e575344a799d4727926b1f8c3d" member_type: VOTER } }
I20260709 12:26:09.451277 11387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [sys.catalog]: This master's current role is: LEADER
I20260709 12:26:09.453131 11388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [sys.catalog]: This master's current role is: LEADER
I20260709 12:26:09.453780 11396 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:26:09.460544 11396 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:26:09.461997  1517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:26:09.474982 11396 catalog_manager.cc:1383] Generated new cluster ID: 034929160f2f4186b8c8ac252100e6d2
I20260709 12:26:09.475443 11396 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:26:09.496979 11396 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:26:09.499033 11396 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:26:09.518405 11396 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d: Generated new TSK 0
I20260709 12:26:09.519515 11396 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:26:09.529791  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:09.539582 11404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:09.540872 11405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:09.543171 11407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:09.545106  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:09.547735  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:09.547977  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:09.548096  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599969548085 us; error 0 us; skew 500 ppm
I20260709 12:26:09.548664  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:09.552690  1517 webserver.cc:533] Webserver started at http://127.1.123.65:44853/ using document root <none> and password file <none>
I20260709 12:26:09.553882  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:09.554131  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:09.554394  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:09.555927  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-0-root/instance:
uuid: "2cc1d50178294d2aa84107ddd51b0243"
format_stamp: "Formatted at 2026-07-09 12:26:09 on dist-test-slave-08l4"
I20260709 12:26:09.562597  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.003s
I20260709 12:26:09.570298 11412 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:09.572111  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:26:09.572697  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-0-root
uuid: "2cc1d50178294d2aa84107ddd51b0243"
format_stamp: "Formatted at 2026-07-09 12:26:09 on dist-test-slave-08l4"
I20260709 12:26:09.573392  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:09.611730  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:09.613233  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:09.615288  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:09.619647  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:09.619963  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:09.620245  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:09.620440  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:09.694139  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.65:34807
I20260709 12:26:09.694335 11474 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.65:34807 every 8 connection(s)
I20260709 12:26:09.700347  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:09.713825 11479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:09.715142 11480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:09.718367 11482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:09.721243  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:09.722747 11475 heartbeater.cc:344] Connected to a master server at 127.1.123.126:42475
I20260709 12:26:09.723376 11475 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:09.724514 11475 heartbeater.cc:507] Master 127.1.123.126:42475 requested a full tablet report, sending...
I20260709 12:26:09.725359  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:09.725909  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:09.726151  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599969726127 us; error 0 us; skew 500 ppm
I20260709 12:26:09.727072  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:09.727802 11348 ts_manager.cc:194] Registered new tserver with Master: 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807)
I20260709 12:26:09.732102  1517 webserver.cc:533] Webserver started at http://127.1.123.66:42011/ using document root <none> and password file <none>
I20260709 12:26:09.733009  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:09.733290  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:09.733366 11348 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46374
I20260709 12:26:09.733697  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:09.736699  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-1-root/instance:
uuid: "4913cffad83a48c9884efb39bd85add1"
format_stamp: "Formatted at 2026-07-09 12:26:09 on dist-test-slave-08l4"
I20260709 12:26:09.744285  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:26:09.750010 11487 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:09.751833  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:26:09.752319  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-1-root
uuid: "4913cffad83a48c9884efb39bd85add1"
format_stamp: "Formatted at 2026-07-09 12:26:09 on dist-test-slave-08l4"
I20260709 12:26:09.752681  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:09.766866  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:09.768579  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:09.770537  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:09.773768  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:09.774070  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:09.774343  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:09.774521  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:09.842617  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.66:37319
I20260709 12:26:09.842870 11549 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.66:37319 every 8 connection(s)
I20260709 12:26:09.849412  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:09.860128 11553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:09.860486 11554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:09.867332 11550 heartbeater.cc:344] Connected to a master server at 127.1.123.126:42475
I20260709 12:26:09.868068 11550 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:09.868489  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:09.869246 11550 heartbeater.cc:507] Master 127.1.123.126:42475 requested a full tablet report, sending...
W20260709 12:26:09.869740 11556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:09.871246  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:09.871605  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:09.871796  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599969871779 us; error 0 us; skew 500 ppm
I20260709 12:26:09.872121 11348 ts_manager.cc:194] Registered new tserver with Master: 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:09.872440  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:09.874372 11348 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46382
I20260709 12:26:09.876088  1517 webserver.cc:533] Webserver started at http://127.1.123.67:36703/ using document root <none> and password file <none>
I20260709 12:26:09.877130  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:09.877718  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:09.878145  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:09.879630  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-2-root/instance:
uuid: "7e09e4b3d55044cd82210564ac064e9a"
format_stamp: "Formatted at 2026-07-09 12:26:09 on dist-test-slave-08l4"
I20260709 12:26:09.885968  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:26:09.890846 11561 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:09.892005  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:26:09.892469  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-2-root
uuid: "7e09e4b3d55044cd82210564ac064e9a"
format_stamp: "Formatted at 2026-07-09 12:26:09 on dist-test-slave-08l4"
I20260709 12:26:09.892858  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:09.912595  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:09.914314  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:09.916247  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:09.919257  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:09.919580  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:09.919909  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:09.920149  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:09.988390  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.67:38901
I20260709 12:26:09.988721 11623 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.67:38901 every 8 connection(s)
I20260709 12:26:10.008430 11624 heartbeater.cc:344] Connected to a master server at 127.1.123.126:42475
I20260709 12:26:10.009253 11624 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:10.010784 11624 heartbeater.cc:507] Master 127.1.123.126:42475 requested a full tablet report, sending...
I20260709 12:26:10.013715 11348 ts_manager.cc:194] Registered new tserver with Master: 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:10.013942  1517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.021364084s
I20260709 12:26:10.016026 11348 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46392
I20260709 12:26:10.055217 11348 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46394:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:26:10.058540 11348 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:26:10.179548 11588 tablet_service.cc:1511] Processing CreateTablet for tablet 264556df3cbe4b28bd544ab1863a7cbe (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:26:10.179754 11589 tablet_service.cc:1511] Processing CreateTablet for tablet f6dd4115f9464d44a08b8e58f95486cb (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:26:10.181753 11588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 264556df3cbe4b28bd544ab1863a7cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.183912 11589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6dd4115f9464d44a08b8e58f95486cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.183521 11587 tablet_service.cc:1511] Processing CreateTablet for tablet 7f0e176bc87049d19ea24e510329e131 (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:26:10.185621 11587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f0e176bc87049d19ea24e510329e131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.197291 11515 tablet_service.cc:1511] Processing CreateTablet for tablet f6dd4115f9464d44a08b8e58f95486cb (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:26:10.197944 11514 tablet_service.cc:1511] Processing CreateTablet for tablet 264556df3cbe4b28bd544ab1863a7cbe (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:26:10.199105 11515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6dd4115f9464d44a08b8e58f95486cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.199873 11514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 264556df3cbe4b28bd544ab1863a7cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.205443 11439 tablet_service.cc:1511] Processing CreateTablet for tablet 264556df3cbe4b28bd544ab1863a7cbe (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:26:10.206766 11440 tablet_service.cc:1511] Processing CreateTablet for tablet f6dd4115f9464d44a08b8e58f95486cb (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:26:10.208062 11439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 264556df3cbe4b28bd544ab1863a7cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.210057 11440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6dd4115f9464d44a08b8e58f95486cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.213686 11438 tablet_service.cc:1511] Processing CreateTablet for tablet 7f0e176bc87049d19ea24e510329e131 (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:26:10.216926 11438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f0e176bc87049d19ea24e510329e131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.234628 11513 tablet_service.cc:1511] Processing CreateTablet for tablet 7f0e176bc87049d19ea24e510329e131 (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:26:10.235605 11437 tablet_service.cc:1511] Processing CreateTablet for tablet a5750167a0c848b2afc5b03bda200dcd (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:26:10.236577 11513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f0e176bc87049d19ea24e510329e131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.237341 11437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5750167a0c848b2afc5b03bda200dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.247026 11586 tablet_service.cc:1511] Processing CreateTablet for tablet a5750167a0c848b2afc5b03bda200dcd (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:26:10.249120 11586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5750167a0c848b2afc5b03bda200dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.248178 11436 tablet_service.cc:1511] Processing CreateTablet for tablet 519e696ec65c4f19af8bda167a33baa3 (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:26:10.261512 11436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 519e696ec65c4f19af8bda167a33baa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.268447 11512 tablet_service.cc:1511] Processing CreateTablet for tablet a5750167a0c848b2afc5b03bda200dcd (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:26:10.270578 11512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5750167a0c848b2afc5b03bda200dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.275882 11585 tablet_service.cc:1511] Processing CreateTablet for tablet 519e696ec65c4f19af8bda167a33baa3 (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:26:10.277261 11435 tablet_service.cc:1511] Processing CreateTablet for tablet 1337a2bbcb024c7694ba59774bf25c17 (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:26:10.278905 11585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 519e696ec65c4f19af8bda167a33baa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.279268 11435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1337a2bbcb024c7694ba59774bf25c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.287134 11511 tablet_service.cc:1511] Processing CreateTablet for tablet 519e696ec65c4f19af8bda167a33baa3 (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:26:10.289144 11511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 519e696ec65c4f19af8bda167a33baa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.307675 11642 tablet_bootstrap.cc:492] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a: Bootstrap starting.
I20260709 12:26:10.311863 11510 tablet_service.cc:1511] Processing CreateTablet for tablet 1337a2bbcb024c7694ba59774bf25c17 (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:26:10.316838 11510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1337a2bbcb024c7694ba59774bf25c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.342212 11642 tablet_bootstrap.cc:654] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.366918 11509 tablet_service.cc:1511] Processing CreateTablet for tablet 19f60a8d99194fde8fa4b969c5a35b3a (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:26:10.368808 11509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19f60a8d99194fde8fa4b969c5a35b3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.354034 11589 tablet_service.cc:1511] Processing CreateTablet for tablet 1337a2bbcb024c7694ba59774bf25c17 (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:26:10.354295 11434 tablet_service.cc:1511] Processing CreateTablet for tablet 19f60a8d99194fde8fa4b969c5a35b3a (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:26:10.372186 11434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19f60a8d99194fde8fa4b969c5a35b3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.379626 11589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1337a2bbcb024c7694ba59774bf25c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.386376 11585 tablet_service.cc:1511] Processing CreateTablet for tablet 19f60a8d99194fde8fa4b969c5a35b3a (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:26:10.422892 11585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19f60a8d99194fde8fa4b969c5a35b3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.399186 11433 tablet_service.cc:1511] Processing CreateTablet for tablet 93d2f6a8f3a7462c90f0047ae8cdf4fa (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:26:10.430101 11642 tablet_bootstrap.cc:492] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a: No bootstrap required, opened a new log
I20260709 12:26:10.431263 11433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93d2f6a8f3a7462c90f0047ae8cdf4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.448944 11588 tablet_service.cc:1511] Processing CreateTablet for tablet 93d2f6a8f3a7462c90f0047ae8cdf4fa (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:26:10.451830 11588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93d2f6a8f3a7462c90f0047ae8cdf4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.455143 11438 tablet_service.cc:1511] Processing CreateTablet for tablet 0e67ad15454a410a9d37970d963b319b (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:26:10.457278 11438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e67ad15454a410a9d37970d963b319b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.461843 11510 tablet_service.cc:1511] Processing CreateTablet for tablet 93d2f6a8f3a7462c90f0047ae8cdf4fa (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:26:10.463517 11510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93d2f6a8f3a7462c90f0047ae8cdf4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.464603 11509 tablet_service.cc:1511] Processing CreateTablet for tablet 0e67ad15454a410a9d37970d963b319b (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:26:10.466207 11509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e67ad15454a410a9d37970d963b319b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:26:10.457860 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:10.470811 11513 tablet_service.cc:1511] Processing CreateTablet for tablet 2b88be8610a74af1ae2878b2e27da46d (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:26:10.473066 11513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b88be8610a74af1ae2878b2e27da46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.460333 11584 tablet_service.cc:1511] Processing CreateTablet for tablet 0e67ad15454a410a9d37970d963b319b (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:26:10.478264 11642 ts_tablet_manager.cc:1403] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a: Time spent bootstrapping tablet: real 0.171s	user 0.018s	sys 0.013s
I20260709 12:26:10.478989 11584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e67ad15454a410a9d37970d963b319b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.484040 11645 tablet_bootstrap.cc:492] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1: Bootstrap starting.
I20260709 12:26:10.485801 11439 tablet_service.cc:1511] Processing CreateTablet for tablet 2b88be8610a74af1ae2878b2e27da46d (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:26:10.487479 11439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b88be8610a74af1ae2878b2e27da46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.516237 11644 tablet_bootstrap.cc:492] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:10.515669 11642 raft_consensus.cc:359] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.530522 11642 raft_consensus.cc:385] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.531129 11642 raft_consensus.cc:740] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.505208 11586 tablet_service.cc:1511] Processing CreateTablet for tablet 2b88be8610a74af1ae2878b2e27da46d (DEFAULT_TABLE table=test-workload [id=82e2936280b24573b52cadae0fb5cb64]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:26:10.517988 11645 tablet_bootstrap.cc:654] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.533743 11644 tablet_bootstrap.cc:654] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.534044 11642 consensus_queue.cc:260] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.548182 11586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b88be8610a74af1ae2878b2e27da46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:10.567399 11644 tablet_bootstrap.cc:492] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243: No bootstrap required, opened a new log
I20260709 12:26:10.568886 11644 ts_tablet_manager.cc:1403] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.053s	user 0.016s	sys 0.012s
I20260709 12:26:10.570658 11624 heartbeater.cc:499] Master 127.1.123.126:42475 was elected leader, sending a full tablet report...
I20260709 12:26:10.571412 11645 tablet_bootstrap.cc:492] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1: No bootstrap required, opened a new log
I20260709 12:26:10.572211 11645 ts_tablet_manager.cc:1403] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1: Time spent bootstrapping tablet: real 0.090s	user 0.015s	sys 0.004s
I20260709 12:26:10.577739 11644 raft_consensus.cc:359] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:10.579306 11644 raft_consensus.cc:385] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.577999 11645 raft_consensus.cc:359] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:10.579802 11644 raft_consensus.cc:740] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.580127 11645 raft_consensus.cc:385] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.581027 11645 raft_consensus.cc:740] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.581557 11644 consensus_queue.cc:260] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:10.582240 11645 consensus_queue.cc:260] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:10.590941 11645 ts_tablet_manager.cc:1434] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1: Time spent starting tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 12:26:10.592092 11550 heartbeater.cc:499] Master 127.1.123.126:42475 was elected leader, sending a full tablet report...
I20260709 12:26:10.580338 11642 ts_tablet_manager.cc:1434] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a: Time spent starting tablet: real 0.101s	user 0.006s	sys 0.008s
I20260709 12:26:10.596524 11642 tablet_bootstrap.cc:492] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a: Bootstrap starting.
W20260709 12:26:10.600080 11551 tablet.cc:2406] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:26:10.602201 11645 tablet_bootstrap.cc:492] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1: Bootstrap starting.
I20260709 12:26:10.612170 11475 heartbeater.cc:499] Master 127.1.123.126:42475 was elected leader, sending a full tablet report...
I20260709 12:26:10.635649 11644 ts_tablet_manager.cc:1434] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.065s	user 0.025s	sys 0.011s
I20260709 12:26:10.637689 11644 tablet_bootstrap.cc:492] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:10.640543 11645 tablet_bootstrap.cc:654] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.640756 11642 tablet_bootstrap.cc:654] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.667137 11644 tablet_bootstrap.cc:654] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.669284 11642 tablet_bootstrap.cc:492] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a: No bootstrap required, opened a new log
I20260709 12:26:10.670197 11642 ts_tablet_manager.cc:1403] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a: Time spent bootstrapping tablet: real 0.074s	user 0.020s	sys 0.005s
I20260709 12:26:10.675027 11642 raft_consensus.cc:359] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:10.681274 11642 raft_consensus.cc:385] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.681717 11642 raft_consensus.cc:740] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.683427 11642 consensus_queue.cc:260] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:10.689121 11642 ts_tablet_manager.cc:1434] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a: Time spent starting tablet: real 0.019s	user 0.012s	sys 0.000s
I20260709 12:26:10.703826 11642 tablet_bootstrap.cc:492] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a: Bootstrap starting.
I20260709 12:26:10.708755 11644 tablet_bootstrap.cc:492] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243: No bootstrap required, opened a new log
I20260709 12:26:10.709494 11644 ts_tablet_manager.cc:1403] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.072s	user 0.029s	sys 0.019s
I20260709 12:26:10.712641 11642 tablet_bootstrap.cc:654] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.715190 11644 raft_consensus.cc:359] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.716128 11644 raft_consensus.cc:385] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.716478 11644 raft_consensus.cc:740] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.717451 11644 consensus_queue.cc:260] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.723881 11645 tablet_bootstrap.cc:492] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1: No bootstrap required, opened a new log
I20260709 12:26:10.724999 11645 ts_tablet_manager.cc:1403] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1: Time spent bootstrapping tablet: real 0.123s	user 0.022s	sys 0.011s
I20260709 12:26:10.725739 11644 ts_tablet_manager.cc:1434] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.006s
I20260709 12:26:10.727099 11642 tablet_bootstrap.cc:492] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a: No bootstrap required, opened a new log
I20260709 12:26:10.728032 11644 tablet_bootstrap.cc:492] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:10.728108 11642 ts_tablet_manager.cc:1403] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.004s
I20260709 12:26:10.733091 11642 raft_consensus.cc:359] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.734760 11642 raft_consensus.cc:385] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.733429 11645 raft_consensus.cc:359] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.735150 11642 raft_consensus.cc:740] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.735430 11645 raft_consensus.cc:385] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.736153 11642 consensus_queue.cc:260] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.736501 11645 raft_consensus.cc:740] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.738617 11645 consensus_queue.cc:260] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.739776 11644 tablet_bootstrap.cc:654] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.741297 11642 ts_tablet_manager.cc:1434] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 12:26:10.743173 11645 ts_tablet_manager.cc:1434] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.004s
I20260709 12:26:10.746471 11645 tablet_bootstrap.cc:492] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1: Bootstrap starting.
I20260709 12:26:10.750877 11642 tablet_bootstrap.cc:492] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a: Bootstrap starting.
I20260709 12:26:10.755560 11645 tablet_bootstrap.cc:654] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.759915 11642 tablet_bootstrap.cc:654] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.766888 11644 tablet_bootstrap.cc:492] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243: No bootstrap required, opened a new log
I20260709 12:26:10.767889 11644 ts_tablet_manager.cc:1403] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.008s
I20260709 12:26:10.772568 11644 raft_consensus.cc:359] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.773509 11642 tablet_bootstrap.cc:492] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a: No bootstrap required, opened a new log
I20260709 12:26:10.774374 11644 raft_consensus.cc:385] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.774820 11644 raft_consensus.cc:740] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.774874 11642 ts_tablet_manager.cc:1403] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.004s
I20260709 12:26:10.776109 11644 consensus_queue.cc:260] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.778007 11645 tablet_bootstrap.cc:492] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1: No bootstrap required, opened a new log
I20260709 12:26:10.778738 11645 ts_tablet_manager.cc:1403] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.012s
I20260709 12:26:10.779326 11644 ts_tablet_manager.cc:1434] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.011s	user 0.012s	sys 0.000s
I20260709 12:26:10.779287 11642 raft_consensus.cc:359] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.780539 11642 raft_consensus.cc:385] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.780921 11642 raft_consensus.cc:740] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.781032 11644 tablet_bootstrap.cc:492] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:10.782059 11642 consensus_queue.cc:260] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.782541 11645 raft_consensus.cc:359] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.783474 11645 raft_consensus.cc:385] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.783835 11645 raft_consensus.cc:740] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.784955 11645 consensus_queue.cc:260] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.787771 11642 ts_tablet_manager.cc:1434] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 12:26:10.789202 11642 tablet_bootstrap.cc:492] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a: Bootstrap starting.
I20260709 12:26:10.789721 11644 tablet_bootstrap.cc:654] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.792304 11645 ts_tablet_manager.cc:1434] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:26:10.793936 11645 tablet_bootstrap.cc:492] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1: Bootstrap starting.
I20260709 12:26:10.796540 11642 tablet_bootstrap.cc:654] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.801446 11650 raft_consensus.cc:493] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.802513 11650 raft_consensus.cc:515] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:10.804298 11645 tablet_bootstrap.cc:654] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.809885 11642 tablet_bootstrap.cc:492] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a: No bootstrap required, opened a new log
I20260709 12:26:10.810575 11642 ts_tablet_manager.cc:1403] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 12:26:10.820259 11642 raft_consensus.cc:359] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.821730 11650 leader_election.cc:290] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901), 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807)
I20260709 12:26:10.823489 11645 tablet_bootstrap.cc:492] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1: No bootstrap required, opened a new log
I20260709 12:26:10.823784 11642 raft_consensus.cc:385] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.824671 11645 ts_tablet_manager.cc:1403] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1: Time spent bootstrapping tablet: real 0.031s	user 0.023s	sys 0.001s
I20260709 12:26:10.825090 11642 raft_consensus.cc:740] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.827293 11642 consensus_queue.cc:260] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.830757 11645 raft_consensus.cc:359] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.832266 11645 raft_consensus.cc:385] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.832857 11645 raft_consensus.cc:740] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.835237 11642 ts_tablet_manager.cc:1434] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.001s
I20260709 12:26:10.835038 11645 consensus_queue.cc:260] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.841888 11645 ts_tablet_manager.cc:1434] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1: Time spent starting tablet: real 0.017s	user 0.014s	sys 0.003s
I20260709 12:26:10.843633 11645 tablet_bootstrap.cc:492] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1: Bootstrap starting.
I20260709 12:26:10.845072 11642 tablet_bootstrap.cc:492] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a: Bootstrap starting.
I20260709 12:26:10.845337 11649 raft_consensus.cc:493] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.850225 11649 raft_consensus.cc:515] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:10.852144 11646 raft_consensus.cc:493] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.852814 11646 raft_consensus.cc:515] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.855963 11646 leader_election.cc:290] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:10.862932 11645 tablet_bootstrap.cc:654] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.869825 11649 leader_election.cc:290] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:10.908864 11450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6dd4115f9464d44a08b8e58f95486cb" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:10.910306 11450 raft_consensus.cc:2468] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 0.
I20260709 12:26:10.918476 11642 tablet_bootstrap.cc:654] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.930166 11644 tablet_bootstrap.cc:492] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243: No bootstrap required, opened a new log
I20260709 12:26:10.931041 11644 ts_tablet_manager.cc:1403] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.150s	user 0.071s	sys 0.025s
I20260709 12:26:10.935002 11644 raft_consensus.cc:359] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.935892 11644 raft_consensus.cc:385] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.936234 11644 raft_consensus.cc:740] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.939101 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6dd4115f9464d44a08b8e58f95486cb" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a" is_pre_election: true
I20260709 12:26:10.940074 11599 raft_consensus.cc:2468] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 0.
I20260709 12:26:10.941401 11489 leader_election.cc:304] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:10.942703 11650 raft_consensus.cc:2804] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:10.943225 11650 raft_consensus.cc:493] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.943769 11650 raft_consensus.cc:3060] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:10.937167 11644 consensus_queue.cc:260] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.952862 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6dd4115f9464d44a08b8e58f95486cb" candidate_uuid: "2cc1d50178294d2aa84107ddd51b0243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1" is_pre_election: true
I20260709 12:26:10.955694 11645 tablet_bootstrap.cc:492] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1: No bootstrap required, opened a new log
I20260709 12:26:10.956346 11645 ts_tablet_manager.cc:1403] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1: Time spent bootstrapping tablet: real 0.113s	user 0.025s	sys 0.063s
I20260709 12:26:10.959901 11645 raft_consensus.cc:359] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.960610 11645 raft_consensus.cc:385] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:10.960904 11645 raft_consensus.cc:740] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Initialized, Role: FOLLOWER
I20260709 12:26:10.960974 11650 raft_consensus.cc:515] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:10.962082 11644 ts_tablet_manager.cc:1434] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.000s
I20260709 12:26:10.962512 11525 raft_consensus.cc:2393] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2cc1d50178294d2aa84107ddd51b0243 in current term 1: Already voted for candidate 4913cffad83a48c9884efb39bd85add1 in this term.
I20260709 12:26:10.963392 11644 tablet_bootstrap.cc:492] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:10.967820 11646 raft_consensus.cc:493] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.968396 11646 raft_consensus.cc:515] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.970232 11450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:10.971474 11450 raft_consensus.cc:2468] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 0.
I20260709 12:26:10.971880 11646 leader_election.cc:290] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:10.970237 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1" is_pre_election: true
I20260709 12:26:10.976091 11645 consensus_queue.cc:260] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.977662 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6dd4115f9464d44a08b8e58f95486cb" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:10.976176 11650 leader_election.cc:290] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 election: Requested vote from peers 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901), 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807)
I20260709 12:26:10.978631 11599 raft_consensus.cc:3060] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:10.978708 11450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6dd4115f9464d44a08b8e58f95486cb" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:10.979416 11450 raft_consensus.cc:3060] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:10.980170 11563 leader_election.cc:304] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:10.980875 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1" is_pre_election: true
I20260709 12:26:10.981775 11525 raft_consensus.cc:2468] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 0.
I20260709 12:26:10.982780 11644 tablet_bootstrap.cc:654] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:10.983793 11449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:10.984887 11449 raft_consensus.cc:2468] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 0.
I20260709 12:26:10.986999 11646 raft_consensus.cc:2804] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:10.987902 11646 raft_consensus.cc:493] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.988338 11646 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:10.989225 11676 raft_consensus.cc:493] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.989943 11676 raft_consensus.cc:515] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:10.992666 11676 leader_election.cc:290] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
W20260709 12:26:10.995637 11563 leader_election.cc:343] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:10.997185 11563 leader_election.cc:304] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:10.997960 11599 raft_consensus.cc:2468] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 1.
I20260709 12:26:10.999864 11449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0e176bc87049d19ea24e510329e131" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:11.000258 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0e176bc87049d19ea24e510329e131" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1" is_pre_election: true
I20260709 12:26:11.000833 11449 raft_consensus.cc:2468] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 0.
I20260709 12:26:11.001034 11525 raft_consensus.cc:2468] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 0.
I20260709 12:26:10.999311 11676 raft_consensus.cc:2804] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:11.002552 11676 raft_consensus.cc:493] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.002964 11676 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.004354 11491 leader_election.cc:304] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.005657 11650 raft_consensus.cc:2804] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:11.006363 11650 raft_consensus.cc:697] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:11.006722 11642 tablet_bootstrap.cc:492] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a: No bootstrap required, opened a new log
I20260709 12:26:11.007328 11650 consensus_queue.cc:237] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:11.009382 11642 ts_tablet_manager.cc:1403] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a: Time spent bootstrapping tablet: real 0.165s	user 0.016s	sys 0.006s
I20260709 12:26:11.010864 11563 leader_election.cc:304] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.013783 11450 raft_consensus.cc:2468] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 1.
I20260709 12:26:11.015275 11644 tablet_bootstrap.cc:492] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243: No bootstrap required, opened a new log
I20260709 12:26:11.017882 11645 ts_tablet_manager.cc:1434] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.008s
I20260709 12:26:11.018996 11676 raft_consensus.cc:515] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.019543 11645 tablet_bootstrap.cc:492] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1: Bootstrap starting.
I20260709 12:26:11.019930 11642 raft_consensus.cc:359] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.020756 11642 raft_consensus.cc:385] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.021072 11642 raft_consensus.cc:740] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.021486 11676 leader_election.cc:290] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:11.023613 11642 consensus_queue.cc:260] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.023113 11346 catalog_manager.cc:5697] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:11.024951 11646 raft_consensus.cc:515] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.026036 11644 ts_tablet_manager.cc:1403] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.003s
I20260709 12:26:11.030176 11645 tablet_bootstrap.cc:654] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.030165 11450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:11.028088 11646 leader_election.cc:290] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:11.031644 11450 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.022986 11677 raft_consensus.cc:2804] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:11.033310 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:11.035535 11525 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.037128 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6dd4115f9464d44a08b8e58f95486cb" candidate_uuid: "2cc1d50178294d2aa84107ddd51b0243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a" is_pre_election: true
I20260709 12:26:11.037632 11650 raft_consensus.cc:493] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.038117 11599 raft_consensus.cc:2393] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2cc1d50178294d2aa84107ddd51b0243 in current term 1: Already voted for candidate 4913cffad83a48c9884efb39bd85add1 in this term.
I20260709 12:26:11.038199 11650 raft_consensus.cc:515] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.038717 11524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:11.039481 11449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:11.040534 11449 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.041668 11650 leader_election.cc:290] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:11.032984 11644 raft_consensus.cc:359] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.044333 11448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1337a2bbcb024c7694ba59774bf25c17" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:11.044683 11644 raft_consensus.cc:385] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.046653 11644 raft_consensus.cc:740] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.044474 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1337a2bbcb024c7694ba59774bf25c17" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a" is_pre_election: true
W20260709 12:26:11.048393 11489 leader_election.cc:343] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:26:11.049952 11491 leader_election.cc:343] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:11.050453 11491 leader_election.cc:304] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1; no voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a
I20260709 12:26:11.044349 11416 leader_election.cc:304] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243; no voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a
I20260709 12:26:11.053239 11649 raft_consensus.cc:2749] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:11.053709 11677 raft_consensus.cc:493] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.054234 11677 raft_consensus.cc:3060] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.054678 11650 raft_consensus.cc:2749] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:11.055006 11644 consensus_queue.cc:260] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.056871 11645 tablet_bootstrap.cc:492] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1: No bootstrap required, opened a new log
I20260709 12:26:11.057526 11645 ts_tablet_manager.cc:1403] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.000s
I20260709 12:26:11.033828 11642 ts_tablet_manager.cc:1434] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.002s
I20260709 12:26:11.062463 11644 ts_tablet_manager.cc:1434] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.002s
I20260709 12:26:11.062770 11642 tablet_bootstrap.cc:492] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a: Bootstrap starting.
W20260709 12:26:11.044905 11563 leader_election.cc:343] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:11.053050 11450 raft_consensus.cc:2468] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 1.
I20260709 12:26:11.065304 11449 raft_consensus.cc:2468] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 1.
I20260709 12:26:11.066174 11525 raft_consensus.cc:2468] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 1.
I20260709 12:26:11.066694 11644 tablet_bootstrap.cc:492] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:11.067772 11563 leader_election.cc:304] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.068884 11646 raft_consensus.cc:2804] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:11.069789 11646 raft_consensus.cc:697] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:11.071848 11646 consensus_queue.cc:237] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.075420 11644 tablet_bootstrap.cc:654] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.076009 11563 leader_election.cc:304] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 4913cffad83a48c9884efb39bd85add1
I20260709 12:26:11.077706 11676 raft_consensus.cc:2804] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:11.079461 11642 tablet_bootstrap.cc:654] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.090116 11677 raft_consensus.cc:515] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.097463 11645 raft_consensus.cc:359] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.098457 11645 raft_consensus.cc:385] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.098773 11645 raft_consensus.cc:740] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.099077 11449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0e176bc87049d19ea24e510329e131" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:11.099730 11449 raft_consensus.cc:3060] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.099643 11645 consensus_queue.cc:260] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.092702 11676 raft_consensus.cc:697] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:11.102376 11676 consensus_queue.cc:237] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.103291 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0e176bc87049d19ea24e510329e131" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:11.101070 11677 leader_election.cc:290] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:11.103667 11645 ts_tablet_manager.cc:1434] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.003s
I20260709 12:26:11.104105 11525 raft_consensus.cc:3060] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.107959 11645 tablet_bootstrap.cc:492] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1: Bootstrap starting.
I20260709 12:26:11.110481 11449 raft_consensus.cc:2468] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 1.
I20260709 12:26:11.113612 11644 tablet_bootstrap.cc:492] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243: No bootstrap required, opened a new log
I20260709 12:26:11.114604 11563 leader_election.cc:304] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.115692 11677 raft_consensus.cc:2804] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:11.116518 11677 raft_consensus.cc:697] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:11.117630 11645 tablet_bootstrap.cc:654] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.117642 11677 consensus_queue.cc:237] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.118986 11348 catalog_manager.cc:5697] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:11.120805 11642 tablet_bootstrap.cc:492] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a: No bootstrap required, opened a new log
I20260709 12:26:11.122469 11642 ts_tablet_manager.cc:1403] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.003s
I20260709 12:26:11.127918 11642 raft_consensus.cc:359] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.131247 11642 raft_consensus.cc:385] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.132249 11642 raft_consensus.cc:740] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.134500 11644 ts_tablet_manager.cc:1403] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.011s
I20260709 12:26:11.135409 11642 consensus_queue.cc:260] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.127779 11525 raft_consensus.cc:2468] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 1.
I20260709 12:26:11.141758 11642 ts_tablet_manager.cc:1434] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:26:11.144193 11642 tablet_bootstrap.cc:492] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a: Bootstrap starting.
I20260709 12:26:11.148978 11644 raft_consensus.cc:359] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.150064 11644 raft_consensus.cc:385] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.150362 11644 raft_consensus.cc:740] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.151489 11644 consensus_queue.cc:260] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.153039 11645 tablet_bootstrap.cc:492] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1: No bootstrap required, opened a new log
I20260709 12:26:11.153584 11642 tablet_bootstrap.cc:654] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.154023 11645 ts_tablet_manager.cc:1403] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1: Time spent bootstrapping tablet: real 0.046s	user 0.022s	sys 0.023s
I20260709 12:26:11.158663 11645 raft_consensus.cc:359] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.159780 11645 raft_consensus.cc:385] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.160229 11645 raft_consensus.cc:740] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.161350 11642 tablet_bootstrap.cc:492] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a: No bootstrap required, opened a new log
I20260709 12:26:11.162125 11642 ts_tablet_manager.cc:1403] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 12:26:11.161368 11645 consensus_queue.cc:260] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.163187 11644 ts_tablet_manager.cc:1434] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:26:11.164520 11644 tablet_bootstrap.cc:492] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:11.164963 11348 catalog_manager.cc:5697] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:11.168952 11645 ts_tablet_manager.cc:1434] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:26:11.165829 11642 raft_consensus.cc:359] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.169879 11642 raft_consensus.cc:385] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.170420 11642 raft_consensus.cc:740] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.171281 11645 tablet_bootstrap.cc:492] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1: Bootstrap starting.
I20260709 12:26:11.171663 11642 consensus_queue.cc:260] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.173995 11644 tablet_bootstrap.cc:654] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.175024 11642 ts_tablet_manager.cc:1434] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:26:11.176443 11642 tablet_bootstrap.cc:492] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a: Bootstrap starting.
I20260709 12:26:11.177521 11348 catalog_manager.cc:5697] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:11.181916 11645 tablet_bootstrap.cc:654] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.183566 11642 tablet_bootstrap.cc:654] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.188833 11644 tablet_bootstrap.cc:492] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243: No bootstrap required, opened a new log
I20260709 12:26:11.189687 11644 ts_tablet_manager.cc:1403] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.007s
I20260709 12:26:11.193699 11644 raft_consensus.cc:359] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.194496 11644 raft_consensus.cc:385] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.194811 11644 raft_consensus.cc:740] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.196092 11645 tablet_bootstrap.cc:492] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1: No bootstrap required, opened a new log
I20260709 12:26:11.195794 11644 consensus_queue.cc:260] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.196746 11645 ts_tablet_manager.cc:1403] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.005s
I20260709 12:26:11.200356 11645 raft_consensus.cc:359] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.201571 11645 raft_consensus.cc:385] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.202375 11645 raft_consensus.cc:740] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.203545 11645 consensus_queue.cc:260] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.204927 11642 tablet_bootstrap.cc:492] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a: No bootstrap required, opened a new log
I20260709 12:26:11.201601 11644 ts_tablet_manager.cc:1434] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 12:26:11.205698 11642 ts_tablet_manager.cc:1403] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a: Time spent bootstrapping tablet: real 0.030s	user 0.022s	sys 0.006s
I20260709 12:26:11.206115 11645 ts_tablet_manager.cc:1434] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:26:11.207098 11644 tablet_bootstrap.cc:492] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:11.208046 11645 tablet_bootstrap.cc:492] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1: Bootstrap starting.
I20260709 12:26:11.209559 11642 raft_consensus.cc:359] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.210429 11642 raft_consensus.cc:385] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.210716 11642 raft_consensus.cc:740] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.210558 11650 raft_consensus.cc:493] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.211725 11642 consensus_queue.cc:260] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.211715 11650 raft_consensus.cc:515] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.214654 11642 ts_tablet_manager.cc:1434] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:26:11.215070 11645 tablet_bootstrap.cc:654] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.215857 11642 tablet_bootstrap.cc:492] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a: Bootstrap starting.
I20260709 12:26:11.216588 11449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f60a8d99194fde8fa4b969c5a35b3a" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:11.217960 11449 raft_consensus.cc:2468] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 0.
I20260709 12:26:11.220342 11489 leader_election.cc:304] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:11.221846 11674 raft_consensus.cc:2804] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:11.222443 11674 raft_consensus.cc:493] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.222855 11674 raft_consensus.cc:3060] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.225044 11642 tablet_bootstrap.cc:654] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.226387 11650 leader_election.cc:290] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:11.226858 11649 raft_consensus.cc:493] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.229704 11650 raft_consensus.cc:493] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.229493 11649 raft_consensus.cc:515] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.238418 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd" candidate_uuid: "2cc1d50178294d2aa84107ddd51b0243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a" is_pre_election: true
I20260709 12:26:11.239779 11599 raft_consensus.cc:2468] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cc1d50178294d2aa84107ddd51b0243 in term 0.
I20260709 12:26:11.242084 11416 leader_election.cc:304] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.247069 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd" candidate_uuid: "2cc1d50178294d2aa84107ddd51b0243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1" is_pre_election: true
I20260709 12:26:11.248878 11649 leader_election.cc:290] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:11.251639 11649 raft_consensus.cc:2804] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:11.253252 11649 raft_consensus.cc:493] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.252291 11650 raft_consensus.cc:515] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.255774 11525 raft_consensus.cc:2468] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cc1d50178294d2aa84107ddd51b0243 in term 0.
I20260709 12:26:11.254251 11649 raft_consensus.cc:3060] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.265636 11449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:11.274533 11676 raft_consensus.cc:493] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.275417 11676 raft_consensus.cc:515] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.287762 11676 leader_election.cc:290] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:11.290628 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1337a2bbcb024c7694ba59774bf25c17" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1" is_pre_election: true
I20260709 12:26:11.291553 11525 raft_consensus.cc:2468] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 0.
I20260709 12:26:11.292933 11563 leader_election.cc:304] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.293931 11676 raft_consensus.cc:2804] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:11.294332 11676 raft_consensus.cc:493] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.294682 11676 raft_consensus.cc:3060] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.304124 11676 raft_consensus.cc:515] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.307132 11649 raft_consensus.cc:515] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
W20260709 12:26:11.314582 11416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:38901, user_credentials={real_user=slave}} blocked reactor thread for 72671.2us
I20260709 12:26:11.315164 11449 raft_consensus.cc:2393] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4913cffad83a48c9884efb39bd85add1 in current term 1: Already voted for candidate 2cc1d50178294d2aa84107ddd51b0243 in this term.
I20260709 12:26:11.316849 11448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1337a2bbcb024c7694ba59774bf25c17" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:11.317204 11450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1337a2bbcb024c7694ba59774bf25c17" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:11.317598 11448 raft_consensus.cc:3060] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.319962 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f60a8d99194fde8fa4b969c5a35b3a" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a" is_pre_election: true
I20260709 12:26:11.321184 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd" candidate_uuid: "2cc1d50178294d2aa84107ddd51b0243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:11.321383 11599 raft_consensus.cc:2468] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 0.
I20260709 12:26:11.321931 11525 raft_consensus.cc:3060] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.326313 11642 tablet_bootstrap.cc:492] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a: No bootstrap required, opened a new log
I20260709 12:26:11.327023 11642 ts_tablet_manager.cc:1403] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a: Time spent bootstrapping tablet: real 0.111s	user 0.020s	sys 0.051s
I20260709 12:26:11.320168 11598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a" is_pre_election: true
I20260709 12:26:11.328624 11598 raft_consensus.cc:2468] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 0.
I20260709 12:26:11.329260 11676 leader_election.cc:290] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:11.330633 11524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1337a2bbcb024c7694ba59774bf25c17" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:11.331408 11524 raft_consensus.cc:3060] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.332136 11644 tablet_bootstrap.cc:654] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.311797 11650 leader_election.cc:290] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:11.336891 11695 raft_consensus.cc:493] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.338446 11695 raft_consensus.cc:515] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.339787 11491 leader_election.cc:304] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 2cc1d50178294d2aa84107ddd51b0243
I20260709 12:26:11.346252 11695 leader_election.cc:290] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:11.347586 11674 raft_consensus.cc:515] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.350688 11674 leader_election.cc:290] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:11.349459 11450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:11.352626 11450 raft_consensus.cc:2393] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4913cffad83a48c9884efb39bd85add1 in current term 1: Already voted for candidate 7e09e4b3d55044cd82210564ac064e9a in this term.
I20260709 12:26:11.352439 11449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f60a8d99194fde8fa4b969c5a35b3a" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:11.354818 11449 raft_consensus.cc:3060] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.355314 11642 raft_consensus.cc:359] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.356729 11642 raft_consensus.cc:385] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.357046 11642 raft_consensus.cc:740] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.358273 11642 consensus_queue.cc:260] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.360277 11598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a" is_pre_election: true
I20260709 12:26:11.360642 11642 ts_tablet_manager.cc:1434] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:26:11.361685 11525 raft_consensus.cc:2468] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cc1d50178294d2aa84107ddd51b0243 in term 1.
I20260709 12:26:11.363121 11650 raft_consensus.cc:2764] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:26:11.364238 11414 leader_election.cc:304] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:11.365654 11697 raft_consensus.cc:2804] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:11.368613 11645 tablet_bootstrap.cc:492] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1: No bootstrap required, opened a new log
I20260709 12:26:11.369015 11524 raft_consensus.cc:2468] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 1.
I20260709 12:26:11.369304 11645 ts_tablet_manager.cc:1403] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1: Time spent bootstrapping tablet: real 0.161s	user 0.053s	sys 0.079s
I20260709 12:26:11.373294 11644 tablet_bootstrap.cc:492] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243: No bootstrap required, opened a new log
I20260709 12:26:11.373349 11645 raft_consensus.cc:359] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.373932 11563 leader_election.cc:304] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.374065 11644 ts_tablet_manager.cc:1403] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.167s	user 0.017s	sys 0.012s
I20260709 12:26:11.374471 11645 raft_consensus.cc:385] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.375272 11645 raft_consensus.cc:740] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.376081 11645 consensus_queue.cc:260] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.376693 11491 leader_election.cc:304] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1; no voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a
I20260709 12:26:11.378008 11449 raft_consensus.cc:2468] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 1.
I20260709 12:26:11.378448 11644 raft_consensus.cc:359] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.378988 11645 ts_tablet_manager.cc:1434] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:26:11.379210 11644 raft_consensus.cc:385] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.379542 11489 leader_election.cc:304] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:11.380313 11645 tablet_bootstrap.cc:492] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1: Bootstrap starting.
I20260709 12:26:11.380553 11598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f60a8d99194fde8fa4b969c5a35b3a" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:11.380791 11448 raft_consensus.cc:2468] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 1.
I20260709 12:26:11.381270 11598 raft_consensus.cc:3060] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.381868 11650 raft_consensus.cc:2804] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:11.382457 11649 leader_election.cc:290] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [CANDIDATE]: Term 1 election: Requested vote from peers 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:11.382529 11650 raft_consensus.cc:697] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:11.379624 11644 raft_consensus.cc:740] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.386309 11650 consensus_queue.cc:237] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.389755 11695 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.391578 11677 raft_consensus.cc:2804] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:11.392514 11677 raft_consensus.cc:697] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:11.394953 11677 consensus_queue.cc:237] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.412029 11644 consensus_queue.cc:260] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.415515 11644 ts_tablet_manager.cc:1434] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.012s
I20260709 12:26:11.417174 11644 tablet_bootstrap.cc:492] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:11.418872 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd" candidate_uuid: "2cc1d50178294d2aa84107ddd51b0243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:11.420454 11599 raft_consensus.cc:3060] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.428822 11645 tablet_bootstrap.cc:654] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.431056 11697 raft_consensus.cc:697] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 1 LEADER]: Becoming Leader. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Running, Role: LEADER
I20260709 12:26:11.432154 11697 consensus_queue.cc:237] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.434407 11674 consensus_queue.cc:1048] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:11.434773 11348 catalog_manager.cc:5697] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:11.421945 11491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:38901, user_credentials={real_user=slave}} blocked reactor thread for 45338.4us
I20260709 12:26:11.425341 11676 raft_consensus.cc:493] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.443536 11676 raft_consensus.cc:515] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.446245 11599 raft_consensus.cc:2468] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cc1d50178294d2aa84107ddd51b0243 in term 1.
I20260709 12:26:11.445389 11598 raft_consensus.cc:2468] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 1.
I20260709 12:26:11.451473 11695 raft_consensus.cc:2749] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:11.459112 11677 raft_consensus.cc:493] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.459724 11448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:11.459954 11677 raft_consensus.cc:515] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.454365 11676 leader_election.cc:290] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
W20260709 12:26:11.465704 11563 leader_election.cc:343] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:11.469148 11524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1" is_pre_election: true
W20260709 12:26:11.471396 11563 leader_election.cc:343] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:11.471426 11448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:11.472083 11563 leader_election.cc:304] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09e4b3d55044cd82210564ac064e9a; no voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1
I20260709 12:26:11.474252 11677 leader_election.cc:290] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:11.478722 11677 raft_consensus.cc:2749] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:11.478405 11695 consensus_queue.cc:1048] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:26:11.485037 11644 tablet_bootstrap.cc:654] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.489639 11524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1" is_pre_election: true
I20260709 12:26:11.490469 11524 raft_consensus.cc:2468] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 0.
I20260709 12:26:11.496790 11348 catalog_manager.cc:5697] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "2cc1d50178294d2aa84107ddd51b0243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:11.496170 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:26:11.493323 11563 leader_election.cc:343] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:11.520046 11645 tablet_bootstrap.cc:492] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1: No bootstrap required, opened a new log
I20260709 12:26:11.520839 11645 ts_tablet_manager.cc:1403] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1: Time spent bootstrapping tablet: real 0.141s	user 0.021s	sys 0.012s
I20260709 12:26:11.524797 11645 raft_consensus.cc:359] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.525632 11645 raft_consensus.cc:385] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.525921 11645 raft_consensus.cc:740] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.526924 11645 consensus_queue.cc:260] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.533576 11645 ts_tablet_manager.cc:1434] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:26:11.535492 11563 leader_election.cc:304] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 2cc1d50178294d2aa84107ddd51b0243
I20260709 12:26:11.542273 11644 tablet_bootstrap.cc:492] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243: No bootstrap required, opened a new log
I20260709 12:26:11.543342 11644 ts_tablet_manager.cc:1403] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.126s	user 0.013s	sys 0.001s
I20260709 12:26:11.541220 11677 raft_consensus.cc:2804] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:11.544661 11677 raft_consensus.cc:493] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.545027 11677 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.545921 11646 raft_consensus.cc:493] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.546602 11646 raft_consensus.cc:515] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.549719 11646 leader_election.cc:290] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:11.541312 11676 consensus_queue.cc:1048] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:26:11.550496 11646 consensus_queue.cc:1048] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:11.542964 11683 consensus_queue.cc:1048] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:11.553974 11644 raft_consensus.cc:359] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.555896 11644 raft_consensus.cc:385] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.556447 11644 raft_consensus.cc:740] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.561177 11644 consensus_queue.cc:260] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.574190 11677 raft_consensus.cc:515] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.577524 11644 ts_tablet_manager.cc:1434] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:26:11.578218 11347 catalog_manager.cc:5697] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:11.585803 11449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:11.586918 11449 raft_consensus.cc:2468] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 0.
I20260709 12:26:11.589861 11524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1" is_pre_election: true
I20260709 12:26:11.591116 11524 raft_consensus.cc:2468] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 0.
I20260709 12:26:11.593225 11644 tablet_bootstrap.cc:492] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:11.597513 11646 consensus_queue.cc:1048] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:11.600730 11644 tablet_bootstrap.cc:654] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:11.600297 11563 leader_election.cc:304] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.603415 11676 raft_consensus.cc:2804] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:11.604212 11676 raft_consensus.cc:493] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.605280 11677 leader_election.cc:290] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:11.605655 11676 raft_consensus.cc:3060] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.606216 11677 consensus_queue.cc:1048] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:11.616101 11699 raft_consensus.cc:493] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.617573 11699 raft_consensus.cc:515] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.621527 11523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "2cc1d50178294d2aa84107ddd51b0243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1" is_pre_election: true
I20260709 12:26:11.622327 11523 raft_consensus.cc:2468] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cc1d50178294d2aa84107ddd51b0243 in term 0.
I20260709 12:26:11.623836 11414 leader_election.cc:304] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:11.626040 11697 raft_consensus.cc:2804] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:11.626441 11697 raft_consensus.cc:493] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.626789 11697 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.631525 11450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:11.632887 11522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:11.634089 11522 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.640581 11699 leader_election.cc:290] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:11.636605 11676 raft_consensus.cc:515] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.645313 11676 leader_election.cc:290] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:11.649179 11522 raft_consensus.cc:2468] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 1.
I20260709 12:26:11.649998 11697 raft_consensus.cc:515] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.651638 11450 raft_consensus.cc:2393] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09e4b3d55044cd82210564ac064e9a in current term 1: Already voted for candidate 2cc1d50178294d2aa84107ddd51b0243 in this term.
I20260709 12:26:11.656179 11598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "2cc1d50178294d2aa84107ddd51b0243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a" is_pre_election: true
I20260709 12:26:11.656904 11697 leader_election.cc:290] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [CANDIDATE]: Term 1 election: Requested vote from peers 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:11.657723 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:11.658560 11525 raft_consensus.cc:3060] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.662137 11450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:11.663275 11450 raft_consensus.cc:3060] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.664871 11644 tablet_bootstrap.cc:492] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243: No bootstrap required, opened a new log
I20260709 12:26:11.665652 11644 ts_tablet_manager.cc:1403] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.073s	user 0.029s	sys 0.021s
I20260709 12:26:11.669140 11644 raft_consensus.cc:359] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.669988 11644 raft_consensus.cc:385] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:11.670279 11644 raft_consensus.cc:740] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:11.657074 11598 raft_consensus.cc:2393] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2cc1d50178294d2aa84107ddd51b0243 in current term 1: Already voted for candidate 7e09e4b3d55044cd82210564ac064e9a in this term.
I20260709 12:26:11.671121 11644 consensus_queue.cc:260] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.672495 11522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "2cc1d50178294d2aa84107ddd51b0243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:11.673385 11522 raft_consensus.cc:2393] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2cc1d50178294d2aa84107ddd51b0243 in current term 1: Already voted for candidate 7e09e4b3d55044cd82210564ac064e9a in this term.
I20260709 12:26:11.673645 11644 ts_tablet_manager.cc:1434] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:26:11.662237 11563 leader_election.cc:304] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.675438 11677 raft_consensus.cc:2804] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:11.675958 11677 raft_consensus.cc:697] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:11.676863 11677 consensus_queue.cc:237] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.677816 11674 raft_consensus.cc:493] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.678500 11674 raft_consensus.cc:515] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.677871 11676 consensus_queue.cc:1048] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:26:11.682401 11598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a" is_pre_election: true
I20260709 12:26:11.683018 11674 leader_election.cc:290] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:11.683235 11598 raft_consensus.cc:2468] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 0.
I20260709 12:26:11.684788 11491 leader_election.cc:304] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.687215 11598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "2cc1d50178294d2aa84107ddd51b0243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:11.685952 11650 raft_consensus.cc:2804] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:11.690640 11448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243" is_pre_election: true
I20260709 12:26:11.691444 11448 raft_consensus.cc:2468] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 0.
I20260709 12:26:11.696024 11650 raft_consensus.cc:493] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.696533 11650 raft_consensus.cc:3060] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.706382 11416 leader_election.cc:304] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243; no voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a
I20260709 12:26:11.708084 11699 raft_consensus.cc:2749] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:26:11.717751 11450 raft_consensus.cc:2468] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 1.
I20260709 12:26:11.722172 11563 leader_election.cc:304] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.724056 11683 raft_consensus.cc:2804] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:11.724773 11683 raft_consensus.cc:697] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:11.725888 11683 consensus_queue.cc:237] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.730527 11525 raft_consensus.cc:2468] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 1.
I20260709 12:26:11.757975 11650 raft_consensus.cc:515] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.748282 11348 catalog_manager.cc:5697] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:11.764228 11449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:11.765590 11449 raft_consensus.cc:3060] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.772990 11650 leader_election.cc:290] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:11.775105 11598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:11.775976 11598 raft_consensus.cc:3060] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:11.787348 11598 raft_consensus.cc:2468] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 1.
I20260709 12:26:11.788925 11491 leader_election.cc:304] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:11.790114 11650 raft_consensus.cc:2804] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:11.790220 11449 raft_consensus.cc:2468] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 1.
I20260709 12:26:11.790855 11650 raft_consensus.cc:697] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:11.791873 11650 consensus_queue.cc:237] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:11.805171 11347 catalog_manager.cc:5697] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 1 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:11.806380 11348 catalog_manager.cc:5697] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:11.823603 11697 consensus_queue.cc:1048] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:11.839155 11697 consensus_queue.cc:1048] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:11.937040  1517 ts_manager.cc:295] Set tserver state for 2cc1d50178294d2aa84107ddd51b0243 to MAINTENANCE_MODE
I20260709 12:26:11.938135  1517 tablet_server.cc:179] TabletServer@127.1.123.65:0 shutting down...
I20260709 12:26:11.941663 11650 consensus_queue.cc:1048] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:26:11.961680 11489 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 147 similar messages]
I20260709 12:26:11.962479 11646 consensus_queue.cc:1048] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:26:11.967726 11489 consensus_peers.cc:597] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:12.007079 11563 consensus_peers.cc:597] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:12.028028  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:12.029469  1517 tablet_replica.cc:333] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243: stopping tablet replica
I20260709 12:26:12.031563  1517 raft_consensus.cc:2243] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:12.032109  1517 raft_consensus.cc:2272] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:12.036038  1517 tablet_replica.cc:333] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243: stopping tablet replica
I20260709 12:26:12.036806  1517 raft_consensus.cc:2243] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:12.037428  1517 raft_consensus.cc:2272] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:12.040259  1517 tablet_replica.cc:333] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243: stopping tablet replica
I20260709 12:26:12.040957  1517 raft_consensus.cc:2243] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:12.041559  1517 raft_consensus.cc:2272] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:12.043766  1517 tablet_replica.cc:333] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243: stopping tablet replica
I20260709 12:26:12.044345  1517 raft_consensus.cc:2243] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:12.044749  1517 raft_consensus.cc:2272] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:12.047012  1517 tablet_replica.cc:333] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243: stopping tablet replica
I20260709 12:26:12.047933  1517 raft_consensus.cc:2243] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:12.048424  1517 raft_consensus.cc:2272] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:12.051110  1517 tablet_replica.cc:333] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243: stopping tablet replica
I20260709 12:26:12.052632  1517 raft_consensus.cc:2243] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:12.054328  1517 raft_consensus.cc:2272] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:12.058501  1517 tablet_replica.cc:333] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243: stopping tablet replica
I20260709 12:26:12.059576  1517 raft_consensus.cc:2243] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:12.060279  1517 raft_consensus.cc:2272] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:12.064886  1517 tablet_replica.cc:333] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243: stopping tablet replica
I20260709 12:26:12.065980  1517 raft_consensus.cc:2243] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:12.066716  1517 raft_consensus.cc:2272] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:12.071692  1517 tablet_replica.cc:333] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243: stopping tablet replica
I20260709 12:26:12.072422  1517 raft_consensus.cc:2243] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:12.072956  1517 raft_consensus.cc:2272] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:12.077212  1517 tablet_replica.cc:333] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243: stopping tablet replica
I20260709 12:26:12.078171  1517 raft_consensus.cc:2243] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:12.079001  1517 raft_consensus.cc:2272] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:12.118654  1517 tablet_server.cc:196] TabletServer@127.1.123.65:0 shutdown complete.
W20260709 12:26:12.118686 11563 consensus_peers.cc:597] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:12.159211 11722 consensus_queue.cc:1048] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:26:12.176247 11563 consensus_peers.cc:597] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:12.191519 11683 consensus_queue.cc:1048] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:26:12.208562 11563 consensus_peers.cc:597] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:12.238902 11489 consensus_peers.cc:597] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:12.279228 11563 consensus_peers.cc:597] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:12.324715 11489 consensus_peers.cc:597] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:12.383628 11695 consensus_queue.cc:1048] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:26:12.410789 11563 consensus_peers.cc:597] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:12.505784 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:26:13.339900 11683 raft_consensus.cc:493] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2cc1d50178294d2aa84107ddd51b0243)
I20260709 12:26:13.340632 11683 raft_consensus.cc:515] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:13.342625 11683 leader_election.cc:290] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:13.344800 11650 raft_consensus.cc:493] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2cc1d50178294d2aa84107ddd51b0243)
I20260709 12:26:13.345010 11522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1" is_pre_election: true
I20260709 12:26:13.345588 11650 raft_consensus.cc:515] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:13.346801 11522 raft_consensus.cc:2468] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 1.
W20260709 12:26:13.347940 11563 leader_election.cc:336] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111)
I20260709 12:26:13.348999 11650 leader_election.cc:290] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:13.348989 11563 leader_election.cc:304] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 2cc1d50178294d2aa84107ddd51b0243
I20260709 12:26:13.350172 11683 raft_consensus.cc:2804] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:26:13.350610 11683 raft_consensus.cc:493] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2cc1d50178294d2aa84107ddd51b0243)
I20260709 12:26:13.350494 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09e4b3d55044cd82210564ac064e9a" is_pre_election: true
I20260709 12:26:13.351486 11683 raft_consensus.cc:3060] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:26:13.353816 11489 leader_election.cc:336] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111)
I20260709 12:26:13.360531 11683 raft_consensus.cc:515] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:13.362277 11599 raft_consensus.cc:2393] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4913cffad83a48c9884efb39bd85add1 in current term 2: Already voted for candidate 7e09e4b3d55044cd82210564ac064e9a in this term.
I20260709 12:26:13.362742 11683 leader_election.cc:290] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 2 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:13.364868 11522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:13.365828 11522 raft_consensus.cc:3060] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.365969 11491 leader_election.cc:304] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1; no voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a
W20260709 12:26:13.368897 11563 leader_election.cc:336] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111)
I20260709 12:26:13.377147 11522 raft_consensus.cc:2468] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 2.
I20260709 12:26:13.377995 11650 raft_consensus.cc:2749] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:26:13.380220 11563 leader_election.cc:304] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 2cc1d50178294d2aa84107ddd51b0243
I20260709 12:26:13.382429 11683 raft_consensus.cc:2804] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:13.383646 11683 raft_consensus.cc:697] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:13.385392 11683 consensus_queue.cc:237] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:13.397046 11347 catalog_manager.cc:5697] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 1 to 2, leader changed from 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65) to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 2 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:13.507509 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:26:13.995914 11563 consensus_peers.cc:597] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:14.018438 11525 raft_consensus.cc:1275] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:14.021440 11683 consensus_queue.cc:1048] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:26:14.387291 11563 consensus_peers.cc:597] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:14.454020 11489 consensus_peers.cc:597] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:14.463069 11563 consensus_peers.cc:597] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:14.508823 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:26:14.510475 11563 consensus_peers.cc:597] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:14.668949 11563 consensus_peers.cc:597] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:14.887722 11489 consensus_peers.cc:597] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:14.925680 11563 consensus_peers.cc:597] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:14.940595 11489 consensus_peers.cc:597] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:14.948956 11563 consensus_peers.cc:597] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:15.510080 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:26:16.405130 11563 consensus_peers.cc:597] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:16.511646 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:26:16.820418 11563 consensus_peers.cc:597] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:16.923596 11563 consensus_peers.cc:597] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:16.948889 11489 consensus_peers.cc:597] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:16.955474 11563 consensus_peers.cc:597] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:17.049747 11563 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111) [suppressed 100 similar messages]
W20260709 12:26:17.053174 11563 consensus_peers.cc:597] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:17.148344  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:17.160068 11746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:17.161744 11747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:17.163453 11749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:17.167235  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:17.171469  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:17.171880  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:17.172096  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599977172070 us; error 0 us; skew 500 ppm
I20260709 12:26:17.172752  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:17.177088  1517 webserver.cc:533] Webserver started at http://127.1.123.65:44853/ using document root <none> and password file <none>
I20260709 12:26:17.177923  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:17.178176  1517 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:26:17.182855  1517 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:26:17.189198 11754 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:17.190593  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:26:17.191180  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-0-root
uuid: "2cc1d50178294d2aa84107ddd51b0243"
format_stamp: "Formatted at 2026-07-09 12:26:09 on dist-test-slave-08l4"
I20260709 12:26:17.191530  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599626433458-1517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:17.240876  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:17.242532  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:17.245112  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:17.252554 11761 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 12:26:17.322763 11489 consensus_peers.cc:597] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:17.368800  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:26:17.369642  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.119s	user 0.005s	sys 0.000s
I20260709 12:26:17.370069  1517 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:26:17.377646 11761 tablet_bootstrap.cc:492] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:17.397403 11761 tablet_bootstrap.cc:492] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:17.398422 11761 tablet_bootstrap.cc:492] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap complete.
I20260709 12:26:17.399284 11761 ts_tablet_manager.cc:1403] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 12:26:17.402972 11761 raft_consensus.cc:359] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.404279 11761 raft_consensus.cc:740] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:17.405650 11761 consensus_queue.cc:260] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.409654 11761 ts_tablet_manager.cc:1434] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.010s	user 0.012s	sys 0.000s
I20260709 12:26:17.411182 11761 tablet_bootstrap.cc:492] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
W20260709 12:26:17.510737 11563 consensus_peers.cc:597] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:17.515174 11761 tablet_bootstrap.cc:492] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:26:17.515065 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:26:17.517974 11761 tablet_bootstrap.cc:492] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap complete.
I20260709 12:26:17.520330 11761 ts_tablet_manager.cc:1403] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.110s	user 0.031s	sys 0.023s
I20260709 12:26:17.526410 11761 raft_consensus.cc:359] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.527952 11761 raft_consensus.cc:740] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:17.530360 11761 consensus_queue.cc:260] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.532974 11761 ts_tablet_manager.cc:1434] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.008s
I20260709 12:26:17.534451 11761 tablet_bootstrap.cc:492] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:17.539640  1517 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:26:17.541162  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.171s	user 0.086s	sys 0.015s
W20260709 12:26:17.543239 11563 consensus_peers.cc:597] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:17.565229 11761 tablet_bootstrap.cc:492] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:17.570019 11761 tablet_bootstrap.cc:492] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap complete.
I20260709 12:26:17.572791 11761 ts_tablet_manager.cc:1403] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.012s
W20260709 12:26:17.586817 11489 consensus_peers.cc:597] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:17.587930 11761 raft_consensus.cc:359] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.589581 11761 raft_consensus.cc:740] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:17.590888 11761 consensus_queue.cc:260] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.594184 11761 ts_tablet_manager.cc:1434] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.021s	user 0.011s	sys 0.002s
I20260709 12:26:17.596968 11761 tablet_bootstrap.cc:492] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:17.617499 11761 tablet_bootstrap.cc:492] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:17.619345 11761 tablet_bootstrap.cc:492] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap complete.
I20260709 12:26:17.622301 11761 ts_tablet_manager.cc:1403] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.007s
I20260709 12:26:17.626940 11761 raft_consensus.cc:359] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.628338 11761 raft_consensus.cc:740] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:17.629561 11761 consensus_queue.cc:260] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.633144 11761 ts_tablet_manager.cc:1434] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:26:17.634822 11761 tablet_bootstrap.cc:492] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:17.658811 11761 tablet_bootstrap.cc:492] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:17.660341 11761 tablet_bootstrap.cc:492] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap complete.
I20260709 12:26:17.661594 11761 ts_tablet_manager.cc:1403] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.028s	user 0.024s	sys 0.000s
I20260709 12:26:17.666285 11761 raft_consensus.cc:359] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.667346 11761 raft_consensus.cc:740] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:17.668469 11761 consensus_queue.cc:260] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.671229 11761 ts_tablet_manager.cc:1434] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:26:17.672430 11761 tablet_bootstrap.cc:492] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:17.698082 11761 tablet_bootstrap.cc:492] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:17.699051 11761 tablet_bootstrap.cc:492] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap complete.
I20260709 12:26:17.699712 11761 ts_tablet_manager.cc:1403] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.004s
I20260709 12:26:17.703866  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.65:34807
I20260709 12:26:17.704296 11828 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.65:34807 every 8 connection(s)
I20260709 12:26:17.704463 11761 raft_consensus.cc:359] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.705523 11761 raft_consensus.cc:740] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:17.706462 11761 consensus_queue.cc:260] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.708124 11761 ts_tablet_manager.cc:1434] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:26:17.708899 11761 tablet_bootstrap.cc:492] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:17.723701  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:17.751432 11761 tablet_bootstrap.cc:492] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:17.759225 11761 tablet_bootstrap.cc:492] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap complete.
I20260709 12:26:17.760109 11761 ts_tablet_manager.cc:1403] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.051s	user 0.021s	sys 0.013s
I20260709 12:26:17.764484 11829 heartbeater.cc:344] Connected to a master server at 127.1.123.126:42475
I20260709 12:26:17.765851 11761 raft_consensus.cc:359] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.766733 11829 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:17.767140 11761 raft_consensus.cc:740] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:17.768878 11761 consensus_queue.cc:260] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.769253 11829 heartbeater.cc:507] Master 127.1.123.126:42475 requested a full tablet report, sending...
I20260709 12:26:17.774225 11761 ts_tablet_manager.cc:1434] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.004s
I20260709 12:26:17.777693 11761 tablet_bootstrap.cc:492] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:17.788123 11347 ts_manager.cc:194] Re-registered known tserver with Master: 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807)
I20260709 12:26:17.804957 11599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
mode: GRACEFUL
new_leader_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:53838
I20260709 12:26:17.806819 11599 raft_consensus.cc:606] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Received request to transfer leadership to TS 4913cffad83a48c9884efb39bd85add1
I20260709 12:26:17.806118 11761 tablet_bootstrap.cc:492] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:17.808485 11761 tablet_bootstrap.cc:492] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap complete.
I20260709 12:26:17.810369 11761 ts_tablet_manager.cc:1403] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.033s	user 0.020s	sys 0.004s
I20260709 12:26:17.815544 11599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
mode: GRACEFUL
new_leader_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:53838
I20260709 12:26:17.815496 11761 raft_consensus.cc:359] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.816740 11761 raft_consensus.cc:740] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:17.816125 11347 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46400
I20260709 12:26:17.817435 11599 raft_consensus.cc:606] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Received request to transfer leadership to TS 4913cffad83a48c9884efb39bd85add1
I20260709 12:26:17.817943 11802 raft_consensus.cc:3060] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:17.817967 11761 consensus_queue.cc:260] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.821943 11599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
mode: GRACEFUL
new_leader_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:53838
I20260709 12:26:17.822947 11599 raft_consensus.cc:606] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Received request to transfer leadership to TS 4913cffad83a48c9884efb39bd85add1
I20260709 12:26:17.822999 11829 heartbeater.cc:499] Master 127.1.123.126:42475 was elected leader, sending a full tablet report...
I20260709 12:26:17.824281 11761 ts_tablet_manager.cc:1434] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:26:17.825886 11761 tablet_bootstrap.cc:492] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:17.826499  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:26:17.827586  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:17.828641  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:17.855317 11761 tablet_bootstrap.cc:492] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:17.857975 11761 tablet_bootstrap.cc:492] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap complete.
I20260709 12:26:17.859606 11761 ts_tablet_manager.cc:1403] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.034s	user 0.027s	sys 0.000s
I20260709 12:26:17.863729 11761 raft_consensus.cc:359] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.864971 11761 raft_consensus.cc:740] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:17.866285 11761 consensus_queue.cc:260] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:17.868896 11761 ts_tablet_manager.cc:1434] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:26:17.870800 11761 tablet_bootstrap.cc:492] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap starting.
I20260709 12:26:17.893014 11761 tablet_bootstrap.cc:492] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:17.894389 11761 tablet_bootstrap.cc:492] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243: Bootstrap complete.
I20260709 12:26:17.895603 11761 ts_tablet_manager.cc:1403] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243: Time spent bootstrapping tablet: real 0.025s	user 0.022s	sys 0.000s
I20260709 12:26:17.898456 11761 raft_consensus.cc:359] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:17.899181 11761 raft_consensus.cc:740] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d50178294d2aa84107ddd51b0243, State: Initialized, Role: FOLLOWER
I20260709 12:26:17.900935 11761 consensus_queue.cc:260] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } }
I20260709 12:26:17.905108 11761 ts_tablet_manager.cc:1434] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:26:18.047103 11767 raft_consensus.cc:993] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a: : Instructing follower 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:18.047959 11683 raft_consensus.cc:1081] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Signalling peer 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:18.050693 11525 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:26:18.051465 11525 raft_consensus.cc:493] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:18.051784 11525 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:18.062542 11525 raft_consensus.cc:515] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:18.066087 11525 leader_election.cc:290] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 2 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:18.066850 11802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:18.068252 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:18.068413 11802 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:18.069068 11599 raft_consensus.cc:3055] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:18.070048 11599 raft_consensus.cc:740] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:18.071679 11599 consensus_queue.cc:260] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:18.076956 11599 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:18.080914 11802 raft_consensus.cc:2468] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 2.
I20260709 12:26:18.083503 11489 leader_election.cc:304] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:18.086391 11771 raft_consensus.cc:2804] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:18.087998 11771 raft_consensus.cc:697] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:18.093473 11599 raft_consensus.cc:2468] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 2.
I20260709 12:26:18.098162 11771 consensus_queue.cc:237] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:18.111294 11767 raft_consensus.cc:993] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a: : Instructing follower 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:18.112044 11767 raft_consensus.cc:1081] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Signalling peer 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:18.118644 11522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:26:18.119827 11522 raft_consensus.cc:493] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:18.120386 11522 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:18.123456 11347 catalog_manager.cc:5697] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 1 to 2, leader changed from 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67) to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:18.143611 11522 raft_consensus.cc:515] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:18.146471 11522 leader_election.cc:290] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 2 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:18.148150 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:18.149219 11801 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:18.148897 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:18.150035 11599 raft_consensus.cc:3055] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:18.150451 11599 raft_consensus.cc:740] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:18.152444 11599 consensus_queue.cc:260] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:18.155762 11599 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:18.161924 11801 raft_consensus.cc:2468] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 2.
I20260709 12:26:18.163921 11489 leader_election.cc:304] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:18.165237 11771 raft_consensus.cc:2804] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:18.165740 11599 raft_consensus.cc:2468] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 2.
I20260709 12:26:18.166409 11771 raft_consensus.cc:697] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:18.168569 11771 consensus_queue.cc:237] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:18.179883 11347 catalog_manager.cc:5697] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 1 to 2, leader changed from 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67) to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:18.220455 11767 raft_consensus.cc:993] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a: : Instructing follower 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:18.221607 11683 raft_consensus.cc:1081] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Signalling peer 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:18.224941 11522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:26:18.225896 11522 raft_consensus.cc:493] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:18.227255 11522 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:18.235806 11522 raft_consensus.cc:515] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:18.242152 11522 leader_election.cc:290] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 2 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:18.244058 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:18.245057 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:18.245316 11599 raft_consensus.cc:3055] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:18.245870 11599 raft_consensus.cc:740] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:18.246021 11801 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:18.247291 11599 consensus_queue.cc:260] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:18.250087 11599 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:18.258098 11801 raft_consensus.cc:2468] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 2.
I20260709 12:26:18.260910 11599 raft_consensus.cc:2468] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 2.
I20260709 12:26:18.261801 11489 leader_election.cc:304] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:18.262897 11771 raft_consensus.cc:2804] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:18.263410 11771 raft_consensus.cc:697] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:18.265139 11771 consensus_queue.cc:237] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:18.277941 11347 catalog_manager.cc:5697] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 1 to 2, leader changed from 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67) to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:18.517892 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:26:18.520629 11801 raft_consensus.cc:1275] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:18.522825 11857 consensus_queue.cc:1048] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:18.537341 11599 raft_consensus.cc:1275] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:18.540992 11771 consensus_queue.cc:1048] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:18.694705 11598 raft_consensus.cc:1275] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:18.698262 11801 raft_consensus.cc:1275] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:18.698202 11857 consensus_queue.cc:1048] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:18.700851 11857 consensus_queue.cc:1048] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:18.735948 11801 raft_consensus.cc:1275] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:18.738158 11771 consensus_queue.cc:1048] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:18.755097 11599 raft_consensus.cc:1275] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:18.760637 11674 consensus_queue.cc:1048] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:18.843364  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:18.868834 11522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
mode: GRACEFUL
new_leader_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:47218
I20260709 12:26:18.869495 11522 raft_consensus.cc:606] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Received request to transfer leadership to TS 7e09e4b3d55044cd82210564ac064e9a
I20260709 12:26:18.873910 11522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
mode: GRACEFUL
new_leader_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:47218
I20260709 12:26:18.874822 11522 raft_consensus.cc:606] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Received request to transfer leadership to TS 7e09e4b3d55044cd82210564ac064e9a
I20260709 12:26:18.879233 11598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
mode: GRACEFUL
new_leader_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:53838
I20260709 12:26:18.879971 11598 raft_consensus.cc:606] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Received request to transfer leadership to TS 4913cffad83a48c9884efb39bd85add1
I20260709 12:26:18.882076  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:26:18.882884  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:18.883790  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:19.016345 11849 raft_consensus.cc:993] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a: : Instructing follower 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:19.017355 11854 raft_consensus.cc:1081] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Signalling peer 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:19.019661 11522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:26:19.020251 11522 raft_consensus.cc:493] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:19.020543 11522 raft_consensus.cc:3060] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:19.029282 11522 raft_consensus.cc:515] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:19.031432 11522 leader_election.cc:290] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 3 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:19.032706 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:19.032825 11598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5750167a0c848b2afc5b03bda200dcd" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:19.033833 11598 raft_consensus.cc:3055] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:19.034319 11598 raft_consensus.cc:740] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:19.035594 11598 consensus_queue.cc:260] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:19.039027 11598 raft_consensus.cc:3060] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:19.033828 11801 raft_consensus.cc:3060] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:19.048978 11771 raft_consensus.cc:993] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1: : Instructing follower 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:19.049978 11867 raft_consensus.cc:1081] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Signalling peer 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:19.052559 11599 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:53822
I20260709 12:26:19.053330 11598 raft_consensus.cc:2468] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 3.
I20260709 12:26:19.053597 11599 raft_consensus.cc:493] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:19.054575 11599 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:19.055624 11491 leader_election.cc:304] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:19.055965 11801 raft_consensus.cc:2468] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 3.
I20260709 12:26:19.059391 11867 raft_consensus.cc:2804] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:19.060141 11867 raft_consensus.cc:697] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 3 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:19.061249 11867 consensus_queue.cc:237] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:19.064986 11599 raft_consensus.cc:515] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:19.068285 11599 leader_election.cc:290] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 3 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:19.073077 11522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:19.073874 11522 raft_consensus.cc:3055] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:19.073521 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:19.075364 11801 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:19.075076 11522 raft_consensus.cc:740] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:19.075002 11344 catalog_manager.cc:5697] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 2 to 3, leader changed from 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67) to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:19.077406 11522 consensus_queue.cc:260] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:19.081749 11522 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:19.085495 11801 raft_consensus.cc:2468] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 3.
I20260709 12:26:19.087195 11563 leader_election.cc:304] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:19.088050 11854 raft_consensus.cc:2804] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:19.088567 11854 raft_consensus.cc:697] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:19.089468 11854 consensus_queue.cc:237] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:19.101505 11522 raft_consensus.cc:2468] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 3.
I20260709 12:26:19.101222 11343 catalog_manager.cc:5697] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 2 to 3, leader changed from 4913cffad83a48c9884efb39bd85add1 (127.1.123.66) to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:19.188395 11674 raft_consensus.cc:993] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1: : Instructing follower 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:19.189110 11674 raft_consensus.cc:1081] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Signalling peer 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:19.193125 11598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:53822
I20260709 12:26:19.194108 11598 raft_consensus.cc:493] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:19.194590 11598 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:19.201354 11598 raft_consensus.cc:515] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:19.206449 11802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:19.207926 11802 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:19.208240 11522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:19.206115 11598 leader_election.cc:290] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 3 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:19.209208 11522 raft_consensus.cc:3055] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:19.209951 11522 raft_consensus.cc:740] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:19.211772 11522 consensus_queue.cc:260] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:19.213521 11522 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:19.219175 11802 raft_consensus.cc:2468] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 3.
I20260709 12:26:19.221431 11563 leader_election.cc:304] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:19.222404 11854 raft_consensus.cc:2804] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:19.223214 11854 raft_consensus.cc:697] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:19.224082 11854 consensus_queue.cc:237] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:19.228725 11522 raft_consensus.cc:2468] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 3.
I20260709 12:26:19.233426 11343 catalog_manager.cc:5697] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 2 to 3, leader changed from 4913cffad83a48c9884efb39bd85add1 (127.1.123.66) to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:19.457620 11802 raft_consensus.cc:1275] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:19.459378 11674 consensus_queue.cc:1048] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:19.474048 11598 raft_consensus.cc:1275] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:19.476863 11867 consensus_queue.cc:1048] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:19.505110 11802 raft_consensus.cc:1275] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:19.507378 11854 consensus_queue.cc:1048] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:19.518254 11522 raft_consensus.cc:1275] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:26:19.519378 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:26:19.521282 11878 consensus_queue.cc:1048] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:19.684396 11522 raft_consensus.cc:1275] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:19.686661 11849 consensus_queue.cc:1048] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:19.696861 11802 raft_consensus.cc:1275] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:19.700798 11849 consensus_queue.cc:1048] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:19.892166  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:19.902444 11598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
mode: GRACEFUL
new_leader_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:53838
I20260709 12:26:19.903122 11598 raft_consensus.cc:606] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Received request to transfer leadership to TS 4913cffad83a48c9884efb39bd85add1
I20260709 12:26:19.906723 11522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
mode: GRACEFUL
new_leader_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:47218
I20260709 12:26:19.907307 11522 raft_consensus.cc:606] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Received request to transfer leadership to TS 7e09e4b3d55044cd82210564ac064e9a
I20260709 12:26:19.911028 11598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
mode: GRACEFUL
new_leader_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:53838
I20260709 12:26:19.911788 11598 raft_consensus.cc:606] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Received request to transfer leadership to TS 4913cffad83a48c9884efb39bd85add1
I20260709 12:26:19.913357  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:26:19.914321  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:19.915385  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:20.199646 11878 raft_consensus.cc:993] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a: : Instructing follower 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:20.200392 11878 raft_consensus.cc:1081] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Signalling peer 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:20.202728 11522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:26:20.203483 11522 raft_consensus.cc:493] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:20.204011 11522 raft_consensus.cc:3060] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:20.212512 11522 raft_consensus.cc:515] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:20.214810 11522 leader_election.cc:290] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 2 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:20.216089 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:20.216207 11598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:20.217046 11801 raft_consensus.cc:3060] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:20.217125 11598 raft_consensus.cc:3055] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:20.217754 11598 raft_consensus.cc:740] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:20.218806 11598 consensus_queue.cc:260] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:20.220311 11598 raft_consensus.cc:3060] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:20.224933 11801 raft_consensus.cc:2468] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 2.
I20260709 12:26:20.226233 11598 raft_consensus.cc:2468] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 2.
I20260709 12:26:20.227113 11489 leader_election.cc:304] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:20.227975 11771 raft_consensus.cc:2804] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:20.228417 11771 raft_consensus.cc:697] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:20.229902 11771 consensus_queue.cc:237] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:20.243129 11344 catalog_manager.cc:5697] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 1 to 2, leader changed from 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67) to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:20.249475 11849 raft_consensus.cc:993] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a: : Instructing follower 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:20.250332 11878 raft_consensus.cc:1081] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Signalling peer 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:20.252702 11522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:26:20.253749 11522 raft_consensus.cc:493] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:20.254845 11522 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:20.263724 11674 raft_consensus.cc:993] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1: : Instructing follower 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:20.264927 11674 raft_consensus.cc:1081] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Signalling peer 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:20.264861 11522 raft_consensus.cc:515] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:20.268355 11598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:53822
I20260709 12:26:20.268850 11522 leader_election.cc:290] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 4 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:20.269132 11598 raft_consensus.cc:493] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:20.269850 11598 raft_consensus.cc:3060] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:20.269994 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:20.270751 11801 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:20.272487 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d2f6a8f3a7462c90f0047ae8cdf4fa" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:20.274135 11599 raft_consensus.cc:3055] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:20.274605 11599 raft_consensus.cc:740] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:20.275506 11599 consensus_queue.cc:260] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:20.276599 11599 raft_consensus.cc:3060] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:20.280911 11801 raft_consensus.cc:2468] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 4.
I20260709 12:26:20.282747 11489 leader_election.cc:304] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:20.282428 11598 raft_consensus.cc:515] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:20.283828 11674 raft_consensus.cc:2804] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:20.284381 11674 raft_consensus.cc:697] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 4 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:20.285303 11674 consensus_queue.cc:237] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:20.290390 11599 raft_consensus.cc:2468] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 4.
I20260709 12:26:20.291850 11598 leader_election.cc:290] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 2 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:20.292069 11522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:20.292778 11522 raft_consensus.cc:3055] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:20.293112 11522 raft_consensus.cc:740] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:20.294524 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:20.294724 11522 consensus_queue.cc:260] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:20.295572 11801 raft_consensus.cc:3060] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:20.296689 11522 raft_consensus.cc:3060] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:20.301129 11344 catalog_manager.cc:5697] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 3 to 4, leader changed from 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67) to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 4 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:20.307070 11801 raft_consensus.cc:2468] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 2.
I20260709 12:26:20.307147 11522 raft_consensus.cc:2468] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 2.
I20260709 12:26:20.308929 11563 leader_election.cc:304] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:20.309861 11878 raft_consensus.cc:2804] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:20.310395 11878 raft_consensus.cc:697] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:20.311506 11878 consensus_queue.cc:237] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:20.325507 11344 catalog_manager.cc:5697] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 1 to 2, leader changed from 4913cffad83a48c9884efb39bd85add1 (127.1.123.66) to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 2 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:20.520740 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:26:20.712219 11598 raft_consensus.cc:1275] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:20.714175 11674 consensus_queue.cc:1048] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:20.723639 11599 raft_consensus.cc:1275] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:20.731189 11674 consensus_queue.cc:1048] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:20.744601 11801 raft_consensus.cc:1275] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:20.747480 11674 consensus_queue.cc:1048] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:20.768860 11522 raft_consensus.cc:1275] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:20.773355 11878 consensus_queue.cc:1048] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:20.795475 11802 raft_consensus.cc:1275] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:20.800915 11771 consensus_queue.cc:1048] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:20.830528 11801 raft_consensus.cc:1275] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:20.836499 11878 consensus_queue.cc:1048] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:26:20.924252  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:20.934963 11522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
mode: GRACEFUL
new_leader_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:47218
I20260709 12:26:20.935614 11522 raft_consensus.cc:606] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Received request to transfer leadership to TS 7e09e4b3d55044cd82210564ac064e9a
I20260709 12:26:20.939129 11522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
mode: GRACEFUL
new_leader_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:47218
I20260709 12:26:20.939766 11522 raft_consensus.cc:606] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Received request to transfer leadership to TS 7e09e4b3d55044cd82210564ac064e9a
I20260709 12:26:20.943586 11599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
mode: GRACEFUL
new_leader_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:53838
I20260709 12:26:20.944618 11599 raft_consensus.cc:606] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Received request to transfer leadership to TS 4913cffad83a48c9884efb39bd85add1
I20260709 12:26:20.946455  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:26:20.947629  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:20.948357  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:21.370258 11674 raft_consensus.cc:993] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1: : Instructing follower 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:21.370923 11674 raft_consensus.cc:1081] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Signalling peer 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:21.373245 11599 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:53822
I20260709 12:26:21.374326 11599 raft_consensus.cc:493] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:21.374989 11599 raft_consensus.cc:3060] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:21.381876 11599 raft_consensus.cc:515] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:21.384359 11599 leader_election.cc:290] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 3 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:21.385118 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:21.385670 11522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e67ad15454a410a9d37970d963b319b" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:21.386351 11522 raft_consensus.cc:3055] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:21.386696 11522 raft_consensus.cc:740] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:21.387094 11800 raft_consensus.cc:3060] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:21.387594 11522 consensus_queue.cc:260] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:21.389178 11849 raft_consensus.cc:993] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a: : Instructing follower 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:21.389392 11522 raft_consensus.cc:3060] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:21.390000 11896 raft_consensus.cc:1081] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Signalling peer 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:21.392048 11525 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:26:21.392985 11525 raft_consensus.cc:493] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:21.393378 11525 raft_consensus.cc:3060] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:21.400766 11522 raft_consensus.cc:2468] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 3.
I20260709 12:26:21.401443 11800 raft_consensus.cc:2468] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 3.
I20260709 12:26:21.402943 11563 leader_election.cc:304] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:21.403982 11896 raft_consensus.cc:2804] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:21.404544 11896 raft_consensus.cc:697] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:21.405696 11896 consensus_queue.cc:237] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:21.405757 11525 raft_consensus.cc:515] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:21.407981 11525 leader_election.cc:290] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 3 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:21.409696 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:21.410530 11800 raft_consensus.cc:3060] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:21.411778 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b88be8610a74af1ae2878b2e27da46d" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:21.414861 11599 raft_consensus.cc:3055] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:21.415419 11599 raft_consensus.cc:740] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:21.417006 11599 consensus_queue.cc:260] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:21.418383 11599 raft_consensus.cc:3060] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:21.419771 11348 catalog_manager.cc:5697] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 2 to 3, leader changed from 4913cffad83a48c9884efb39bd85add1 (127.1.123.66) to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:21.423164 11800 raft_consensus.cc:2468] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 3.
I20260709 12:26:21.425158 11489 leader_election.cc:304] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:21.426683 11887 raft_consensus.cc:2804] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:21.427244 11887 raft_consensus.cc:697] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 3 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:21.428251 11887 consensus_queue.cc:237] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:21.430744 11599 raft_consensus.cc:2468] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 3.
I20260709 12:26:21.432969 11771 raft_consensus.cc:993] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1: : Instructing follower 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:21.434610 11887 raft_consensus.cc:1081] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Signalling peer 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:21.438041 11599 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:53822
I20260709 12:26:21.439293 11599 raft_consensus.cc:493] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:21.440085 11599 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:21.446162 11344 catalog_manager.cc:5697] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 2 to 3, leader changed from 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67) to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 3 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:21.450318 11599 raft_consensus.cc:515] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:21.452111 11599 leader_election.cc:290] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 3 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:21.453125 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:21.454085 11800 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:21.455499 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:21.456342 11525 raft_consensus.cc:3055] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:21.456703 11525 raft_consensus.cc:740] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:21.457625 11525 consensus_queue.cc:260] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:21.459478 11525 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:21.464979 11800 raft_consensus.cc:2468] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 3.
I20260709 12:26:21.466392 11563 leader_election.cc:304] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:21.467355 11896 raft_consensus.cc:2804] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:21.468839 11896 raft_consensus.cc:697] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:21.469980 11896 consensus_queue.cc:237] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:21.470772 11525 raft_consensus.cc:2468] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 3.
I20260709 12:26:21.481930 11344 catalog_manager.cc:5697] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 2 to 3, leader changed from 4913cffad83a48c9884efb39bd85add1 (127.1.123.66) to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 3 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:21.521916 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:26:21.846161 11599 raft_consensus.cc:1275] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:21.848460 11887 consensus_queue.cc:1048] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:21.866569 11801 raft_consensus.cc:1275] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:21.870129 11674 consensus_queue.cc:1048] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:21.876935 11801 raft_consensus.cc:1275] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:21.879009 11849 consensus_queue.cc:1048] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:21.898061 11525 raft_consensus.cc:1275] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:21.901260 11849 consensus_queue.cc:1048] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:21.912201 11801 raft_consensus.cc:1275] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:21.915096 11912 consensus_queue.cc:1048] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:21.924041 11525 raft_consensus.cc:1275] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:21.927358 11912 consensus_queue.cc:1048] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:21.957370  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:21.968654 11525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19f60a8d99194fde8fa4b969c5a35b3a"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
mode: GRACEFUL
new_leader_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:47218
I20260709 12:26:21.969422 11525 raft_consensus.cc:606] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Received request to transfer leadership to TS 7e09e4b3d55044cd82210564ac064e9a
I20260709 12:26:21.974598 11598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
mode: GRACEFUL
new_leader_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:53838
I20260709 12:26:21.975358 11598 raft_consensus.cc:606] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Received request to transfer leadership to TS 4913cffad83a48c9884efb39bd85add1
I20260709 12:26:21.979911 11598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
mode: GRACEFUL
new_leader_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:53838
I20260709 12:26:21.980857 11598 raft_consensus.cc:606] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Received request to transfer leadership to TS 4913cffad83a48c9884efb39bd85add1
I20260709 12:26:21.983733  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:26:21.985246  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:21.986652  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:22.286960 11901 raft_consensus.cc:993] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1: : Instructing follower 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:22.287705 11674 raft_consensus.cc:1081] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Signalling peer 7e09e4b3d55044cd82210564ac064e9a to start an election
I20260709 12:26:22.290628 11598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "19f60a8d99194fde8fa4b969c5a35b3a"
dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
 from {username='slave'} at 127.0.0.1:53822
I20260709 12:26:22.291872 11598 raft_consensus.cc:493] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:22.292258 11598 raft_consensus.cc:3060] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:22.299625 11598 raft_consensus.cc:515] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:22.303995 11598 leader_election.cc:290] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 2 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319)
I20260709 12:26:22.304167 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f60a8d99194fde8fa4b969c5a35b3a" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:22.305109 11801 raft_consensus.cc:3060] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:22.305238 11525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f60a8d99194fde8fa4b969c5a35b3a" candidate_uuid: "7e09e4b3d55044cd82210564ac064e9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4913cffad83a48c9884efb39bd85add1"
I20260709 12:26:22.306658 11525 raft_consensus.cc:3055] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:22.307096 11525 raft_consensus.cc:740] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:22.308220 11525 consensus_queue.cc:260] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:22.310173 11525 raft_consensus.cc:3060] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:22.316165 11801 raft_consensus.cc:2468] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 2.
I20260709 12:26:22.318430 11563 leader_election.cc:304] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:22.319996 11896 raft_consensus.cc:2804] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:22.320627 11525 raft_consensus.cc:2468] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09e4b3d55044cd82210564ac064e9a in term 2.
I20260709 12:26:22.321063 11896 raft_consensus.cc:697] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Becoming Leader. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:22.322808 11896 consensus_queue.cc:237] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:22.336052 11344 catalog_manager.cc:5697] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a reported cstate change: term changed from 1 to 2, leader changed from 4913cffad83a48c9884efb39bd85add1 (127.1.123.66) to 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67). New cstate: current_term: 2 leader_uuid: "7e09e4b3d55044cd82210564ac064e9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:22.447628 11920 raft_consensus.cc:993] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a: : Instructing follower 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:22.448554 11920 raft_consensus.cc:1081] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Signalling peer 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:22.450215 11920 raft_consensus.cc:993] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a: : Instructing follower 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:22.450964 11896 raft_consensus.cc:1081] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Signalling peer 4913cffad83a48c9884efb39bd85add1 to start an election
I20260709 12:26:22.451586 11525 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:26:22.452330 11525 raft_consensus.cc:493] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:22.452732 11525 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:22.453600 11522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3"
dest_uuid: "4913cffad83a48c9884efb39bd85add1"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:26:22.454285 11522 raft_consensus.cc:493] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:22.454679 11522 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:22.459919 11525 raft_consensus.cc:515] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:22.462842 11522 raft_consensus.cc:515] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:22.463402 11525 leader_election.cc:290] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 4 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:22.464782 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:22.465790 11801 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:22.466065 11598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264556df3cbe4b28bd544ab1863a7cbe" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:22.466770 11598 raft_consensus.cc:3055] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:22.467074 11598 raft_consensus.cc:740] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:22.467965 11598 consensus_queue.cc:260] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:22.469388 11598 raft_consensus.cc:3060] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:22.470469 11599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7e09e4b3d55044cd82210564ac064e9a"
I20260709 12:26:22.471253 11599 raft_consensus.cc:3055] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:22.471607 11599 raft_consensus.cc:740] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e09e4b3d55044cd82210564ac064e9a, State: Running, Role: LEADER
I20260709 12:26:22.472496 11599 consensus_queue.cc:260] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:22.469388 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519e696ec65c4f19af8bda167a33baa3" candidate_uuid: "4913cffad83a48c9884efb39bd85add1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2cc1d50178294d2aa84107ddd51b0243"
I20260709 12:26:22.469568 11522 leader_election.cc:290] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 4 election: Requested vote from peers 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807), 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67:38901)
I20260709 12:26:22.473721 11599 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:22.475497 11801 raft_consensus.cc:2468] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 4.
I20260709 12:26:22.474048 11800 raft_consensus.cc:3060] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:22.477059 11489 leader_election.cc:304] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1d50178294d2aa84107ddd51b0243, 4913cffad83a48c9884efb39bd85add1; no voters: 
I20260709 12:26:22.478801 11598 raft_consensus.cc:2468] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 4.
I20260709 12:26:22.481336 11674 raft_consensus.cc:2804] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:22.482337 11674 raft_consensus.cc:697] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 4 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:22.483407 11674 consensus_queue.cc:237] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:22.486173 11599 raft_consensus.cc:2468] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 4.
I20260709 12:26:22.487967 11491 leader_election.cc:304] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4913cffad83a48c9884efb39bd85add1, 7e09e4b3d55044cd82210564ac064e9a; no voters: 
I20260709 12:26:22.488950 11901 raft_consensus.cc:2804] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:22.489501 11901 raft_consensus.cc:697] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 4 LEADER]: Becoming Leader. State: Replica: 4913cffad83a48c9884efb39bd85add1, State: Running, Role: LEADER
I20260709 12:26:22.490973 11800 raft_consensus.cc:2468] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4913cffad83a48c9884efb39bd85add1 in term 4.
I20260709 12:26:22.490679 11901 consensus_queue.cc:237] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } }
I20260709 12:26:22.495939 11344 catalog_manager.cc:5697] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 3 to 4, leader changed from 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67) to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 4 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:22.516431 11344 catalog_manager.cc:5697] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 reported cstate change: term changed from 3 to 4, leader changed from 7e09e4b3d55044cd82210564ac064e9a (127.1.123.67) to 4913cffad83a48c9884efb39bd85add1 (127.1.123.66). New cstate: current_term: 4 leader_uuid: "4913cffad83a48c9884efb39bd85add1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:22.527053 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:26:22.764050 11522 raft_consensus.cc:1275] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:22.766822 11896 consensus_queue.cc:1048] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4913cffad83a48c9884efb39bd85add1" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:22.783547 11800 raft_consensus.cc:1275] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Refusing update from remote peer 7e09e4b3d55044cd82210564ac064e9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:22.785490 11920 consensus_queue.cc:1048] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:22.997117 11800 raft_consensus.cc:1275] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:22.999342  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:22.999308 11901 consensus_queue.cc:1048] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:23.016177 11599 raft_consensus.cc:1275] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:23.024292 11901 consensus_queue.cc:1048] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:26:23.063936 11800 raft_consensus.cc:1275] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:23.068723 11901 consensus_queue.cc:1048] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1d50178294d2aa84107ddd51b0243" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:26:23.086265 11599 raft_consensus.cc:1275] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Refusing update from remote peer 4913cffad83a48c9884efb39bd85add1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:23.089221  1517 tablet_server.cc:179] TabletServer@127.1.123.65:34807 shutting down...
I20260709 12:26:23.089262 11929 consensus_queue.cc:1048] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09e4b3d55044cd82210564ac064e9a" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:26:23.143806 11563 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 13 similar messages]
W20260709 12:26:23.156097 11563 consensus_peers.cc:597] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:23.174239  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:23.175926  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:26:23.176874  1517 raft_consensus.cc:2243] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.177793  1517 raft_consensus.cc:2272] T 0e67ad15454a410a9d37970d963b319b P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.184638  1517 tablet_replica.cc:333] stopping tablet replica
W20260709 12:26:23.187063 11489 consensus_peers.cc:597] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:23.188040  1517 raft_consensus.cc:2243] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.188694  1517 raft_consensus.cc:2272] T 7f0e176bc87049d19ea24e510329e131 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.191632  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:26:23.192426  1517 raft_consensus.cc:2243] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.193030  1517 raft_consensus.cc:2272] T a5750167a0c848b2afc5b03bda200dcd P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.197391  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:26:23.198853  1517 raft_consensus.cc:2243] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.199644  1517 raft_consensus.cc:2272] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:23.206427 11563 consensus_peers.cc:597] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:23.207648  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:26:23.209100  1517 raft_consensus.cc:2243] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.210038  1517 raft_consensus.cc:2272] T 19f60a8d99194fde8fa4b969c5a35b3a P 2cc1d50178294d2aa84107ddd51b0243 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.215225  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:26:23.216629  1517 raft_consensus.cc:2243] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.217319  1517 raft_consensus.cc:2272] T 1337a2bbcb024c7694ba59774bf25c17 P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.220677  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:26:23.221521  1517 raft_consensus.cc:2243] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.222416  1517 raft_consensus.cc:2272] T 2b88be8610a74af1ae2878b2e27da46d P 2cc1d50178294d2aa84107ddd51b0243 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.225004  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:26:23.226203  1517 raft_consensus.cc:2243] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.226730  1517 raft_consensus.cc:2272] T 519e696ec65c4f19af8bda167a33baa3 P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.231451  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:26:23.232163  1517 raft_consensus.cc:2243] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.232952  1517 raft_consensus.cc:2272] T f6dd4115f9464d44a08b8e58f95486cb P 2cc1d50178294d2aa84107ddd51b0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.238180  1517 tablet_replica.cc:333] stopping tablet replica
I20260709 12:26:23.239319  1517 raft_consensus.cc:2243] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.240224  1517 raft_consensus.cc:2272] T 264556df3cbe4b28bd544ab1863a7cbe P 2cc1d50178294d2aa84107ddd51b0243 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.297464  1517 tablet_server.cc:196] TabletServer@127.1.123.65:34807 shutdown complete.
I20260709 12:26:23.332896  1517 tablet_server.cc:179] TabletServer@127.1.123.66:0 shutting down...
I20260709 12:26:23.394340  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:23.395411  1517 tablet_replica.cc:333] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1: stopping tablet replica
I20260709 12:26:23.396217  1517 raft_consensus.cc:2243] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:26:23.397691  1517 raft_consensus.cc:2272] T 2b88be8610a74af1ae2878b2e27da46d P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.400696  1517 tablet_replica.cc:333] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1: stopping tablet replica
I20260709 12:26:23.401343  1517 raft_consensus.cc:2243] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.401902  1517 raft_consensus.cc:2272] T 0e67ad15454a410a9d37970d963b319b P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.404510  1517 tablet_replica.cc:333] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1: stopping tablet replica
I20260709 12:26:23.405313  1517 raft_consensus.cc:2243] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:26:23.406360  1517 raft_consensus.cc:2272] T 519e696ec65c4f19af8bda167a33baa3 P 4913cffad83a48c9884efb39bd85add1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.409209  1517 tablet_replica.cc:333] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1: stopping tablet replica
I20260709 12:26:23.409940  1517 raft_consensus.cc:2243] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:26:23.411582  1517 raft_consensus.cc:2272] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 4913cffad83a48c9884efb39bd85add1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.414352  1517 tablet_replica.cc:333] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1: stopping tablet replica
I20260709 12:26:23.415133  1517 raft_consensus.cc:2243] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:26:23.416455  1517 raft_consensus.cc:2272] T a5750167a0c848b2afc5b03bda200dcd P 4913cffad83a48c9884efb39bd85add1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.419379  1517 tablet_replica.cc:333] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1: stopping tablet replica
I20260709 12:26:23.420583  1517 raft_consensus.cc:2243] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.421141  1517 raft_consensus.cc:2272] T 1337a2bbcb024c7694ba59774bf25c17 P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.424685  1517 tablet_replica.cc:333] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1: stopping tablet replica
I20260709 12:26:23.425730  1517 raft_consensus.cc:2243] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.426566  1517 raft_consensus.cc:2272] T 19f60a8d99194fde8fa4b969c5a35b3a P 4913cffad83a48c9884efb39bd85add1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.431272  1517 tablet_replica.cc:333] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1: stopping tablet replica
I20260709 12:26:23.431967  1517 raft_consensus.cc:2243] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.432554  1517 raft_consensus.cc:2272] T 7f0e176bc87049d19ea24e510329e131 P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.437245  1517 tablet_replica.cc:333] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1: stopping tablet replica
I20260709 12:26:23.437912  1517 raft_consensus.cc:2243] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:26:23.438881  1517 raft_consensus.cc:2272] T 264556df3cbe4b28bd544ab1863a7cbe P 4913cffad83a48c9884efb39bd85add1 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:23.440240 11563 consensus_peers.cc:597] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a -> Peer 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319): Couldn't send request to peer 4913cffad83a48c9884efb39bd85add1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:23.442808  1517 tablet_replica.cc:333] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1: stopping tablet replica
I20260709 12:26:23.443812  1517 raft_consensus.cc:2243] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:23.445232  1517 raft_consensus.cc:2272] T f6dd4115f9464d44a08b8e58f95486cb P 4913cffad83a48c9884efb39bd85add1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.494258  1517 tablet_server.cc:196] TabletServer@127.1.123.66:0 shutdown complete.
W20260709 12:26:23.527700 11563 consensus_peers.cc:597] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:23.528617 11395 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:26:23.546543  1517 tablet_server.cc:179] TabletServer@127.1.123.67:0 shutting down...
I20260709 12:26:23.618501  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:23.620659  1517 tablet_replica.cc:333] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a: stopping tablet replica
I20260709 12:26:23.622216  1517 raft_consensus.cc:2243] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.622942  1517 raft_consensus.cc:2272] T 2b88be8610a74af1ae2878b2e27da46d P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.625880  1517 tablet_replica.cc:333] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a: stopping tablet replica
W20260709 12:26:23.626303 11563 consensus_peers.cc:597] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a -> Peer 2cc1d50178294d2aa84107ddd51b0243 (127.1.123.65:34807): Couldn't send request to peer 2cc1d50178294d2aa84107ddd51b0243. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:34807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:23.627226  1517 raft_consensus.cc:2243] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:26:23.628482  1517 raft_consensus.cc:2272] T 0e67ad15454a410a9d37970d963b319b P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.635622  1517 tablet_replica.cc:333] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a: stopping tablet replica
I20260709 12:26:23.636861  1517 raft_consensus.cc:2243] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:23.639035  1517 raft_consensus.cc:2272] T 1337a2bbcb024c7694ba59774bf25c17 P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.643846  1517 tablet_replica.cc:333] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a: stopping tablet replica
I20260709 12:26:23.644754  1517 raft_consensus.cc:2243] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:26:23.648478 11563 consensus_peers.cc:597] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a -> Peer 4913cffad83a48c9884efb39bd85add1 (127.1.123.66:37319): Couldn't send request to peer 4913cffad83a48c9884efb39bd85add1. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:37319: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:23.649219  1517 raft_consensus.cc:2272] T 19f60a8d99194fde8fa4b969c5a35b3a P 7e09e4b3d55044cd82210564ac064e9a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.653406  1517 tablet_replica.cc:333] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a: stopping tablet replica
I20260709 12:26:23.654210  1517 raft_consensus.cc:2243] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:23.655457  1517 raft_consensus.cc:2272] T 7f0e176bc87049d19ea24e510329e131 P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.661310  1517 tablet_replica.cc:333] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a: stopping tablet replica
I20260709 12:26:23.662992  1517 raft_consensus.cc:2243] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.663949  1517 raft_consensus.cc:2272] T 93d2f6a8f3a7462c90f0047ae8cdf4fa P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.666955  1517 tablet_replica.cc:333] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a: stopping tablet replica
I20260709 12:26:23.667658  1517 raft_consensus.cc:2243] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.668181  1517 raft_consensus.cc:2272] T a5750167a0c848b2afc5b03bda200dcd P 7e09e4b3d55044cd82210564ac064e9a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.671224  1517 tablet_replica.cc:333] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a: stopping tablet replica
I20260709 12:26:23.672544  1517 raft_consensus.cc:2243] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.673146  1517 raft_consensus.cc:2272] T 519e696ec65c4f19af8bda167a33baa3 P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.681077  1517 tablet_replica.cc:333] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a: stopping tablet replica
I20260709 12:26:23.681888  1517 raft_consensus.cc:2243] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.683053  1517 raft_consensus.cc:2272] T f6dd4115f9464d44a08b8e58f95486cb P 7e09e4b3d55044cd82210564ac064e9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.685516  1517 tablet_replica.cc:333] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a: stopping tablet replica
I20260709 12:26:23.686636  1517 raft_consensus.cc:2243] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:23.688395  1517 raft_consensus.cc:2272] T 264556df3cbe4b28bd544ab1863a7cbe P 7e09e4b3d55044cd82210564ac064e9a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.753249  1517 tablet_server.cc:196] TabletServer@127.1.123.67:0 shutdown complete.
I20260709 12:26:23.793013  1517 master.cc:562] Master@127.1.123.126:42475 shutting down...
I20260709 12:26:23.862254  1517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:23.862949  1517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:23.863942  1517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4efc77e575344a799d4727926b1f8c3d: stopping tablet replica
I20260709 12:26:23.894675  1517 master.cc:584] Master@127.1.123.126:42475 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (14712 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:26:23.973102  1517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.123.126:33463
I20260709 12:26:23.980135  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:23.995868 11943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:23.995558 11941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:23.995498 11940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:24.000707  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:24.002524  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:24.002825  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:24.003023  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599984002998 us; error 0 us; skew 500 ppm
I20260709 12:26:24.003820  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:24.010699  1517 webserver.cc:533] Webserver started at http://127.1.123.126:41799/ using document root <none> and password file <none>
I20260709 12:26:24.011430  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:24.011632  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:24.012049  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:24.013769  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/master-0-root/instance:
uuid: "48fbbb944fd243329e221c93b5d9437a"
format_stamp: "Formatted at 2026-07-09 12:26:24 on dist-test-slave-08l4"
I20260709 12:26:24.023092  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:26:24.028846 11948 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:24.030275  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:26:24.030638  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/master-0-root
uuid: "48fbbb944fd243329e221c93b5d9437a"
format_stamp: "Formatted at 2026-07-09 12:26:24 on dist-test-slave-08l4"
I20260709 12:26:24.030922  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:24.062835  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:24.066404  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:24.163749  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.126:33463
I20260709 12:26:24.163885 11999 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.126:33463 every 8 connection(s)
I20260709 12:26:24.170397 12000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:24.194152 12000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a: Bootstrap starting.
I20260709 12:26:24.201952 12000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:24.208978 12000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a: No bootstrap required, opened a new log
I20260709 12:26:24.212011 12000 raft_consensus.cc:359] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48fbbb944fd243329e221c93b5d9437a" member_type: VOTER }
I20260709 12:26:24.212580 12000 raft_consensus.cc:385] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:24.213289 12000 raft_consensus.cc:740] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbbb944fd243329e221c93b5d9437a, State: Initialized, Role: FOLLOWER
I20260709 12:26:24.214270 12000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48fbbb944fd243329e221c93b5d9437a" member_type: VOTER }
I20260709 12:26:24.214915 12000 raft_consensus.cc:399] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:26:24.215207 12000 raft_consensus.cc:493] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:26:24.215530 12000 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:24.223644 12000 raft_consensus.cc:515] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48fbbb944fd243329e221c93b5d9437a" member_type: VOTER }
I20260709 12:26:24.224431 12000 leader_election.cc:304] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48fbbb944fd243329e221c93b5d9437a; no voters: 
I20260709 12:26:24.226255 12000 leader_election.cc:290] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:26:24.226881 12003 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:24.228935 12003 raft_consensus.cc:697] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [term 1 LEADER]: Becoming Leader. State: Replica: 48fbbb944fd243329e221c93b5d9437a, State: Running, Role: LEADER
I20260709 12:26:24.229682 12000 sys_catalog.cc:565] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:26:24.229823 12003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48fbbb944fd243329e221c93b5d9437a" member_type: VOTER }
I20260709 12:26:24.237270 12004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48fbbb944fd243329e221c93b5d9437a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48fbbb944fd243329e221c93b5d9437a" member_type: VOTER } }
I20260709 12:26:24.238358 12004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [sys.catalog]: This master's current role is: LEADER
I20260709 12:26:24.238445 12005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48fbbb944fd243329e221c93b5d9437a. Latest consensus state: current_term: 1 leader_uuid: "48fbbb944fd243329e221c93b5d9437a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48fbbb944fd243329e221c93b5d9437a" member_type: VOTER } }
I20260709 12:26:24.239600 12005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [sys.catalog]: This master's current role is: LEADER
I20260709 12:26:24.245033 12012 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:26:24.255980 12012 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:26:24.263433  1517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:26:24.270736 12012 catalog_manager.cc:1383] Generated new cluster ID: 3b403a0fc7b74ccfb464d38eb1f3c636
I20260709 12:26:24.271231 12012 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:26:24.292285 12012 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:26:24.294430 12012 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:26:24.308851 12012 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a: Generated new TSK 0
I20260709 12:26:24.310254 12012 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:26:24.332360  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:24.341635 12021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:24.343123 12022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:24.347476  1517 server_base.cc:1061] running on GCE node
W20260709 12:26:24.347961 12024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:24.349285  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:24.349628  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:24.349787  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599984349774 us; error 0 us; skew 500 ppm
I20260709 12:26:24.350440  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:24.354943  1517 webserver.cc:533] Webserver started at http://127.1.123.65:33929/ using document root <none> and password file <none>
I20260709 12:26:24.355572  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:24.355782  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:24.356009  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:24.357430  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-0-root/instance:
uuid: "2603f5a8c64c4d01951548904c8165cf"
format_stamp: "Formatted at 2026-07-09 12:26:24 on dist-test-slave-08l4"
I20260709 12:26:24.367316  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:26:24.374531 12029 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:24.377892  1517 fs_manager.cc:730] Time spent opening block manager: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:26:24.378371  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-0-root
uuid: "2603f5a8c64c4d01951548904c8165cf"
format_stamp: "Formatted at 2026-07-09 12:26:24 on dist-test-slave-08l4"
I20260709 12:26:24.379336  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:24.402230  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:24.403928  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:24.406129  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:24.410378  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:24.411126  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 12:26:24.411470  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:24.411667  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:24.493858  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.65:42417
I20260709 12:26:24.493945 12091 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.65:42417 every 8 connection(s)
I20260709 12:26:24.500867  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:24.523496 12096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:24.527745 12097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:24.541184 12100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:24.541770  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:24.543623  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:24.543921  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:24.544131  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599984544111 us; error 0 us; skew 500 ppm
I20260709 12:26:24.545837 12092 heartbeater.cc:344] Connected to a master server at 127.1.123.126:33463
I20260709 12:26:24.546557 12092 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:24.545910  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:24.547819 12092 heartbeater.cc:507] Master 127.1.123.126:33463 requested a full tablet report, sending...
I20260709 12:26:24.551476 11965 ts_manager.cc:194] Registered new tserver with Master: 2603f5a8c64c4d01951548904c8165cf (127.1.123.65:42417)
I20260709 12:26:24.552665  1517 webserver.cc:533] Webserver started at http://127.1.123.66:34279/ using document root <none> and password file <none>
I20260709 12:26:24.553460  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:24.554018  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:24.554590 11965 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33334
I20260709 12:26:24.554630  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:24.561440  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-1-root/instance:
uuid: "d7c6b284049b41a5b9446b3e3057914d"
format_stamp: "Formatted at 2026-07-09 12:26:24 on dist-test-slave-08l4"
I20260709 12:26:24.572803  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.010s	user 0.005s	sys 0.004s
I20260709 12:26:24.578369 12104 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:24.580401  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 12:26:24.581211  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-1-root
uuid: "d7c6b284049b41a5b9446b3e3057914d"
format_stamp: "Formatted at 2026-07-09 12:26:24 on dist-test-slave-08l4"
I20260709 12:26:24.581739  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:24.623574  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:24.625871  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:24.629328  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:24.634428  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:24.635095  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 12:26:24.635516  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:24.636026  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:24.716105  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.66:36389
I20260709 12:26:24.716208 12166 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.66:36389 every 8 connection(s)
I20260709 12:26:24.724471  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:24.738651 12170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:24.741285 12171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:24.748178 12173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:24.750334  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:24.751353 12167 heartbeater.cc:344] Connected to a master server at 127.1.123.126:33463
I20260709 12:26:24.752568 12167 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:24.752801  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:24.753188  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:24.753374  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599984753327 us; error 0 us; skew 500 ppm
I20260709 12:26:24.754545  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:24.754925 12167 heartbeater.cc:507] Master 127.1.123.126:33463 requested a full tablet report, sending...
I20260709 12:26:24.758600  1517 webserver.cc:533] Webserver started at http://127.1.123.67:37537/ using document root <none> and password file <none>
I20260709 12:26:24.758638 11965 ts_manager.cc:194] Registered new tserver with Master: d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389)
I20260709 12:26:24.759505  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:24.759800  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:24.760124  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:24.762621  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-2-root/instance:
uuid: "6afa6b902e81431297c421d1d4479eb3"
format_stamp: "Formatted at 2026-07-09 12:26:24 on dist-test-slave-08l4"
I20260709 12:26:24.762862 11965 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33338
I20260709 12:26:24.773746  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:26:24.780361 12178 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:24.782821  1517 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:26:24.783406  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-2-root
uuid: "6afa6b902e81431297c421d1d4479eb3"
format_stamp: "Formatted at 2026-07-09 12:26:24 on dist-test-slave-08l4"
I20260709 12:26:24.784045  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599626433458-1517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:24.814291  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:24.815970  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:24.818431  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:24.821260  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:24.821729  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.001s	sys 0.000s
I20260709 12:26:24.822044  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:24.822220  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:24.898154  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.67:38515
I20260709 12:26:24.898387 12240 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.67:38515 every 8 connection(s)
I20260709 12:26:24.930823 12241 heartbeater.cc:344] Connected to a master server at 127.1.123.126:33463
I20260709 12:26:24.931671 12241 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:24.932591 12241 heartbeater.cc:507] Master 127.1.123.126:33463 requested a full tablet report, sending...
I20260709 12:26:24.936509 11965 ts_manager.cc:194] Registered new tserver with Master: 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67:38515)
I20260709 12:26:24.936843  1517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.03029502s
I20260709 12:26:24.939268 11965 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33340
I20260709 12:26:24.981209 11965 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33352:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:26:24.983736 11965 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:26:25.077289 12057 tablet_service.cc:1511] Processing CreateTablet for tablet cb20be30d73d48fe8188662d226c28ba (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0d77de76e5fe4dc5afe386eb647ae783]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:26:25.079268 12057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb20be30d73d48fe8188662d226c28ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.078728 12132 tablet_service.cc:1511] Processing CreateTablet for tablet cb20be30d73d48fe8188662d226c28ba (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0d77de76e5fe4dc5afe386eb647ae783]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:26:25.081737 12132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb20be30d73d48fe8188662d226c28ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.082129 12131 tablet_service.cc:1511] Processing CreateTablet for tablet 53d829995fac45d4b59556791396d37b (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0d77de76e5fe4dc5afe386eb647ae783]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:26:25.083636 12205 tablet_service.cc:1511] Processing CreateTablet for tablet 53d829995fac45d4b59556791396d37b (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0d77de76e5fe4dc5afe386eb647ae783]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:26:25.086374 12056 tablet_service.cc:1511] Processing CreateTablet for tablet 53d829995fac45d4b59556791396d37b (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0d77de76e5fe4dc5afe386eb647ae783]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:26:25.087569 12206 tablet_service.cc:1511] Processing CreateTablet for tablet cb20be30d73d48fe8188662d226c28ba (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0d77de76e5fe4dc5afe386eb647ae783]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:26:25.089821 12205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53d829995fac45d4b59556791396d37b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.090754 12131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53d829995fac45d4b59556791396d37b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.093205 12206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb20be30d73d48fe8188662d226c28ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.089385 12056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53d829995fac45d4b59556791396d37b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.133792 12259 tablet_bootstrap.cc:492] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf: Bootstrap starting.
I20260709 12:26:25.143306 12259 tablet_bootstrap.cc:654] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:25.160879 12261 tablet_bootstrap.cc:492] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d: Bootstrap starting.
I20260709 12:26:25.161751 12262 tablet_bootstrap.cc:492] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3: Bootstrap starting.
I20260709 12:26:25.163801 12259 tablet_bootstrap.cc:492] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf: No bootstrap required, opened a new log
I20260709 12:26:25.167193 12259 ts_tablet_manager.cc:1403] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.008s
I20260709 12:26:25.172828 12259 raft_consensus.cc:359] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.173732 12259 raft_consensus.cc:385] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.174347 12261 tablet_bootstrap.cc:654] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:25.175225 12259 raft_consensus.cc:740] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2603f5a8c64c4d01951548904c8165cf, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.176296 12262 tablet_bootstrap.cc:654] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:25.176577 12259 consensus_queue.cc:260] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.187177 12261 tablet_bootstrap.cc:492] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d: No bootstrap required, opened a new log
I20260709 12:26:25.186008 12092 heartbeater.cc:499] Master 127.1.123.126:33463 was elected leader, sending a full tablet report...
I20260709 12:26:25.188808 12259 ts_tablet_manager.cc:1434] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.009s
I20260709 12:26:25.191215 12261 ts_tablet_manager.cc:1403] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.011s
I20260709 12:26:25.191931 12259 tablet_bootstrap.cc:492] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf: Bootstrap starting.
I20260709 12:26:25.197081 12261 raft_consensus.cc:359] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.197962 12262 tablet_bootstrap.cc:492] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3: No bootstrap required, opened a new log
I20260709 12:26:25.199646 12261 raft_consensus.cc:385] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.200116 12262 ts_tablet_manager.cc:1403] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.012s
I20260709 12:26:25.200188 12261 raft_consensus.cc:740] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c6b284049b41a5b9446b3e3057914d, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.200984 12259 tablet_bootstrap.cc:654] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:25.203737 12261 consensus_queue.cc:260] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.206969 12262 raft_consensus.cc:359] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.208178 12262 raft_consensus.cc:385] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.208467 12262 raft_consensus.cc:740] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6afa6b902e81431297c421d1d4479eb3, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.209348 12262 consensus_queue.cc:260] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.218992 12259 tablet_bootstrap.cc:492] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf: No bootstrap required, opened a new log
I20260709 12:26:25.219837 12259 ts_tablet_manager.cc:1403] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.004s
I20260709 12:26:25.221485 12167 heartbeater.cc:499] Master 127.1.123.126:33463 was elected leader, sending a full tablet report...
I20260709 12:26:25.224249 12259 raft_consensus.cc:359] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.225219 12259 raft_consensus.cc:385] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.225721 12259 raft_consensus.cc:740] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2603f5a8c64c4d01951548904c8165cf, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.227255 12259 consensus_queue.cc:260] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.227860 12241 heartbeater.cc:499] Master 127.1.123.126:33463 was elected leader, sending a full tablet report...
I20260709 12:26:25.228960 12262 ts_tablet_manager.cc:1434] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.020s
I20260709 12:26:25.230386 12262 tablet_bootstrap.cc:492] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3: Bootstrap starting.
I20260709 12:26:25.230562 12259 ts_tablet_manager.cc:1434] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.008s
I20260709 12:26:25.233892 12261 ts_tablet_manager.cc:1434] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d: Time spent starting tablet: real 0.042s	user 0.016s	sys 0.006s
I20260709 12:26:25.235630 12261 tablet_bootstrap.cc:492] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d: Bootstrap starting.
I20260709 12:26:25.239425 12262 tablet_bootstrap.cc:654] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:25.244513 12261 tablet_bootstrap.cc:654] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d: Neither blocks nor log segments found. Creating new log.
W20260709 12:26:25.246963 12013 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:25.267019 12261 tablet_bootstrap.cc:492] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d: No bootstrap required, opened a new log
I20260709 12:26:25.268258 12261 ts_tablet_manager.cc:1403] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.003s
I20260709 12:26:25.269608 12262 tablet_bootstrap.cc:492] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3: No bootstrap required, opened a new log
I20260709 12:26:25.270581 12264 raft_consensus.cc:493] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.270836 12262 ts_tablet_manager.cc:1403] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3: Time spent bootstrapping tablet: real 0.041s	user 0.032s	sys 0.004s
I20260709 12:26:25.271260 12264 raft_consensus.cc:515] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.272605 12261 raft_consensus.cc:359] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.273851 12261 raft_consensus.cc:385] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.274184 12261 raft_consensus.cc:740] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c6b284049b41a5b9446b3e3057914d, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.274997 12262 raft_consensus.cc:359] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.276470 12262 raft_consensus.cc:385] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.276801 12262 raft_consensus.cc:740] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6afa6b902e81431297c421d1d4479eb3, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.277632 12262 consensus_queue.cc:260] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.278460 12268 raft_consensus.cc:493] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.278929 12264 leader_election.cc:290] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389), 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67:38515)
I20260709 12:26:25.279716 12268 raft_consensus.cc:515] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.289027 12262 ts_tablet_manager.cc:1434] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.003s
I20260709 12:26:25.275061 12261 consensus_queue.cc:260] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.293740 12268 leader_election.cc:290] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389), 2603f5a8c64c4d01951548904c8165cf (127.1.123.65:42417)
I20260709 12:26:25.306807 12261 ts_tablet_manager.cc:1434] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d: Time spent starting tablet: real 0.038s	user 0.013s	sys 0.000s
I20260709 12:26:25.324045 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba" candidate_uuid: "6afa6b902e81431297c421d1d4479eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2603f5a8c64c4d01951548904c8165cf" is_pre_election: true
I20260709 12:26:25.325392 12067 raft_consensus.cc:2468] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6afa6b902e81431297c421d1d4479eb3 in term 0.
I20260709 12:26:25.327579 12182 leader_election.cc:304] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2603f5a8c64c4d01951548904c8165cf, 6afa6b902e81431297c421d1d4479eb3; no voters: 
I20260709 12:26:25.328734 12268 raft_consensus.cc:2804] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:25.329228 12268 raft_consensus.cc:493] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.329504 12268 raft_consensus.cc:3060] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.328513 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba" candidate_uuid: "6afa6b902e81431297c421d1d4479eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c6b284049b41a5b9446b3e3057914d" is_pre_election: true
I20260709 12:26:25.333110 12142 raft_consensus.cc:2468] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6afa6b902e81431297c421d1d4479eb3 in term 0.
I20260709 12:26:25.337500 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c6b284049b41a5b9446b3e3057914d" is_pre_election: true
I20260709 12:26:25.338466 12142 raft_consensus.cc:2468] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 0.
I20260709 12:26:25.340195 12030 leader_election.cc:304] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2603f5a8c64c4d01951548904c8165cf, d7c6b284049b41a5b9446b3e3057914d; no voters: 
I20260709 12:26:25.341238 12264 raft_consensus.cc:2804] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:25.341673 12264 raft_consensus.cc:493] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.341971 12264 raft_consensus.cc:3060] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.347074 12268 raft_consensus.cc:515] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.352662 12216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afa6b902e81431297c421d1d4479eb3" is_pre_election: true
I20260709 12:26:25.353004 12268 leader_election.cc:290] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [CANDIDATE]: Term 1 election: Requested vote from peers d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389), 2603f5a8c64c4d01951548904c8165cf (127.1.123.65:42417)
I20260709 12:26:25.356755 12216 raft_consensus.cc:2393] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2603f5a8c64c4d01951548904c8165cf in current term 1: Already voted for candidate 6afa6b902e81431297c421d1d4479eb3 in this term.
I20260709 12:26:25.357025 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba" candidate_uuid: "6afa6b902e81431297c421d1d4479eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2603f5a8c64c4d01951548904c8165cf"
I20260709 12:26:25.357450 12264 raft_consensus.cc:515] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.359431 12067 raft_consensus.cc:2393] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6afa6b902e81431297c421d1d4479eb3 in current term 1: Already voted for candidate 2603f5a8c64c4d01951548904c8165cf in this term.
I20260709 12:26:25.359546 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba" candidate_uuid: "6afa6b902e81431297c421d1d4479eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c6b284049b41a5b9446b3e3057914d"
I20260709 12:26:25.363289 12142 raft_consensus.cc:3060] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.369752 12264 leader_election.cc:290] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 election: Requested vote from peers d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389), 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67:38515)
I20260709 12:26:25.369231 12141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c6b284049b41a5b9446b3e3057914d"
I20260709 12:26:25.369200 12216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afa6b902e81431297c421d1d4479eb3"
I20260709 12:26:25.371788 12216 raft_consensus.cc:2393] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2603f5a8c64c4d01951548904c8165cf in current term 1: Already voted for candidate 6afa6b902e81431297c421d1d4479eb3 in this term.
I20260709 12:26:25.373433 12031 leader_election.cc:304] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2603f5a8c64c4d01951548904c8165cf; no voters: 6afa6b902e81431297c421d1d4479eb3, d7c6b284049b41a5b9446b3e3057914d
I20260709 12:26:25.374506 12264 raft_consensus.cc:2749] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:26:25.381656 12142 raft_consensus.cc:2468] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6afa6b902e81431297c421d1d4479eb3 in term 1.
I20260709 12:26:25.386610 12179 leader_election.cc:304] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6afa6b902e81431297c421d1d4479eb3, d7c6b284049b41a5b9446b3e3057914d; no voters: 2603f5a8c64c4d01951548904c8165cf
I20260709 12:26:25.388067 12268 raft_consensus.cc:2804] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:25.388917 12268 raft_consensus.cc:697] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 6afa6b902e81431297c421d1d4479eb3, State: Running, Role: LEADER
I20260709 12:26:25.390650 12268 consensus_queue.cc:237] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.406452 11965 catalog_manager.cc:5697] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "6afa6b902e81431297c421d1d4479eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:25.482041 12264 raft_consensus.cc:493] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.482954 12264 raft_consensus.cc:515] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.485773 12264 leader_election.cc:290] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389), 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67:38515)
I20260709 12:26:25.487012 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d829995fac45d4b59556791396d37b" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c6b284049b41a5b9446b3e3057914d" is_pre_election: true
I20260709 12:26:25.487370 12216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d829995fac45d4b59556791396d37b" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afa6b902e81431297c421d1d4479eb3" is_pre_election: true
I20260709 12:26:25.488034 12142 raft_consensus.cc:2468] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 0.
I20260709 12:26:25.488245 12216 raft_consensus.cc:2468] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 0.
I20260709 12:26:25.489416 12030 leader_election.cc:304] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2603f5a8c64c4d01951548904c8165cf, d7c6b284049b41a5b9446b3e3057914d; no voters: 
I20260709 12:26:25.490422 12264 raft_consensus.cc:2804] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:25.491001 12264 raft_consensus.cc:493] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.491397 12264 raft_consensus.cc:3060] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.499275 12264 raft_consensus.cc:515] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.501655 12264 leader_election.cc:290] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 election: Requested vote from peers d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389), 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67:38515)
I20260709 12:26:25.503350 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d829995fac45d4b59556791396d37b" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c6b284049b41a5b9446b3e3057914d"
I20260709 12:26:25.503628 12216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d829995fac45d4b59556791396d37b" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afa6b902e81431297c421d1d4479eb3"
I20260709 12:26:25.504050 12142 raft_consensus.cc:3060] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.504885 12216 raft_consensus.cc:3060] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.512889 12216 raft_consensus.cc:2468] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 1.
I20260709 12:26:25.514536 12142 raft_consensus.cc:2468] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 1.
I20260709 12:26:25.515250 12031 leader_election.cc:304] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2603f5a8c64c4d01951548904c8165cf, 6afa6b902e81431297c421d1d4479eb3; no voters: 
I20260709 12:26:25.516527 12264 raft_consensus.cc:2804] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:25.518014 12264 raft_consensus.cc:697] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Becoming Leader. State: Replica: 2603f5a8c64c4d01951548904c8165cf, State: Running, Role: LEADER
I20260709 12:26:25.519052 12264 consensus_queue.cc:237] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.529951 11965 catalog_manager.cc:5697] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 2603f5a8c64c4d01951548904c8165cf (127.1.123.65). New cstate: current_term: 1 leader_uuid: "2603f5a8c64c4d01951548904c8165cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:25.619069 11965 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33366:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:26:25.621918 11965 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:26:25.657024 12057 tablet_service.cc:1511] Processing CreateTablet for tablet 5b23aa41043c41f795dfd248e74e828e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a3bda8e516e149eda22a66c9bf1cb02b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:26:25.658339 12132 tablet_service.cc:1511] Processing CreateTablet for tablet 5b23aa41043c41f795dfd248e74e828e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a3bda8e516e149eda22a66c9bf1cb02b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:26:25.660383 12206 tablet_service.cc:1511] Processing CreateTablet for tablet 5b23aa41043c41f795dfd248e74e828e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a3bda8e516e149eda22a66c9bf1cb02b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:26:25.660933 12132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b23aa41043c41f795dfd248e74e828e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.660933 12057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b23aa41043c41f795dfd248e74e828e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.662405 12206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b23aa41043c41f795dfd248e74e828e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.666316 12056 tablet_service.cc:1511] Processing CreateTablet for tablet 10090369098e4618b6228fdb03888413 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a3bda8e516e149eda22a66c9bf1cb02b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:26:25.667182 12131 tablet_service.cc:1511] Processing CreateTablet for tablet 10090369098e4618b6228fdb03888413 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a3bda8e516e149eda22a66c9bf1cb02b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:26:25.667789 12056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10090369098e4618b6228fdb03888413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.669425 12131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10090369098e4618b6228fdb03888413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.678666 12205 tablet_service.cc:1511] Processing CreateTablet for tablet 10090369098e4618b6228fdb03888413 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a3bda8e516e149eda22a66c9bf1cb02b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:26:25.680545 12205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10090369098e4618b6228fdb03888413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:25.697712 12259 tablet_bootstrap.cc:492] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf: Bootstrap starting.
I20260709 12:26:25.718361 12259 tablet_bootstrap.cc:654] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:25.720389 12262 tablet_bootstrap.cc:492] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3: Bootstrap starting.
I20260709 12:26:25.724325 12261 tablet_bootstrap.cc:492] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d: Bootstrap starting.
I20260709 12:26:25.734908 12261 tablet_bootstrap.cc:654] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:25.740655 12262 tablet_bootstrap.cc:654] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:25.742656 12261 tablet_bootstrap.cc:492] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d: No bootstrap required, opened a new log
I20260709 12:26:25.743373 12261 ts_tablet_manager.cc:1403] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:26:25.749325 12261 raft_consensus.cc:359] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.751582 12259 tablet_bootstrap.cc:492] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf: No bootstrap required, opened a new log
I20260709 12:26:25.751825 12261 raft_consensus.cc:385] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.749390 12262 tablet_bootstrap.cc:492] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3: No bootstrap required, opened a new log
I20260709 12:26:25.752653 12261 raft_consensus.cc:740] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c6b284049b41a5b9446b3e3057914d, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.752653 12259 ts_tablet_manager.cc:1403] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf: Time spent bootstrapping tablet: real 0.056s	user 0.027s	sys 0.013s
I20260709 12:26:25.753624 12262 ts_tablet_manager.cc:1403] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.005s
I20260709 12:26:25.754009 12261 consensus_queue.cc:260] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.757038 12261 ts_tablet_manager.cc:1434] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 12:26:25.758240 12262 raft_consensus.cc:359] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.758057 12259 raft_consensus.cc:359] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.758818 12261 tablet_bootstrap.cc:492] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d: Bootstrap starting.
I20260709 12:26:25.759255 12262 raft_consensus.cc:385] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.759317 12259 raft_consensus.cc:385] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.759845 12262 raft_consensus.cc:740] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6afa6b902e81431297c421d1d4479eb3, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.761670 12262 consensus_queue.cc:260] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.759900 12259 raft_consensus.cc:740] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2603f5a8c64c4d01951548904c8165cf, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.763551 12259 consensus_queue.cc:260] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.767061 12262 ts_tablet_manager.cc:1434] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:26:25.769088 12262 tablet_bootstrap.cc:492] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3: Bootstrap starting.
I20260709 12:26:25.773489 12261 tablet_bootstrap.cc:654] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:25.774255 12259 ts_tablet_manager.cc:1434] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.001s
I20260709 12:26:25.775857 12259 tablet_bootstrap.cc:492] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf: Bootstrap starting.
I20260709 12:26:25.779515 12262 tablet_bootstrap.cc:654] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:25.786542 12259 tablet_bootstrap.cc:654] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:25.796200 12262 tablet_bootstrap.cc:492] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3: No bootstrap required, opened a new log
I20260709 12:26:25.797276 12262 ts_tablet_manager.cc:1403] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.012s
I20260709 12:26:25.801331 12262 raft_consensus.cc:359] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.802441 12262 raft_consensus.cc:385] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.802805 12262 raft_consensus.cc:740] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6afa6b902e81431297c421d1d4479eb3, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.804219 12259 tablet_bootstrap.cc:492] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf: No bootstrap required, opened a new log
I20260709 12:26:25.805218 12259 ts_tablet_manager.cc:1403] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.005s
I20260709 12:26:25.804368 12262 consensus_queue.cc:260] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.808320 12262 ts_tablet_manager.cc:1434] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:26:25.808844 12259 raft_consensus.cc:359] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.810034 12259 raft_consensus.cc:385] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.810385 12259 raft_consensus.cc:740] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2603f5a8c64c4d01951548904c8165cf, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.811442 12259 consensus_queue.cc:260] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.815102 12259 ts_tablet_manager.cc:1434] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:26:25.819665 12261 tablet_bootstrap.cc:492] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d: No bootstrap required, opened a new log
I20260709 12:26:25.820441 12261 ts_tablet_manager.cc:1403] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d: Time spent bootstrapping tablet: real 0.062s	user 0.022s	sys 0.027s
I20260709 12:26:25.825270 12261 raft_consensus.cc:359] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.826305 12261 raft_consensus.cc:385] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:25.826664 12261 raft_consensus.cc:740] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c6b284049b41a5b9446b3e3057914d, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.827498 12261 consensus_queue.cc:260] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.830277 12261 ts_tablet_manager.cc:1434] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 12:26:25.866533 12264 raft_consensus.cc:493] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.867403 12264 raft_consensus.cc:515] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.870723 12264 leader_election.cc:290] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389), 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67:38515)
I20260709 12:26:25.872117 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10090369098e4618b6228fdb03888413" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c6b284049b41a5b9446b3e3057914d" is_pre_election: true
I20260709 12:26:25.872165 12216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10090369098e4618b6228fdb03888413" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afa6b902e81431297c421d1d4479eb3" is_pre_election: true
I20260709 12:26:25.873421 12142 raft_consensus.cc:2468] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 0.
I20260709 12:26:25.873514 12216 raft_consensus.cc:2468] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 0.
I20260709 12:26:25.875401 12030 leader_election.cc:304] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2603f5a8c64c4d01951548904c8165cf, d7c6b284049b41a5b9446b3e3057914d; no voters: 
I20260709 12:26:25.876592 12264 raft_consensus.cc:2804] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:25.877107 12264 raft_consensus.cc:493] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.877379 12264 raft_consensus.cc:3060] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.884797 12264 raft_consensus.cc:515] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.886909 12264 leader_election.cc:290] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 election: Requested vote from peers d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389), 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67:38515)
I20260709 12:26:25.888588 12216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10090369098e4618b6228fdb03888413" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afa6b902e81431297c421d1d4479eb3"
I20260709 12:26:25.888588 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10090369098e4618b6228fdb03888413" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c6b284049b41a5b9446b3e3057914d"
I20260709 12:26:25.890125 12216 raft_consensus.cc:3060] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.890125 12142 raft_consensus.cc:3060] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.893987 12264 raft_consensus.cc:493] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.894842 12264 raft_consensus.cc:515] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.899992 12216 raft_consensus.cc:2468] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 1.
I20260709 12:26:25.900077 12142 raft_consensus.cc:2468] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 1.
I20260709 12:26:25.903046 12141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b23aa41043c41f795dfd248e74e828e" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c6b284049b41a5b9446b3e3057914d" is_pre_election: true
I20260709 12:26:25.904330 12031 leader_election.cc:304] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2603f5a8c64c4d01951548904c8165cf, 6afa6b902e81431297c421d1d4479eb3; no voters: 
I20260709 12:26:25.904570 12264 leader_election.cc:290] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389), 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67:38515)
I20260709 12:26:25.905741 12277 raft_consensus.cc:2804] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:25.906373 12277 raft_consensus.cc:697] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Becoming Leader. State: Replica: 2603f5a8c64c4d01951548904c8165cf, State: Running, Role: LEADER
I20260709 12:26:25.905145 12216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b23aa41043c41f795dfd248e74e828e" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afa6b902e81431297c421d1d4479eb3" is_pre_election: true
I20260709 12:26:25.904362 12141 raft_consensus.cc:2468] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 0.
I20260709 12:26:25.907552 12277 consensus_queue.cc:237] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.908555 12216 raft_consensus.cc:2468] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 0.
I20260709 12:26:25.910537 12030 leader_election.cc:304] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2603f5a8c64c4d01951548904c8165cf, d7c6b284049b41a5b9446b3e3057914d; no voters: 
I20260709 12:26:25.912348 12264 raft_consensus.cc:2804] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:25.913015 12264 raft_consensus.cc:493] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.914077 12264 raft_consensus.cc:3060] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.915565 12268 consensus_queue.cc:1048] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:25.926482 12264 raft_consensus.cc:515] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.925063 11963 catalog_manager.cc:5697] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 2603f5a8c64c4d01951548904c8165cf (127.1.123.65). New cstate: current_term: 1 leader_uuid: "2603f5a8c64c4d01951548904c8165cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:25.931499 12216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b23aa41043c41f795dfd248e74e828e" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afa6b902e81431297c421d1d4479eb3"
I20260709 12:26:25.932493 12216 raft_consensus.cc:3060] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.934363 12264 leader_election.cc:290] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 election: Requested vote from peers d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389), 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67:38515)
I20260709 12:26:25.934517 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b23aa41043c41f795dfd248e74e828e" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c6b284049b41a5b9446b3e3057914d"
I20260709 12:26:25.935405 12142 raft_consensus.cc:3060] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:25.944386 12216 raft_consensus.cc:2468] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 1.
I20260709 12:26:25.949512 12031 leader_election.cc:304] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2603f5a8c64c4d01951548904c8165cf, 6afa6b902e81431297c421d1d4479eb3; no voters: 
I20260709 12:26:25.950845 12264 raft_consensus.cc:2804] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:25.951882 12264 raft_consensus.cc:697] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Becoming Leader. State: Replica: 2603f5a8c64c4d01951548904c8165cf, State: Running, Role: LEADER
I20260709 12:26:25.953487 12264 consensus_queue.cc:237] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:25.968149 12142 raft_consensus.cc:2468] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 1.
I20260709 12:26:25.974162 11964 catalog_manager.cc:5697] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 2603f5a8c64c4d01951548904c8165cf (127.1.123.65). New cstate: current_term: 1 leader_uuid: "2603f5a8c64c4d01951548904c8165cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:25.976876 12296 consensus_queue.cc:1048] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:26.006641 12264 consensus_queue.cc:1048] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:26.041345 12264 consensus_queue.cc:1048] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:26.059307 12215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba"
dest_uuid: "6afa6b902e81431297c421d1d4479eb3"
mode: GRACEFUL
new_leader_uuid: "2603f5a8c64c4d01951548904c8165cf"
 from {username='slave'} at 127.0.0.1:51556
I20260709 12:26:26.060415 12215 raft_consensus.cc:606] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 1 LEADER]: Received request to transfer leadership to TS 2603f5a8c64c4d01951548904c8165cf
I20260709 12:26:26.082875 12067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53d829995fac45d4b59556791396d37b"
dest_uuid: "2603f5a8c64c4d01951548904c8165cf"
mode: GRACEFUL
new_leader_uuid: "6afa6b902e81431297c421d1d4479eb3"
 from {username='slave'} at 127.0.0.1:50018
I20260709 12:26:26.083623 12067 raft_consensus.cc:606] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Received request to transfer leadership to TS 6afa6b902e81431297c421d1d4479eb3
I20260709 12:26:26.129062  1517 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:26:26.131201  1517 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:26:26.131760  1517 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:26:26.150251 12067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10090369098e4618b6228fdb03888413"
dest_uuid: "2603f5a8c64c4d01951548904c8165cf"
mode: GRACEFUL
new_leader_uuid: "d7c6b284049b41a5b9446b3e3057914d"
 from {username='slave'} at 127.0.0.1:50024
I20260709 12:26:26.150978 12067 raft_consensus.cc:606] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Received request to transfer leadership to TS d7c6b284049b41a5b9446b3e3057914d
I20260709 12:26:26.152966  1517 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:26:26.365826 12303 consensus_queue.cc:1048] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:26.380352 12277 consensus_queue.cc:1048] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:26.440303 12264 consensus_queue.cc:1048] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:26.443868 12277 raft_consensus.cc:993] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf: : Instructing follower 6afa6b902e81431297c421d1d4479eb3 to start an election
I20260709 12:26:26.444499 12264 raft_consensus.cc:1081] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Signalling peer 6afa6b902e81431297c421d1d4479eb3 to start an election
I20260709 12:26:26.446549 12216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53d829995fac45d4b59556791396d37b"
dest_uuid: "6afa6b902e81431297c421d1d4479eb3"
 from {username='slave'} at 127.0.0.1:51544
I20260709 12:26:26.447470 12216 raft_consensus.cc:493] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:26.447898 12216 raft_consensus.cc:3060] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:26.455354 12277 raft_consensus.cc:993] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf: : Instructing follower d7c6b284049b41a5b9446b3e3057914d to start an election
I20260709 12:26:26.456116 12277 raft_consensus.cc:1081] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Signalling peer d7c6b284049b41a5b9446b3e3057914d to start an election
I20260709 12:26:26.456472 12216 raft_consensus.cc:515] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:26.459573 12141 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10090369098e4618b6228fdb03888413"
dest_uuid: "d7c6b284049b41a5b9446b3e3057914d"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:26.460745 12141 raft_consensus.cc:493] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:26.461205 12141 raft_consensus.cc:3060] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:26.463027 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d829995fac45d4b59556791396d37b" candidate_uuid: "6afa6b902e81431297c421d1d4479eb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2603f5a8c64c4d01951548904c8165cf"
I20260709 12:26:26.463891 12067 raft_consensus.cc:3055] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:26.463706 12140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d829995fac45d4b59556791396d37b" candidate_uuid: "6afa6b902e81431297c421d1d4479eb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7c6b284049b41a5b9446b3e3057914d"
I20260709 12:26:26.464390 12067 raft_consensus.cc:740] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2603f5a8c64c4d01951548904c8165cf, State: Running, Role: LEADER
I20260709 12:26:26.464587 12140 raft_consensus.cc:3060] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:26.465469 12067 consensus_queue.cc:260] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:26.467147 12067 raft_consensus.cc:3060] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:26.469808 12264 consensus_queue.cc:1048] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:26.461246 12216 leader_election.cc:290] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [CANDIDATE]: Term 2 election: Requested vote from peers 2603f5a8c64c4d01951548904c8165cf (127.1.123.65:42417), d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389)
I20260709 12:26:26.477918 12141 raft_consensus.cc:515] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:26.484210 12142 raft_consensus.cc:1240] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 2 FOLLOWER]: Rejecting Update request from peer 2603f5a8c64c4d01951548904c8165cf for earlier term 1. Current term is 2. Ops: []
I20260709 12:26:26.489179 12140 raft_consensus.cc:2468] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6afa6b902e81431297c421d1d4479eb3 in term 2.
I20260709 12:26:26.490823 12141 leader_election.cc:290] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [CANDIDATE]: Term 2 election: Requested vote from peers 2603f5a8c64c4d01951548904c8165cf (127.1.123.65:42417), 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67:38515)
I20260709 12:26:26.495143 12179 leader_election.cc:304] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6afa6b902e81431297c421d1d4479eb3, d7c6b284049b41a5b9446b3e3057914d; no voters: 
I20260709 12:26:26.490805 12067 raft_consensus.cc:2468] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6afa6b902e81431297c421d1d4479eb3 in term 2.
I20260709 12:26:26.498210 12264 consensus_queue.cc:1059] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:26.499719 12264 raft_consensus.cc:3055] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:26.500097 12264 raft_consensus.cc:740] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2603f5a8c64c4d01951548904c8165cf, State: Running, Role: LEADER
I20260709 12:26:26.498266 12296 raft_consensus.cc:2804] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:26.501087 12264 consensus_queue.cc:260] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:26.509600 12264 raft_consensus.cc:3060] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:26.517760 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10090369098e4618b6228fdb03888413" candidate_uuid: "d7c6b284049b41a5b9446b3e3057914d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2603f5a8c64c4d01951548904c8165cf"
I20260709 12:26:26.518302 12296 raft_consensus.cc:697] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 2 LEADER]: Becoming Leader. State: Replica: 6afa6b902e81431297c421d1d4479eb3, State: Running, Role: LEADER
I20260709 12:26:26.520238 12296 consensus_queue.cc:237] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:26.529150 12215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10090369098e4618b6228fdb03888413" candidate_uuid: "d7c6b284049b41a5b9446b3e3057914d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6afa6b902e81431297c421d1d4479eb3"
I20260709 12:26:26.530112 12215 raft_consensus.cc:3060] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:26.535982 12067 raft_consensus.cc:2468] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7c6b284049b41a5b9446b3e3057914d in term 2.
I20260709 12:26:26.535367 11964 catalog_manager.cc:5697] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 reported cstate change: term changed from 1 to 2, leader changed from 2603f5a8c64c4d01951548904c8165cf (127.1.123.65) to 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "6afa6b902e81431297c421d1d4479eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:26.538450 12108 leader_election.cc:304] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2603f5a8c64c4d01951548904c8165cf, d7c6b284049b41a5b9446b3e3057914d; no voters: 
I20260709 12:26:26.539594 12266 raft_consensus.cc:2804] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:26.541415 12215 raft_consensus.cc:2468] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7c6b284049b41a5b9446b3e3057914d in term 2.
I20260709 12:26:26.546419 12313 raft_consensus.cc:993] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3: : Instructing follower 2603f5a8c64c4d01951548904c8165cf to start an election
I20260709 12:26:26.547050 12296 raft_consensus.cc:1081] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 1 LEADER]: Signalling peer 2603f5a8c64c4d01951548904c8165cf to start an election
I20260709 12:26:26.549783 12067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba"
dest_uuid: "2603f5a8c64c4d01951548904c8165cf"
 from {username='slave'} at 127.0.0.1:50016
I20260709 12:26:26.550451 12067 raft_consensus.cc:493] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:26.550829 12067 raft_consensus.cc:3060] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:26.560621 12067 raft_consensus.cc:515] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:26.563699 12266 raft_consensus.cc:697] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 2 LEADER]: Becoming Leader. State: Replica: d7c6b284049b41a5b9446b3e3057914d, State: Running, Role: LEADER
I20260709 12:26:26.564702 12067 leader_election.cc:290] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 2 election: Requested vote from peers d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389), 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67:38515)
I20260709 12:26:26.564599 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7c6b284049b41a5b9446b3e3057914d"
I20260709 12:26:26.565677 12142 raft_consensus.cc:3060] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:26.571192 12215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb20be30d73d48fe8188662d226c28ba" candidate_uuid: "2603f5a8c64c4d01951548904c8165cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6afa6b902e81431297c421d1d4479eb3"
I20260709 12:26:26.572234 12215 raft_consensus.cc:3055] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:26.572609 12215 raft_consensus.cc:740] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6afa6b902e81431297c421d1d4479eb3, State: Running, Role: LEADER
I20260709 12:26:26.573439 12215 consensus_queue.cc:260] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:26.566920 12266 consensus_queue.cc:237] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:26.575415 12215 raft_consensus.cc:3060] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:26.581736 12142 raft_consensus.cc:2468] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 2.
I20260709 12:26:26.583729 12030 leader_election.cc:304] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2603f5a8c64c4d01951548904c8165cf, d7c6b284049b41a5b9446b3e3057914d; no voters: 
I20260709 12:26:26.585134 12264 raft_consensus.cc:2804] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:26.586001 12264 raft_consensus.cc:697] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 2 LEADER]: Becoming Leader. State: Replica: 2603f5a8c64c4d01951548904c8165cf, State: Running, Role: LEADER
I20260709 12:26:26.588557 12264 consensus_queue.cc:237] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } }
I20260709 12:26:26.599192 12215 raft_consensus.cc:2468] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2603f5a8c64c4d01951548904c8165cf in term 2.
I20260709 12:26:26.606690 11961 catalog_manager.cc:5697] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf reported cstate change: term changed from 1 to 2, leader changed from 6afa6b902e81431297c421d1d4479eb3 (127.1.123.67) to 2603f5a8c64c4d01951548904c8165cf (127.1.123.65). New cstate: current_term: 2 leader_uuid: "2603f5a8c64c4d01951548904c8165cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:26.609380 11963 catalog_manager.cc:5697] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d reported cstate change: term changed from 1 to 2, leader changed from 2603f5a8c64c4d01951548904c8165cf (127.1.123.65) to d7c6b284049b41a5b9446b3e3057914d (127.1.123.66). New cstate: current_term: 2 leader_uuid: "d7c6b284049b41a5b9446b3e3057914d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2603f5a8c64c4d01951548904c8165cf" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 42417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7c6b284049b41a5b9446b3e3057914d" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 36389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6afa6b902e81431297c421d1d4479eb3" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:26.849656  1517 tablet_server.cc:179] TabletServer@127.1.123.65:0 shutting down...
I20260709 12:26:26.895881  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:26.897051  1517 tablet_replica.cc:333] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf: stopping tablet replica
I20260709 12:26:26.897873  1517 raft_consensus.cc:2243] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:26.899013  1517 raft_consensus.cc:2272] T 5b23aa41043c41f795dfd248e74e828e P 2603f5a8c64c4d01951548904c8165cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:26.902766  1517 tablet_replica.cc:333] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf: stopping tablet replica
I20260709 12:26:26.903453  1517 raft_consensus.cc:2243] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:26.903898  1517 raft_consensus.cc:2272] T 53d829995fac45d4b59556791396d37b P 2603f5a8c64c4d01951548904c8165cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:26.907610  1517 tablet_replica.cc:333] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf: stopping tablet replica
I20260709 12:26:26.908381  1517 raft_consensus.cc:2243] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:26.908910  1517 raft_consensus.cc:2272] T 10090369098e4618b6228fdb03888413 P 2603f5a8c64c4d01951548904c8165cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:26.911978  1517 tablet_replica.cc:333] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf: stopping tablet replica
I20260709 12:26:26.912617  1517 raft_consensus.cc:2243] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:26.914381  1517 pending_rounds.cc:70] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf: Trying to abort 1 pending ops.
I20260709 12:26:26.915041  1517 pending_rounds.cc:77] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305625545081335808 op_type: NO_OP noop_request { }
I20260709 12:26:26.915704  1517 raft_consensus.cc:2889] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:26:26.916731  1517 raft_consensus.cc:2272] T cb20be30d73d48fe8188662d226c28ba P 2603f5a8c64c4d01951548904c8165cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:26.952898  1517 tablet_server.cc:196] TabletServer@127.1.123.65:0 shutdown complete.
I20260709 12:26:26.975020  1517 tablet_server.cc:179] TabletServer@127.1.123.66:0 shutting down...
I20260709 12:26:27.015792  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:27.016834  1517 tablet_replica.cc:333] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d: stopping tablet replica
I20260709 12:26:27.017670  1517 raft_consensus.cc:2243] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.018167  1517 raft_consensus.cc:2272] T 5b23aa41043c41f795dfd248e74e828e P d7c6b284049b41a5b9446b3e3057914d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.022192  1517 tablet_replica.cc:333] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d: stopping tablet replica
I20260709 12:26:27.022909  1517 raft_consensus.cc:2243] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.025382  1517 raft_consensus.cc:2272] T 53d829995fac45d4b59556791396d37b P d7c6b284049b41a5b9446b3e3057914d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.028936  1517 tablet_replica.cc:333] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d: stopping tablet replica
I20260709 12:26:27.030288  1517 raft_consensus.cc:2243] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.031786  1517 pending_rounds.cc:70] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d: Trying to abort 2 pending ops.
I20260709 12:26:27.032186  1517 pending_rounds.cc:77] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305625545019281408 op_type: NO_OP noop_request { }
I20260709 12:26:27.033277  1517 raft_consensus.cc:2889] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:26:27.034274  1517 pending_rounds.cc:77] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305625542293901312 op_type: NO_OP noop_request { }
I20260709 12:26:27.034759  1517 raft_consensus.cc:2889] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:26:27.035045  1517 raft_consensus.cc:2272] T 10090369098e4618b6228fdb03888413 P d7c6b284049b41a5b9446b3e3057914d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.041051  1517 tablet_replica.cc:333] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d: stopping tablet replica
I20260709 12:26:27.042439  1517 raft_consensus.cc:2243] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.043365  1517 raft_consensus.cc:2272] T cb20be30d73d48fe8188662d226c28ba P d7c6b284049b41a5b9446b3e3057914d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:27.050860 12182 consensus_peers.cc:597] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 -> Peer 2603f5a8c64c4d01951548904c8165cf (127.1.123.65:42417): Couldn't send request to peer 2603f5a8c64c4d01951548904c8165cf. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:42417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:27.061518 12179 connection.cc:573] client connection to 127.1.123.66:36389 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:26:27.062319 12179 consensus_peers.cc:597] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 -> Peer d7c6b284049b41a5b9446b3e3057914d (127.1.123.66:36389): Couldn't send request to peer d7c6b284049b41a5b9446b3e3057914d. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:27.079540  1517 tablet_server.cc:196] TabletServer@127.1.123.66:0 shutdown complete.
I20260709 12:26:27.101444  1517 tablet_server.cc:179] TabletServer@127.1.123.67:0 shutting down...
I20260709 12:26:27.152670  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:27.154006  1517 tablet_replica.cc:333] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3: stopping tablet replica
I20260709 12:26:27.155423  1517 raft_consensus.cc:2243] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.156376  1517 raft_consensus.cc:2272] T 5b23aa41043c41f795dfd248e74e828e P 6afa6b902e81431297c421d1d4479eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.161092  1517 tablet_replica.cc:333] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3: stopping tablet replica
I20260709 12:26:27.162078  1517 raft_consensus.cc:2243] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.163144  1517 pending_rounds.cc:70] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3: Trying to abort 1 pending ops.
I20260709 12:26:27.163364  1517 pending_rounds.cc:77] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305625544800051200 op_type: NO_OP noop_request { }
I20260709 12:26:27.163728  1517 raft_consensus.cc:2889] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:26:27.164007  1517 raft_consensus.cc:2272] T 53d829995fac45d4b59556791396d37b P 6afa6b902e81431297c421d1d4479eb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.167896  1517 tablet_replica.cc:333] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3: stopping tablet replica
I20260709 12:26:27.169312  1517 raft_consensus.cc:2243] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.170063  1517 raft_consensus.cc:2272] T 10090369098e4618b6228fdb03888413 P 6afa6b902e81431297c421d1d4479eb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.173362  1517 tablet_replica.cc:333] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3: stopping tablet replica
I20260709 12:26:27.174103  1517 raft_consensus.cc:2243] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.174636  1517 raft_consensus.cc:2272] T cb20be30d73d48fe8188662d226c28ba P 6afa6b902e81431297c421d1d4479eb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.207195  1517 tablet_server.cc:196] TabletServer@127.1.123.67:0 shutdown complete.
I20260709 12:26:27.232095  1517 master.cc:562] Master@127.1.123.126:33463 shutting down...
I20260709 12:26:27.277333  1517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.278204  1517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.278592  1517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 48fbbb944fd243329e221c93b5d9437a: stopping tablet replica
I20260709 12:26:27.306571  1517 master.cc:584] Master@127.1.123.126:33463 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3379 ms)
[----------] 6 tests from LeaderRebalancerTest (360730 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:26:27.351783  1517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.123.126:43647
I20260709 12:26:27.353839  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:27.362017 12322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:27.363610 12323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:27.365126 12325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:27.365361  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:27.367810  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:27.368077  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:27.368201  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599987368189 us; error 0 us; skew 500 ppm
I20260709 12:26:27.369254  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:27.374621  1517 webserver.cc:533] Webserver started at http://127.1.123.126:45077/ using document root <none> and password file <none>
I20260709 12:26:27.375540  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:27.375766  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:27.376019  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:27.378026  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/master-0-root/instance:
uuid: "f46c096e14be41819c4abf3bda382805"
format_stamp: "Formatted at 2026-07-09 12:26:27 on dist-test-slave-08l4"
I20260709 12:26:27.387885  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:26:27.394342 12330 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:27.395654  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:26:27.396200  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/master-0-root
uuid: "f46c096e14be41819c4abf3bda382805"
format_stamp: "Formatted at 2026-07-09 12:26:27 on dist-test-slave-08l4"
I20260709 12:26:27.396657  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:27.420043  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:27.422559  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:27.487506  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.126:43647
I20260709 12:26:27.487636 12381 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.126:43647 every 8 connection(s)
I20260709 12:26:27.492849 12382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:27.559221 12382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805: Bootstrap starting.
I20260709 12:26:27.573433 12382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:27.587431 12382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805: No bootstrap required, opened a new log
I20260709 12:26:27.594023 12382 raft_consensus.cc:359] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46c096e14be41819c4abf3bda382805" member_type: VOTER }
I20260709 12:26:27.594707 12382 raft_consensus.cc:385] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:27.595052 12382 raft_consensus.cc:740] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46c096e14be41819c4abf3bda382805, State: Initialized, Role: FOLLOWER
I20260709 12:26:27.595981 12382 consensus_queue.cc:260] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46c096e14be41819c4abf3bda382805" member_type: VOTER }
I20260709 12:26:27.596650 12382 raft_consensus.cc:399] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:26:27.596906 12382 raft_consensus.cc:493] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:26:27.598210 12382 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:27.612128 12382 raft_consensus.cc:515] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46c096e14be41819c4abf3bda382805" member_type: VOTER }
I20260709 12:26:27.615398 12382 leader_election.cc:304] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f46c096e14be41819c4abf3bda382805; no voters: 
I20260709 12:26:27.619725 12382 leader_election.cc:290] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:26:27.620409 12386 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:27.622671 12386 raft_consensus.cc:697] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [term 1 LEADER]: Becoming Leader. State: Replica: f46c096e14be41819c4abf3bda382805, State: Running, Role: LEADER
I20260709 12:26:27.627314 12386 consensus_queue.cc:237] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46c096e14be41819c4abf3bda382805" member_type: VOTER }
I20260709 12:26:27.634056 12382 sys_catalog.cc:565] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:26:27.635370 12388 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f46c096e14be41819c4abf3bda382805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46c096e14be41819c4abf3bda382805" member_type: VOTER } }
I20260709 12:26:27.635551 12387 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f46c096e14be41819c4abf3bda382805. Latest consensus state: current_term: 1 leader_uuid: "f46c096e14be41819c4abf3bda382805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46c096e14be41819c4abf3bda382805" member_type: VOTER } }
I20260709 12:26:27.636271 12388 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [sys.catalog]: This master's current role is: LEADER
I20260709 12:26:27.636464 12387 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [sys.catalog]: This master's current role is: LEADER
I20260709 12:26:27.644480 12390 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:26:27.654469 12390 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:26:27.663923  1517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:26:27.676664 12390 catalog_manager.cc:1383] Generated new cluster ID: aff2be44dfe449839fb41a4aeadee8d9
I20260709 12:26:27.677068 12390 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:26:27.720098 12390 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:26:27.722078 12390 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:26:27.748852 12390 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805: Generated new TSK 0
I20260709 12:26:27.749915 12390 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:26:27.797295  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:27.812079 12404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:27.812788 12405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:27.815936 12407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:27.818404  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:27.821111  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:27.821488  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:27.821743  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599987821723 us; error 0 us; skew 500 ppm
I20260709 12:26:27.822465  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:27.827723  1517 webserver.cc:533] Webserver started at http://127.1.123.65:34931/ using document root <none> and password file <none>
I20260709 12:26:27.828444  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:27.828673  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:27.828989  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:27.831270  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-0-root/instance:
uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
format_stamp: "Formatted at 2026-07-09 12:26:27 on dist-test-slave-08l4"
I20260709 12:26:27.839025  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:26:27.845279 12412 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:27.847313  1517 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:26:27.847851  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-0-root
uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
format_stamp: "Formatted at 2026-07-09 12:26:27 on dist-test-slave-08l4"
I20260709 12:26:27.848258  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:27.868971  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:27.870927  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:27.873409  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:27.877835  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:27.878149  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:27.878476  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:27.878700  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:27.959578  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.65:41231
I20260709 12:26:27.959757 12474 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.65:41231 every 8 connection(s)
I20260709 12:26:27.967921  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:27.981338 12479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:27.982643 12480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:27.986405 12482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:27.990969  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:27.993009  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:27.993408  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:27.993686  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599987993659 us; error 0 us; skew 500 ppm
I20260709 12:26:27.994910  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:28.001379  1517 webserver.cc:533] Webserver started at http://127.1.123.66:38815/ using document root <none> and password file <none>
I20260709 12:26:28.001953 12475 heartbeater.cc:344] Connected to a master server at 127.1.123.126:43647
I20260709 12:26:28.002117  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:28.002580  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:28.002954  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:28.002965 12475 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:28.004187 12475 heartbeater.cc:507] Master 127.1.123.126:43647 requested a full tablet report, sending...
I20260709 12:26:28.005079  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-1-root/instance:
uuid: "8335d0003eb04bb89225bc768e89326f"
format_stamp: "Formatted at 2026-07-09 12:26:28 on dist-test-slave-08l4"
I20260709 12:26:28.007899 12347 ts_manager.cc:194] Registered new tserver with Master: dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231)
I20260709 12:26:28.010408 12347 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48692
I20260709 12:26:28.012895  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:26:28.020327 12487 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:28.021925  1517 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:26:28.022522  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-1-root
uuid: "8335d0003eb04bb89225bc768e89326f"
format_stamp: "Formatted at 2026-07-09 12:26:28 on dist-test-slave-08l4"
I20260709 12:26:28.022943  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:28.051085  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:28.053009  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:28.055665  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:28.059938  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:28.060382  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:28.060820  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:28.061005  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:28.142697  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.66:41219
I20260709 12:26:28.143076 12549 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.66:41219 every 8 connection(s)
I20260709 12:26:28.157449  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:28.176287 12553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:28.178694 12554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:28.182706 12556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:28.183768  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:28.185424  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:26:28.185596 12550 heartbeater.cc:344] Connected to a master server at 127.1.123.126:43647
W20260709 12:26:28.185801  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:28.186050  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783599988186030 us; error 0 us; skew 500 ppm
I20260709 12:26:28.186070 12550 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:28.187129  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:28.187333 12550 heartbeater.cc:507] Master 127.1.123.126:43647 requested a full tablet report, sending...
I20260709 12:26:28.191249 12347 ts_manager.cc:194] Registered new tserver with Master: 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:28.191921  1517 webserver.cc:533] Webserver started at http://127.1.123.67:46035/ using document root <none> and password file <none>
I20260709 12:26:28.192595  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:28.192826  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:28.193135  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:28.194062 12347 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48706
I20260709 12:26:28.194669  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-2-root/instance:
uuid: "e63758d19a9946fbb92b90abc13ba7e5"
format_stamp: "Formatted at 2026-07-09 12:26:28 on dist-test-slave-08l4"
I20260709 12:26:28.203671  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.005s	sys 0.004s
I20260709 12:26:28.209159 12561 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:28.210871  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:26:28.211364  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-2-root
uuid: "e63758d19a9946fbb92b90abc13ba7e5"
format_stamp: "Formatted at 2026-07-09 12:26:28 on dist-test-slave-08l4"
I20260709 12:26:28.211694  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599626433458-1517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:28.257189  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:28.258818  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:28.260282  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:28.262835  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:28.263054  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:28.263352  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:28.263573  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:28.327112  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.67:45359
I20260709 12:26:28.327445 12623 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.67:45359 every 8 connection(s)
I20260709 12:26:28.346405 12624 heartbeater.cc:344] Connected to a master server at 127.1.123.126:43647
I20260709 12:26:28.347301 12624 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:28.348528 12624 heartbeater.cc:507] Master 127.1.123.126:43647 requested a full tablet report, sending...
I20260709 12:26:28.352319 12347 ts_manager.cc:194] Registered new tserver with Master: e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:28.352938  1517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.02161041s
I20260709 12:26:28.354349 12347 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48716
I20260709 12:26:28.399673 12347 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48730:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:26:28.403649 12347 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:26:28.529863 12440 tablet_service.cc:1511] Processing CreateTablet for tablet f32c1c4f8e5a4f4d9314d263fbb06e31 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:28.531953 12440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32c1c4f8e5a4f4d9314d263fbb06e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.536482 12439 tablet_service.cc:1511] Processing CreateTablet for tablet 64e61e3600734c4ea12ee399fa362a4e (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:28.544808 12439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e61e3600734c4ea12ee399fa362a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.539340 12437 tablet_service.cc:1511] Processing CreateTablet for tablet d01036a8c7c14415a05dfacc65b7b920 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:28.547329 12437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d01036a8c7c14415a05dfacc65b7b920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.542809 12438 tablet_service.cc:1511] Processing CreateTablet for tablet 673ec45ffbab46d3ba81b5680b57e025 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:28.551649 12515 tablet_service.cc:1511] Processing CreateTablet for tablet f32c1c4f8e5a4f4d9314d263fbb06e31 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:28.556502 12515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32c1c4f8e5a4f4d9314d263fbb06e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.555959 12513 tablet_service.cc:1511] Processing CreateTablet for tablet 673ec45ffbab46d3ba81b5680b57e025 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:28.553296 12511 tablet_service.cc:1511] Processing CreateTablet for tablet 3473a20c790f484b970e807625ce16cf (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:28.557929 12513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673ec45ffbab46d3ba81b5680b57e025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.551669 12512 tablet_service.cc:1511] Processing CreateTablet for tablet d01036a8c7c14415a05dfacc65b7b920 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:28.553094 12514 tablet_service.cc:1511] Processing CreateTablet for tablet 64e61e3600734c4ea12ee399fa362a4e (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:28.554849 12510 tablet_service.cc:1511] Processing CreateTablet for tablet eb15df4e9b1e44bd9b2dea30b4e72e66 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:28.561105 12436 tablet_service.cc:1511] Processing CreateTablet for tablet 3473a20c790f484b970e807625ce16cf (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:28.561781 12438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673ec45ffbab46d3ba81b5680b57e025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.566521 12509 tablet_service.cc:1511] Processing CreateTablet for tablet ab58c31975ba419dacc885725f47efc7 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:28.568377 12436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3473a20c790f484b970e807625ce16cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.568146 12435 tablet_service.cc:1511] Processing CreateTablet for tablet eb15df4e9b1e44bd9b2dea30b4e72e66 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:28.569959 12511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3473a20c790f484b970e807625ce16cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.572047 12509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab58c31975ba419dacc885725f47efc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.572870 12435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb15df4e9b1e44bd9b2dea30b4e72e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.572934 12510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb15df4e9b1e44bd9b2dea30b4e72e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.578933 12508 tablet_service.cc:1511] Processing CreateTablet for tablet ba28428998c24aee8321c9ecc9ce919a (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:28.588294 12507 tablet_service.cc:1511] Processing CreateTablet for tablet 668bfcf0c9b54599b87a77fab1322ee1 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:28.592770 12434 tablet_service.cc:1511] Processing CreateTablet for tablet ab58c31975ba419dacc885725f47efc7 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:28.594699 12434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab58c31975ba419dacc885725f47efc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.596347 12514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e61e3600734c4ea12ee399fa362a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.599625 12588 tablet_service.cc:1511] Processing CreateTablet for tablet 64e61e3600734c4ea12ee399fa362a4e (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:28.603863 12586 tablet_service.cc:1511] Processing CreateTablet for tablet d01036a8c7c14415a05dfacc65b7b920 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:28.607123 12589 tablet_service.cc:1511] Processing CreateTablet for tablet f32c1c4f8e5a4f4d9314d263fbb06e31 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:28.612088 12585 tablet_service.cc:1511] Processing CreateTablet for tablet 3473a20c790f484b970e807625ce16cf (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:28.604763 12587 tablet_service.cc:1511] Processing CreateTablet for tablet 673ec45ffbab46d3ba81b5680b57e025 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:28.615495 12508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba28428998c24aee8321c9ecc9ce919a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.615002 12433 tablet_service.cc:1511] Processing CreateTablet for tablet ba28428998c24aee8321c9ecc9ce919a (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:28.617672 12433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba28428998c24aee8321c9ecc9ce919a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.622942 12512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d01036a8c7c14415a05dfacc65b7b920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.616925 12584 tablet_service.cc:1511] Processing CreateTablet for tablet eb15df4e9b1e44bd9b2dea30b4e72e66 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:28.625121 12507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 668bfcf0c9b54599b87a77fab1322ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.621062 12583 tablet_service.cc:1511] Processing CreateTablet for tablet ab58c31975ba419dacc885725f47efc7 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:28.627923 12432 tablet_service.cc:1511] Processing CreateTablet for tablet 668bfcf0c9b54599b87a77fab1322ee1 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:28.630473 12432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 668bfcf0c9b54599b87a77fab1322ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.631435 12582 tablet_service.cc:1511] Processing CreateTablet for tablet ba28428998c24aee8321c9ecc9ce919a (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:28.627948 12588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e61e3600734c4ea12ee399fa362a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.634577 12586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d01036a8c7c14415a05dfacc65b7b920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.637285 12589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32c1c4f8e5a4f4d9314d263fbb06e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.639639 12585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3473a20c790f484b970e807625ce16cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.643652 12582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba28428998c24aee8321c9ecc9ce919a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:26:28.653066 12396 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:28.681713 12584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb15df4e9b1e44bd9b2dea30b4e72e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.687374 12583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab58c31975ba419dacc885725f47efc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.689308 12587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673ec45ffbab46d3ba81b5680b57e025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.701524 12581 tablet_service.cc:1511] Processing CreateTablet for tablet 668bfcf0c9b54599b87a77fab1322ee1 (DEFAULT_TABLE table=test-workload [id=2e1739cc6969470390d767d90871f4c1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:28.704154 12581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 668bfcf0c9b54599b87a77fab1322ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:28.786582 12642 tablet_bootstrap.cc:492] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:28.807587 12643 tablet_bootstrap.cc:492] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:28.821929 12644 tablet_bootstrap.cc:492] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:28.842295 12643 tablet_bootstrap.cc:654] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:28.846282 12644 tablet_bootstrap.cc:654] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:28.868047 12642 tablet_bootstrap.cc:654] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:28.923386 12644 tablet_bootstrap.cc:492] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:28.924079 12644 ts_tablet_manager.cc:1403] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.103s	user 0.029s	sys 0.026s
I20260709 12:26:28.925828 12643 tablet_bootstrap.cc:492] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:28.927175 12643 ts_tablet_manager.cc:1403] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.120s	user 0.015s	sys 0.017s
I20260709 12:26:28.930953 12644 raft_consensus.cc:359] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } }
I20260709 12:26:28.938126 12644 raft_consensus.cc:385] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:28.938727 12644 raft_consensus.cc:740] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:28.939793 12644 consensus_queue.cc:260] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } }
I20260709 12:26:28.941187 12642 tablet_bootstrap.cc:492] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:28.943325 12642 ts_tablet_manager.cc:1403] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.157s	user 0.018s	sys 0.017s
I20260709 12:26:28.942274 12643 raft_consensus.cc:359] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:28.943967 12643 raft_consensus.cc:385] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:28.944337 12643 raft_consensus.cc:740] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:28.946039 12643 consensus_queue.cc:260] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:28.947992 12642 raft_consensus.cc:359] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:28.949489 12642 raft_consensus.cc:385] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:28.949963 12642 raft_consensus.cc:740] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:28.951725 12624 heartbeater.cc:499] Master 127.1.123.126:43647 was elected leader, sending a full tablet report...
I20260709 12:26:28.952100 12644 ts_tablet_manager.cc:1434] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.028s	user 0.013s	sys 0.008s
I20260709 12:26:28.953384 12644 tablet_bootstrap.cc:492] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:28.952155 12642 consensus_queue.cc:260] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:28.959236 12550 heartbeater.cc:499] Master 127.1.123.126:43647 was elected leader, sending a full tablet report...
I20260709 12:26:28.964807 12642 ts_tablet_manager.cc:1434] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 12:26:28.965943 12475 heartbeater.cc:499] Master 127.1.123.126:43647 was elected leader, sending a full tablet report...
I20260709 12:26:28.966295 12644 tablet_bootstrap.cc:654] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:28.975582 12643 ts_tablet_manager.cc:1434] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.048s	user 0.011s	sys 0.010s
I20260709 12:26:28.978147 12643 tablet_bootstrap.cc:492] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:28.979174 12642 tablet_bootstrap.cc:492] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:28.989673 12643 tablet_bootstrap.cc:654] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:28.994179 12650 raft_consensus.cc:493] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.995491 12642 tablet_bootstrap.cc:654] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:28.995057 12650 raft_consensus.cc:515] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:28.997030 12644 tablet_bootstrap.cc:492] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:28.998852 12644 ts_tablet_manager.cc:1403] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.029s
I20260709 12:26:29.004057 12644 raft_consensus.cc:359] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.005329 12644 raft_consensus.cc:385] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.005806 12644 raft_consensus.cc:740] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.006800 12650 leader_election.cc:290] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:29.006918 12644 consensus_queue.cc:260] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.010844 12644 ts_tablet_manager.cc:1434] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.005s
I20260709 12:26:29.012481 12644 tablet_bootstrap.cc:492] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:29.022828 12644 tablet_bootstrap.cc:654] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.037716 12644 tablet_bootstrap.cc:492] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:29.038348 12644 ts_tablet_manager.cc:1403] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.008s
I20260709 12:26:29.042891 12644 raft_consensus.cc:359] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.044817 12644 raft_consensus.cc:385] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.045221 12644 raft_consensus.cc:740] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.047071 12644 consensus_queue.cc:260] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.051826 12644 ts_tablet_manager.cc:1434] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 12:26:29.053797 12644 tablet_bootstrap.cc:492] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:29.063545 12644 tablet_bootstrap.cc:654] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.079630 12644 tablet_bootstrap.cc:492] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:29.080240 12644 ts_tablet_manager.cc:1403] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.006s
I20260709 12:26:29.087352 12644 raft_consensus.cc:359] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.088403 12644 raft_consensus.cc:385] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.088728 12644 raft_consensus.cc:740] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.089783 12644 consensus_queue.cc:260] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.096894 12644 ts_tablet_manager.cc:1434] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:26:29.099527 12644 tablet_bootstrap.cc:492] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:29.099165 12649 raft_consensus.cc:493] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.101048 12649 raft_consensus.cc:515] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.108445 12649 leader_election.cc:290] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:29.123035 12648 raft_consensus.cc:493] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.124485 12648 raft_consensus.cc:515] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.126374 12642 tablet_bootstrap.cc:492] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:29.127669 12642 ts_tablet_manager.cc:1403] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.149s	user 0.045s	sys 0.060s
I20260709 12:26:29.126359 12644 tablet_bootstrap.cc:654] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.133909 12642 raft_consensus.cc:359] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.143312 12648 leader_election.cc:290] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:29.160993 12643 tablet_bootstrap.cc:492] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:29.161851 12643 ts_tablet_manager.cc:1403] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.184s	user 0.051s	sys 0.066s
I20260709 12:26:29.166961 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3473a20c790f484b970e807625ce16cf" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:29.168287 12599 raft_consensus.cc:2468] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 0.
I20260709 12:26:29.147861 12642 raft_consensus.cc:385] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.170676 12642 raft_consensus.cc:740] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.171756 12642 consensus_queue.cc:260] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.175299 12414 leader_election.cc:304] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:29.182929 12650 raft_consensus.cc:2804] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:29.183728 12650 raft_consensus.cc:493] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.184338 12650 raft_consensus.cc:3060] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.186018 12643 raft_consensus.cc:359] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.187603 12643 raft_consensus.cc:385] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.187947 12643 raft_consensus.cc:740] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.191561 12642 ts_tablet_manager.cc:1434] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.063s	user 0.012s	sys 0.001s
I20260709 12:26:29.214562 12642 tablet_bootstrap.cc:492] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:29.228137 12648 raft_consensus.cc:493] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.230000 12648 raft_consensus.cc:515] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } }
I20260709 12:26:29.232993 12648 leader_election.cc:290] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:29.245649 12650 raft_consensus.cc:515] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.251134 12648 raft_consensus.cc:493] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.251924 12648 raft_consensus.cc:515] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.255260 12648 leader_election.cc:290] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:29.257134 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3473a20c790f484b970e807625ce16cf" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:29.223796 12643 consensus_queue.cc:260] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.258662 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3473a20c790f484b970e807625ce16cf" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:29.262702 12650 leader_election.cc:290] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Requested vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
W20260709 12:26:29.262959 12414 leader_election.cc:343] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.261468 12649 raft_consensus.cc:493] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.266426 12649 raft_consensus.cc:515] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.269856 12644 tablet_bootstrap.cc:492] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:29.271023 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3473a20c790f484b970e807625ce16cf" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:29.274935 12642 tablet_bootstrap.cc:654] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.278957 12649 leader_election.cc:290] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:29.272233 12644 ts_tablet_manager.cc:1403] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.173s	user 0.027s	sys 0.051s
I20260709 12:26:29.284230 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32c1c4f8e5a4f4d9314d263fbb06e31" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:29.289765 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:29.289451 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32c1c4f8e5a4f4d9314d263fbb06e31" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:29.290737 12524 raft_consensus.cc:2468] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
I20260709 12:26:29.274960 12599 raft_consensus.cc:3060] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:26:29.291996 12563 leader_election.cc:343] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.283120 12450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:29.289326 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "668bfcf0c9b54599b87a77fab1322ee1" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
W20260709 12:26:29.294421 12563 leader_election.cc:343] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:26:29.295372 12563 leader_election.cc:343] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:26:29.273787 12414 leader_election.cc:343] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.283250 12448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "668bfcf0c9b54599b87a77fab1322ee1" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:29.298959 12450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba28428998c24aee8321c9ecc9ce919a" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:29.300673 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:29.303143 12644 raft_consensus.cc:359] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.295893 12563 leader_election.cc:304] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e63758d19a9946fbb92b90abc13ba7e5; no voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5
W20260709 12:26:29.305336 12489 leader_election.cc:343] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.310706 12648 raft_consensus.cc:2749] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:29.311115 12563 leader_election.cc:304] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:29.304056 12644 raft_consensus.cc:385] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.319268 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba28428998c24aee8321c9ecc9ce919a" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:29.319978 12599 raft_consensus.cc:2468] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 1.
I20260709 12:26:29.319605 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
W20260709 12:26:29.320722 12489 leader_election.cc:343] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.316342 12648 raft_consensus.cc:2804] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:29.316323 12644 raft_consensus.cc:740] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.322582 12648 raft_consensus.cc:493] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.322820 12414 leader_election.cc:304] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 8335d0003eb04bb89225bc768e89326f
I20260709 12:26:29.324438 12650 raft_consensus.cc:2804] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:29.324119 12644 consensus_queue.cc:260] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.328104 12598 raft_consensus.cc:2468] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 0.
I20260709 12:26:29.327414 12648 raft_consensus.cc:3060] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.336179 12489 leader_election.cc:304] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:29.340924 12666 raft_consensus.cc:2804] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:29.342098 12666 raft_consensus.cc:493] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.342921 12666 raft_consensus.cc:3060] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.342928 12650 raft_consensus.cc:697] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Becoming Leader. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:29.344856 12650 consensus_queue.cc:237] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.347788 12643 ts_tablet_manager.cc:1434] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.186s	user 0.018s	sys 0.014s
I20260709 12:26:29.349215 12642 tablet_bootstrap.cc:492] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:29.351972 12642 ts_tablet_manager.cc:1403] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.153s	user 0.025s	sys 0.029s
I20260709 12:26:29.368949 12648 raft_consensus.cc:515] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
W20260709 12:26:29.371376 12563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:41219, user_credentials={real_user=slave}} blocked reactor thread for 60396us
I20260709 12:26:29.372092 12597 raft_consensus.cc:2393] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8335d0003eb04bb89225bc768e89326f in current term 1: Already voted for candidate e63758d19a9946fbb92b90abc13ba7e5 in this term.
I20260709 12:26:29.354895 12644 ts_tablet_manager.cc:1434] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.056s	user 0.017s	sys 0.000s
I20260709 12:26:29.373862 12644 tablet_bootstrap.cc:492] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:29.378376 12643 tablet_bootstrap.cc:492] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
W20260709 12:26:29.372416 12563 leader_election.cc:343] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.381338 12489 leader_election.cc:304] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5
W20260709 12:26:29.382790 12563 leader_election.cc:343] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.383925 12563 leader_election.cc:304] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e63758d19a9946fbb92b90abc13ba7e5; no voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:29.384094 12649 raft_consensus.cc:3060] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.385339 12642 raft_consensus.cc:359] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.386076 12668 raft_consensus.cc:2749] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:29.393661 12642 raft_consensus.cc:385] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.398901 12642 raft_consensus.cc:740] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.399328 12648 leader_election.cc:290] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:29.400200 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:29.400166 12642 consensus_queue.cc:260] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.402714 12345 catalog_manager.cc:5697] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:29.403644 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:29.404281 12642 ts_tablet_manager.cc:1434] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.037s	user 0.012s	sys 0.000s
W20260709 12:26:29.404558 12563 leader_election.cc:343] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.405691 12642 tablet_bootstrap.cc:492] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:29.405504 12649 raft_consensus.cc:2749] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:29.411846 12643 tablet_bootstrap.cc:654] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.414286 12666 raft_consensus.cc:515] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.421424 12644 tablet_bootstrap.cc:654] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.420758 12642 tablet_bootstrap.cc:654] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.431811 12643 tablet_bootstrap.cc:492] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:29.432900 12643 ts_tablet_manager.cc:1403] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.006s
I20260709 12:26:29.435845 12666 leader_election.cc:290] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:29.439747 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba28428998c24aee8321c9ecc9ce919a" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:29.445623 12449 raft_consensus.cc:3060] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.449734 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba28428998c24aee8321c9ecc9ce919a" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:29.451026 12597 raft_consensus.cc:3060] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.458550 12643 raft_consensus.cc:359] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.468223 12642 tablet_bootstrap.cc:492] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:29.468223 12644 tablet_bootstrap.cc:492] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:29.469211 12642 ts_tablet_manager.cc:1403] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.064s	user 0.019s	sys 0.015s
I20260709 12:26:29.469228 12644 ts_tablet_manager.cc:1403] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.096s	user 0.022s	sys 0.032s
I20260709 12:26:29.473168 12642 raft_consensus.cc:359] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.473306 12644 raft_consensus.cc:359] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.474117 12642 raft_consensus.cc:385] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.474349 12644 raft_consensus.cc:385] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.474463 12642 raft_consensus.cc:740] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.475554 12644 raft_consensus.cc:740] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.477157 12597 raft_consensus.cc:2468] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 1.
I20260709 12:26:29.477259 12644 consensus_queue.cc:260] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.478482 12523 raft_consensus.cc:2468] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:29.479427 12489 leader_election.cc:304] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:29.480860 12643 raft_consensus.cc:385] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.481426 12643 raft_consensus.cc:740] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.482462 12643 consensus_queue.cc:260] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.483006 12644 ts_tablet_manager.cc:1434] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.002s
I20260709 12:26:29.483291 12563 leader_election.cc:304] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:29.486143 12644 tablet_bootstrap.cc:492] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:29.481315 12666 raft_consensus.cc:2804] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:29.487721 12648 raft_consensus.cc:2804] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:29.489239 12648 raft_consensus.cc:697] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:29.485231 12643 ts_tablet_manager.cc:1434] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.052s	user 0.012s	sys 0.000s
I20260709 12:26:29.490932 12648 consensus_queue.cc:237] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.487792 12666 raft_consensus.cc:697] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 1 LEADER]: Becoming Leader. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Running, Role: LEADER
I20260709 12:26:29.493259 12666 consensus_queue.cc:237] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.498065 12643 tablet_bootstrap.cc:492] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:29.497277 12642 consensus_queue.cc:260] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.506327 12642 ts_tablet_manager.cc:1434] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.001s
I20260709 12:26:29.508044 12644 tablet_bootstrap.cc:654] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.508423 12642 tablet_bootstrap.cc:492] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:29.508137 12449 raft_consensus.cc:2468] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 1.
I20260709 12:26:29.513840 12643 tablet_bootstrap.cc:654] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.521291 12642 tablet_bootstrap.cc:654] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.520953 12346 catalog_manager.cc:5697] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:29.524134 12644 tablet_bootstrap.cc:492] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:29.527971 12644 ts_tablet_manager.cc:1403] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.005s
I20260709 12:26:29.531450 12643 tablet_bootstrap.cc:492] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:29.535866 12642 tablet_bootstrap.cc:492] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:29.536718 12643 ts_tablet_manager.cc:1403] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.009s
I20260709 12:26:29.536957 12642 ts_tablet_manager.cc:1403] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.014s
I20260709 12:26:29.540325 12643 raft_consensus.cc:359] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.541219 12643 raft_consensus.cc:385] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.541545 12643 raft_consensus.cc:740] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.542785 12643 consensus_queue.cc:260] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.546137 12644 raft_consensus.cc:359] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.546691 12643 ts_tablet_manager.cc:1434] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:26:29.547070 12644 raft_consensus.cc:385] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.547781 12644 raft_consensus.cc:740] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.548203 12643 tablet_bootstrap.cc:492] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:29.548908 12644 consensus_queue.cc:260] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.550410 12642 raft_consensus.cc:359] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } }
I20260709 12:26:29.551395 12642 raft_consensus.cc:385] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.551702 12642 raft_consensus.cc:740] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.552688 12644 ts_tablet_manager.cc:1434] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 12:26:29.552654 12642 consensus_queue.cc:260] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } }
I20260709 12:26:29.554049 12644 tablet_bootstrap.cc:492] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:29.555464 12642 ts_tablet_manager.cc:1434] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:26:29.556890 12642 tablet_bootstrap.cc:492] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:29.558869 12648 raft_consensus.cc:493] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.559478 12643 tablet_bootstrap.cc:654] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.559824 12648 raft_consensus.cc:515] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.562943 12644 tablet_bootstrap.cc:654] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.566462 12642 tablet_bootstrap.cc:654] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.568542 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d01036a8c7c14415a05dfacc65b7b920" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:29.569796 12449 raft_consensus.cc:2468] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
I20260709 12:26:29.572654 12563 leader_election.cc:304] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:29.574141 12668 raft_consensus.cc:2804] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:29.575104 12668 raft_consensus.cc:493] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.576027 12668 raft_consensus.cc:3060] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.572654 12648 leader_election.cc:290] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:29.582885 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d01036a8c7c14415a05dfacc65b7b920" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:29.583734 12523 raft_consensus.cc:2468] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
I20260709 12:26:29.589077 12668 raft_consensus.cc:515] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.592365 12668 leader_election.cc:290] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:29.574602 12344 catalog_manager.cc:5697] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8335d0003eb04bb89225bc768e89326f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "8335d0003eb04bb89225bc768e89326f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:29.596486 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d01036a8c7c14415a05dfacc65b7b920" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:29.597793 12449 raft_consensus.cc:3060] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.607954 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d01036a8c7c14415a05dfacc65b7b920" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:29.609359 12523 raft_consensus.cc:3060] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.617244 12644 tablet_bootstrap.cc:492] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:29.618013 12644 ts_tablet_manager.cc:1403] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.064s	user 0.015s	sys 0.012s
I20260709 12:26:29.621150 12642 tablet_bootstrap.cc:492] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:29.622166 12642 ts_tablet_manager.cc:1403] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.066s	user 0.016s	sys 0.008s
I20260709 12:26:29.627770 12644 raft_consensus.cc:359] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.629053 12642 raft_consensus.cc:359] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.629861 12449 raft_consensus.cc:2468] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:29.630395 12642 raft_consensus.cc:385] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.630754 12642 raft_consensus.cc:740] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.631578 12563 leader_election.cc:304] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:29.629053 12644 raft_consensus.cc:385] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.632558 12668 raft_consensus.cc:2804] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:29.633100 12644 raft_consensus.cc:740] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.634907 12668 raft_consensus.cc:697] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:29.635094 12644 consensus_queue.cc:260] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.636971 12668 consensus_queue.cc:237] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.640836 12644 ts_tablet_manager.cc:1434] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.008s
I20260709 12:26:29.642299 12644 tablet_bootstrap.cc:492] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:29.645876 12523 raft_consensus.cc:2468] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:29.645903 12643 tablet_bootstrap.cc:492] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:29.647341 12643 ts_tablet_manager.cc:1403] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.099s	user 0.057s	sys 0.030s
I20260709 12:26:29.647294 12642 consensus_queue.cc:260] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.650298 12644 tablet_bootstrap.cc:654] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.653461 12642 ts_tablet_manager.cc:1434] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.003s
I20260709 12:26:29.653970 12643 raft_consensus.cc:359] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.668156 12643 raft_consensus.cc:385] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.668742 12643 raft_consensus.cc:740] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
W20260709 12:26:29.660666 12396 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:29.674213 12642 tablet_bootstrap.cc:492] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:29.674010 12643 consensus_queue.cc:260] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.683373 12643 ts_tablet_manager.cc:1434] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.005s
I20260709 12:26:29.685084 12643 tablet_bootstrap.cc:492] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:29.685672 12644 tablet_bootstrap.cc:492] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:29.686379 12644 ts_tablet_manager.cc:1403] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.044s	user 0.026s	sys 0.006s
I20260709 12:26:29.689425 12644 raft_consensus.cc:359] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.691220 12642 tablet_bootstrap.cc:654] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.691241 12644 raft_consensus.cc:385] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.692898 12644 raft_consensus.cc:740] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.694825 12643 tablet_bootstrap.cc:654] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.694574 12644 consensus_queue.cc:260] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.696370 12648 raft_consensus.cc:493] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.697281 12648 raft_consensus.cc:515] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.702138 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb15df4e9b1e44bd9b2dea30b4e72e66" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
W20260709 12:26:29.704476 12563 leader_election.cc:343] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.706609 12642 tablet_bootstrap.cc:492] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:29.707693 12642 ts_tablet_manager.cc:1403] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.011s
I20260709 12:26:29.709688 12644 ts_tablet_manager.cc:1434] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.023s	user 0.012s	sys 0.000s
I20260709 12:26:29.710153 12344 catalog_manager.cc:5697] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:29.708379 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb15df4e9b1e44bd9b2dea30b4e72e66" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:29.714111 12642 raft_consensus.cc:359] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.715515 12642 raft_consensus.cc:385] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.716658 12642 raft_consensus.cc:740] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
W20260709 12:26:29.718462 12563 leader_election.cc:343] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.718251 12642 consensus_queue.cc:260] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.719358 12563 leader_election.cc:304] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e63758d19a9946fbb92b90abc13ba7e5; no voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:29.707933 12648 leader_election.cc:290] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:29.723618 12648 raft_consensus.cc:2749] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:29.725024 12650 raft_consensus.cc:493] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.725950 12650 raft_consensus.cc:515] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } }
I20260709 12:26:29.726760 12643 tablet_bootstrap.cc:492] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:29.729619 12650 leader_election.cc:290] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:29.730477 12643 ts_tablet_manager.cc:1403] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.046s	user 0.022s	sys 0.011s
I20260709 12:26:29.731967 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32c1c4f8e5a4f4d9314d263fbb06e31" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:29.732909 12597 raft_consensus.cc:2468] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 0.
I20260709 12:26:29.734323 12650 raft_consensus.cc:493] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.734736 12414 leader_election.cc:304] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:29.735013 12650 raft_consensus.cc:515] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.736397 12669 raft_consensus.cc:2804] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:29.736905 12669 raft_consensus.cc:493] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.737707 12669 raft_consensus.cc:3060] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.740214 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:29.741222 12525 raft_consensus.cc:2393] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in current term 1: Already voted for candidate e63758d19a9946fbb92b90abc13ba7e5 in this term.
I20260709 12:26:29.741475 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:29.744218 12414 leader_election.cc:304] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:29.734380 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32c1c4f8e5a4f4d9314d263fbb06e31" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:29.747228 12650 leader_election.cc:290] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:29.749600 12643 raft_consensus.cc:359] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.756552 12643 raft_consensus.cc:385] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.757046 12643 raft_consensus.cc:740] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.748040 12650 raft_consensus.cc:3060] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.758674 12669 raft_consensus.cc:515] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } }
I20260709 12:26:29.761451 12642 ts_tablet_manager.cc:1434] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.053s	user 0.013s	sys 0.000s
I20260709 12:26:29.763325 12642 tablet_bootstrap.cc:492] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:29.765465 12643 consensus_queue.cc:260] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.770851 12669 leader_election.cc:290] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Requested vote from peers e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:29.768857 12666 raft_consensus.cc:493] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.774045 12666 raft_consensus.cc:515] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.782886 12642 tablet_bootstrap.cc:654] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.782845 12691 raft_consensus.cc:493] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.783797 12691 raft_consensus.cc:515] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.785518 12643 ts_tablet_manager.cc:1434] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.054s	user 0.012s	sys 0.001s
I20260709 12:26:29.788692 12691 leader_election.cc:290] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:29.791682 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:29.791867 12666 leader_election.cc:290] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:29.790799 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:29.791178 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3473a20c790f484b970e807625ce16cf" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:29.792407 12523 raft_consensus.cc:2468] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
I20260709 12:26:29.793401 12449 raft_consensus.cc:2468] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
I20260709 12:26:29.793644 12597 raft_consensus.cc:2393] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8335d0003eb04bb89225bc768e89326f in current term 1: Already voted for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in this term.
I20260709 12:26:29.794869 12563 leader_election.cc:304] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:29.796064 12691 raft_consensus.cc:2804] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:29.796521 12691 raft_consensus.cc:493] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.796926 12691 raft_consensus.cc:3060] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:26:29.798483 12414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:45359, user_credentials={real_user=slave}} blocked reactor thread for 54814.5us
I20260709 12:26:29.801292 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32c1c4f8e5a4f4d9314d263fbb06e31" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:29.802321 12597 raft_consensus.cc:3060] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.802452 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32c1c4f8e5a4f4d9314d263fbb06e31" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:29.804610 12643 tablet_bootstrap.cc:492] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:29.807564 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3473a20c790f484b970e807625ce16cf" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:29.819677 12643 tablet_bootstrap.cc:654] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.826678 12642 tablet_bootstrap.cc:492] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
W20260709 12:26:29.811575 12414 leader_election.cc:343] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.827489 12642 ts_tablet_manager.cc:1403] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.064s	user 0.023s	sys 0.024s
W20260709 12:26:29.828474 12414 leader_election.cc:343] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.830375 12489 leader_election.cc:304] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:29.833695 12666 raft_consensus.cc:3060] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.834640 12642 raft_consensus.cc:359] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.835633 12642 raft_consensus.cc:385] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.792136 12650 raft_consensus.cc:2749] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:29.836549 12642 raft_consensus.cc:740] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.840229 12642 consensus_queue.cc:260] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.844184 12597 raft_consensus.cc:2468] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 1.
I20260709 12:26:29.846647 12691 raft_consensus.cc:515] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.850368 12414 leader_election.cc:304] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 8335d0003eb04bb89225bc768e89326f
I20260709 12:26:29.849498 12691 leader_election.cc:290] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:29.852598 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:29.857856 12650 raft_consensus.cc:2804] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:29.858150 12642 ts_tablet_manager.cc:1434] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.030s	user 0.016s	sys 0.000s
I20260709 12:26:29.858623 12449 raft_consensus.cc:3060] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.858846 12650 raft_consensus.cc:697] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Becoming Leader. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:29.859562 12642 tablet_bootstrap.cc:492] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:29.860096 12650 consensus_queue.cc:237] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } }
I20260709 12:26:29.852994 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:29.852470 12666 raft_consensus.cc:2749] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:29.863296 12692 raft_consensus.cc:493] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.863926 12692 raft_consensus.cc:515] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.867623 12692 leader_election.cc:290] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:29.869309 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e61e3600734c4ea12ee399fa362a4e" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:29.870977 12642 tablet_bootstrap.cc:654] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
W20260709 12:26:29.872738 12414 leader_election.cc:343] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.869863 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e61e3600734c4ea12ee399fa362a4e" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:29.870536 12523 raft_consensus.cc:3060] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.874616 12597 raft_consensus.cc:2468] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 0.
I20260709 12:26:29.876236 12414 leader_election.cc:304] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 8335d0003eb04bb89225bc768e89326f
I20260709 12:26:29.877763 12345 catalog_manager.cc:5697] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:29.883284 12642 tablet_bootstrap.cc:492] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:29.884111 12642 ts_tablet_manager.cc:1403] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.008s
I20260709 12:26:29.888078 12642 raft_consensus.cc:359] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.888708 12669 consensus_queue.cc:1048] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:29.889163 12642 raft_consensus.cc:385] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.857373 12643 tablet_bootstrap.cc:492] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:29.889778 12642 raft_consensus.cc:740] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.890409 12643 ts_tablet_manager.cc:1403] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.086s	user 0.027s	sys 0.003s
I20260709 12:26:29.891042 12642 consensus_queue.cc:260] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.877936 12692 raft_consensus.cc:2804] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:29.894515 12692 raft_consensus.cc:493] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.894773 12642 ts_tablet_manager.cc:1434] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 12:26:29.894944 12692 raft_consensus.cc:3060] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.898131 12649 consensus_queue.cc:1048] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:29.904789 12643 raft_consensus.cc:359] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.906802 12523 raft_consensus.cc:2468] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:29.908015 12691 consensus_queue.cc:1048] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:29.911240 12643 raft_consensus.cc:385] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:29.911664 12643 raft_consensus.cc:740] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:29.912643 12643 consensus_queue.cc:260] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.913144 12563 leader_election.cc:304] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:29.916011 12643 ts_tablet_manager.cc:1434] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 12:26:29.916383 12691 raft_consensus.cc:2804] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:29.917055 12691 raft_consensus.cc:697] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:29.917860 12643 tablet_bootstrap.cc:492] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:29.918524 12691 consensus_queue.cc:237] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.908246 12669 consensus_queue.cc:1048] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:29.910374 12449 raft_consensus.cc:2468] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:29.927950 12643 tablet_bootstrap.cc:654] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:29.945719 12692 raft_consensus.cc:515] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:29.962136 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e61e3600734c4ea12ee399fa362a4e" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:29.965526 12692 leader_election.cc:290] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Requested vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
W20260709 12:26:29.965920 12414 leader_election.cc:343] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:29.968153 12668 consensus_queue.cc:1048] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:26:29.969496 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e61e3600734c4ea12ee399fa362a4e" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:29.972028 12599 raft_consensus.cc:3060] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:29.975859 12649 consensus_queue.cc:1048] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:29.998147 12649 raft_consensus.cc:493] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.999258 12649 raft_consensus.cc:515] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.018324 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb15df4e9b1e44bd9b2dea30b4e72e66" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:30.018360 12599 raft_consensus.cc:2468] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 1.
I20260709 12:26:30.019140 12449 raft_consensus.cc:2468] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 0.
I20260709 12:26:30.028203 12414 leader_election.cc:304] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 8335d0003eb04bb89225bc768e89326f
I20260709 12:26:30.029332 12692 raft_consensus.cc:2804] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:30.021799 12489 leader_election.cc:304] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:30.030671 12666 raft_consensus.cc:2804] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:30.031164 12666 raft_consensus.cc:493] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.031527 12666 raft_consensus.cc:3060] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:30.036437 12649 leader_election.cc:290] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:30.038046 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb15df4e9b1e44bd9b2dea30b4e72e66" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:30.038316 12692 raft_consensus.cc:697] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Becoming Leader. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:30.039232 12599 raft_consensus.cc:2468] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 0.
I20260709 12:26:30.039911 12648 consensus_queue.cc:1048] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:30.040256 12692 consensus_queue.cc:237] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.050668 12643 tablet_bootstrap.cc:492] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:30.051301 12643 ts_tablet_manager.cc:1403] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.134s	user 0.032s	sys 0.024s
I20260709 12:26:30.054500 12643 raft_consensus.cc:359] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.055662 12643 raft_consensus.cc:385] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:30.055981 12643 raft_consensus.cc:740] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:30.056910 12643 consensus_queue.cc:260] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.086089 12643 ts_tablet_manager.cc:1434] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.034s	user 0.013s	sys 0.022s
I20260709 12:26:30.088480 12643 tablet_bootstrap.cc:492] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:30.082958 12344 catalog_manager.cc:5697] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:30.097095 12643 tablet_bootstrap.cc:654] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:30.104941 12666 raft_consensus.cc:515] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.119410 12666 leader_election.cc:290] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:30.127657 12448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb15df4e9b1e44bd9b2dea30b4e72e66" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:30.128609 12448 raft_consensus.cc:3060] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:30.130957 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb15df4e9b1e44bd9b2dea30b4e72e66" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:30.134244 12597 raft_consensus.cc:3060] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:30.151804 12643 tablet_bootstrap.cc:492] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:30.152770 12643 ts_tablet_manager.cc:1403] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.066s	user 0.019s	sys 0.011s
I20260709 12:26:30.160513 12597 raft_consensus.cc:2468] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 1.
I20260709 12:26:30.160796 12448 raft_consensus.cc:2468] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 1.
I20260709 12:26:30.161429 12668 consensus_queue.cc:1048] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:30.162250 12643 raft_consensus.cc:359] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } }
I20260709 12:26:30.162303 12489 leader_election.cc:304] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:30.163376 12643 raft_consensus.cc:385] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:30.163955 12643 raft_consensus.cc:740] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:30.163450 12345 catalog_manager.cc:5697] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:30.164641 12666 raft_consensus.cc:2804] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:30.164844 12643 consensus_queue.cc:260] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } }
I20260709 12:26:30.165774 12666 raft_consensus.cc:697] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 1 LEADER]: Becoming Leader. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Running, Role: LEADER
I20260709 12:26:30.166846 12666 consensus_queue.cc:237] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.168008 12643 ts_tablet_manager.cc:1434] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 12:26:30.193461 12345 catalog_manager.cc:5697] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8335d0003eb04bb89225bc768e89326f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "8335d0003eb04bb89225bc768e89326f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:30.223423 12666 raft_consensus.cc:493] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.224131 12666 raft_consensus.cc:515] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } }
I20260709 12:26:30.226531 12666 leader_election.cc:290] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:30.228277 12448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32c1c4f8e5a4f4d9314d263fbb06e31" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:30.228603 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32c1c4f8e5a4f4d9314d263fbb06e31" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:30.229739 12597 raft_consensus.cc:2393] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8335d0003eb04bb89225bc768e89326f in current term 1: Already voted for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in this term.
I20260709 12:26:30.232131 12489 leader_election.cc:304] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:30.233289 12666 raft_consensus.cc:3060] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:30.241225 12666 raft_consensus.cc:2749] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:30.260164 12692 consensus_queue.cc:1048] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:30.278898 12669 consensus_queue.cc:1048] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:30.318354 12669 raft_consensus.cc:493] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.319707 12669 raft_consensus.cc:515] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.322639 12669 leader_election.cc:290] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:30.324694 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "668bfcf0c9b54599b87a77fab1322ee1" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:30.324981 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "668bfcf0c9b54599b87a77fab1322ee1" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:30.325873 12597 raft_consensus.cc:2468] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 0.
I20260709 12:26:30.326330 12523 raft_consensus.cc:2468] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 0.
I20260709 12:26:30.328843 12414 leader_election.cc:304] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:30.330950 12669 raft_consensus.cc:2804] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:30.331653 12669 raft_consensus.cc:493] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.332126 12669 raft_consensus.cc:3060] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:30.341424 12669 raft_consensus.cc:515] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.344515 12669 leader_election.cc:290] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Requested vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:30.346181 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "668bfcf0c9b54599b87a77fab1322ee1" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:30.346462 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "668bfcf0c9b54599b87a77fab1322ee1" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:30.347326 12523 raft_consensus.cc:3060] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:30.347576 12597 raft_consensus.cc:3060] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:30.359453 12523 raft_consensus.cc:2468] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 1.
I20260709 12:26:30.361174 12414 leader_election.cc:304] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:30.361753 12597 raft_consensus.cc:2468] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 1.
I20260709 12:26:30.362344 12692 raft_consensus.cc:2804] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:30.363422 12692 raft_consensus.cc:697] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Becoming Leader. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:30.364941 12692 consensus_queue.cc:237] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.378296 12347 catalog_manager.cc:5697] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:30.425061 12666 raft_consensus.cc:493] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.426318 12666 raft_consensus.cc:515] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.428488 12666 leader_election.cc:290] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:30.430150 12448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e61e3600734c4ea12ee399fa362a4e" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:30.430433 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e61e3600734c4ea12ee399fa362a4e" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:30.431739 12597 raft_consensus.cc:2393] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8335d0003eb04bb89225bc768e89326f in current term 1: Already voted for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in this term.
I20260709 12:26:30.433126 12489 leader_election.cc:304] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:30.434343 12666 raft_consensus.cc:3060] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:30.440428 12666 raft_consensus.cc:2749] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:30.521006 12668 consensus_queue.cc:1048] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:30.533588 12648 consensus_queue.cc:1048] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:30.534715 12692 consensus_queue.cc:1048] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:30.571571 12650 consensus_queue.cc:1048] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:30.604153 12666 consensus_queue.cc:1048] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:30.630744 12715 consensus_queue.cc:1048] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:30.774677 12597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
mode: GRACEFUL
new_leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:47384
I20260709 12:26:30.775327 12597 raft_consensus.cc:606] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Received request to transfer leadership to TS dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:30.784505 12597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d01036a8c7c14415a05dfacc65b7b920"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
mode: GRACEFUL
new_leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:47384
I20260709 12:26:30.785226 12597 raft_consensus.cc:606] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Received request to transfer leadership to TS dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:30.799052 12669 consensus_queue.cc:1048] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:30.811021 12448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3473a20c790f484b970e807625ce16cf"
dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
mode: GRACEFUL
new_leader_uuid: "8335d0003eb04bb89225bc768e89326f"
 from {username='slave'} at 127.0.0.1:34204
I20260709 12:26:30.811966 12448 raft_consensus.cc:606] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Received request to transfer leadership to TS 8335d0003eb04bb89225bc768e89326f
I20260709 12:26:30.824083 12650 consensus_queue.cc:1048] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:30.833909 12599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
mode: GRACEFUL
new_leader_uuid: "8335d0003eb04bb89225bc768e89326f"
 from {username='slave'} at 127.0.0.1:47384
I20260709 12:26:30.835796 12599 raft_consensus.cc:606] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Received request to transfer leadership to TS 8335d0003eb04bb89225bc768e89326f
I20260709 12:26:30.854313 12692 raft_consensus.cc:993] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: : Instructing follower 8335d0003eb04bb89225bc768e89326f to start an election
I20260709 12:26:30.855618 12650 raft_consensus.cc:1081] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Signalling peer 8335d0003eb04bb89225bc768e89326f to start an election
I20260709 12:26:30.862323 12448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "668bfcf0c9b54599b87a77fab1322ee1"
dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
mode: GRACEFUL
new_leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
 from {username='slave'} at 127.0.0.1:34204
I20260709 12:26:30.863706 12448 raft_consensus.cc:606] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Received request to transfer leadership to TS e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:30.867465 12523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3473a20c790f484b970e807625ce16cf"
dest_uuid: "8335d0003eb04bb89225bc768e89326f"
 from {username='slave'} at 127.0.0.1:46818
I20260709 12:26:30.868521 12523 raft_consensus.cc:493] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:30.869318 12523 raft_consensus.cc:3060] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:30.883530 12523 raft_consensus.cc:515] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.886940 12523 leader_election.cc:290] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 2 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:30.888311 12448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3473a20c790f484b970e807625ce16cf" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:30.889577 12448 raft_consensus.cc:3055] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:30.890082 12448 raft_consensus.cc:740] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:30.891141 12448 consensus_queue.cc:260] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.891651 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3473a20c790f484b970e807625ce16cf" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:30.892340 12597 raft_consensus.cc:3060] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:30.893779 12448 raft_consensus.cc:3060] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:30.903589 12448 raft_consensus.cc:2468] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 2.
I20260709 12:26:30.903913 12597 raft_consensus.cc:2468] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 2.
I20260709 12:26:30.905653 12489 leader_election.cc:304] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:30.906863 12715 raft_consensus.cc:2804] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:30.907569 12715 raft_consensus.cc:697] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 2 LEADER]: Becoming Leader. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Running, Role: LEADER
I20260709 12:26:30.909168 12715 consensus_queue.cc:237] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:30.924525 12344 catalog_manager.cc:5697] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f reported cstate change: term changed from 1 to 2, leader changed from dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65) to 8335d0003eb04bb89225bc768e89326f (127.1.123.66). New cstate: current_term: 2 leader_uuid: "8335d0003eb04bb89225bc768e89326f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:31.058522 12668 raft_consensus.cc:993] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5: : Instructing follower dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:31.059365 12668 raft_consensus.cc:1081] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Signalling peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:31.062189 12448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025"
dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:34188
I20260709 12:26:31.062826 12448 raft_consensus.cc:493] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:31.063202 12448 raft_consensus.cc:3060] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.074023 12448 raft_consensus.cc:515] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.078651 12448 leader_election.cc:290] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Requested vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:31.078912 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:31.080359 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:31.081039 12523 raft_consensus.cc:3060] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.081111 12597 raft_consensus.cc:3055] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:31.082067 12597 raft_consensus.cc:740] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:31.082993 12597 consensus_queue.cc:260] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.084900 12597 raft_consensus.cc:3060] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.091701 12523 raft_consensus.cc:2468] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:31.093086 12597 raft_consensus.cc:2468] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:31.094666 12414 leader_election.cc:304] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:31.096711 12734 raft_consensus.cc:2804] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:31.097370 12734 raft_consensus.cc:697] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Becoming Leader. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:31.098409 12734 consensus_queue.cc:237] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.112463 12344 catalog_manager.cc:5697] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 reported cstate change: term changed from 1 to 2, leader changed from e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67) to dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:31.119270 12648 raft_consensus.cc:993] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5: : Instructing follower 8335d0003eb04bb89225bc768e89326f to start an election
I20260709 12:26:31.119904 12648 raft_consensus.cc:1081] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Signalling peer 8335d0003eb04bb89225bc768e89326f to start an election
I20260709 12:26:31.122697 12523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7"
dest_uuid: "8335d0003eb04bb89225bc768e89326f"
 from {username='slave'} at 127.0.0.1:46830
I20260709 12:26:31.123613 12523 raft_consensus.cc:493] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:31.124075 12523 raft_consensus.cc:3060] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.134152 12523 raft_consensus.cc:515] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.137455 12523 leader_election.cc:290] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 2 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:31.139298 12448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:31.140123 12448 raft_consensus.cc:3060] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.139681 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:31.142016 12597 raft_consensus.cc:3055] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:31.142722 12597 raft_consensus.cc:740] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:31.143784 12597 consensus_queue.cc:260] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.144930 12597 raft_consensus.cc:3060] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.150244 12448 raft_consensus.cc:2468] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 2.
I20260709 12:26:31.153005 12489 leader_election.cc:304] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:31.154354 12715 raft_consensus.cc:2804] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:31.154474 12597 raft_consensus.cc:2468] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 2.
I20260709 12:26:31.156179 12715 raft_consensus.cc:697] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 2 LEADER]: Becoming Leader. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Running, Role: LEADER
I20260709 12:26:31.157213 12715 consensus_queue.cc:237] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.167693 12344 catalog_manager.cc:5697] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f reported cstate change: term changed from 1 to 2, leader changed from e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67) to 8335d0003eb04bb89225bc768e89326f (127.1.123.66). New cstate: current_term: 2 leader_uuid: "8335d0003eb04bb89225bc768e89326f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:31.231931 12668 raft_consensus.cc:993] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5: : Instructing follower dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:31.232734 12648 raft_consensus.cc:1081] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Signalling peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:31.235033 12448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d01036a8c7c14415a05dfacc65b7b920"
dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:34188
I20260709 12:26:31.235829 12448 raft_consensus.cc:493] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:31.236140 12448 raft_consensus.cc:3060] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.244256 12448 raft_consensus.cc:515] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.246500 12448 leader_election.cc:290] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Requested vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:31.247970 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d01036a8c7c14415a05dfacc65b7b920" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:31.248373 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d01036a8c7c14415a05dfacc65b7b920" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:31.248736 12523 raft_consensus.cc:3060] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.249214 12599 raft_consensus.cc:3055] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:31.249544 12599 raft_consensus.cc:740] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:31.250538 12599 consensus_queue.cc:260] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.252682 12599 raft_consensus.cc:3060] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.256487 12523 raft_consensus.cc:2468] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:31.258424 12414 leader_election.cc:304] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:31.260344 12734 raft_consensus.cc:2804] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:31.260896 12734 raft_consensus.cc:697] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Becoming Leader. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:31.262135 12734 consensus_queue.cc:237] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.262403 12599 raft_consensus.cc:2468] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:31.275662 12344 catalog_manager.cc:5697] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 reported cstate change: term changed from 1 to 2, leader changed from e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67) to dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:31.315152 12599 raft_consensus.cc:1275] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Refusing update from remote peer 8335d0003eb04bb89225bc768e89326f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:31.315775 12692 raft_consensus.cc:993] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: : Instructing follower e63758d19a9946fbb92b90abc13ba7e5 to start an election
I20260709 12:26:31.316372 12692 raft_consensus.cc:1081] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Signalling peer e63758d19a9946fbb92b90abc13ba7e5 to start an election
I20260709 12:26:31.317286 12715 consensus_queue.cc:1048] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:31.319029 12599 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "668bfcf0c9b54599b87a77fab1322ee1"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
 from {username='slave'} at 127.0.0.1:47380
I20260709 12:26:31.320416 12599 raft_consensus.cc:493] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:31.320833 12599 raft_consensus.cc:3060] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.330545 12599 raft_consensus.cc:515] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.334642 12448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "668bfcf0c9b54599b87a77fab1322ee1" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:31.335458 12448 raft_consensus.cc:3055] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:31.335902 12448 raft_consensus.cc:740] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:31.335677 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "668bfcf0c9b54599b87a77fab1322ee1" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:31.336944 12523 raft_consensus.cc:3060] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.337251 12448 consensus_queue.cc:260] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.339061 12448 raft_consensus.cc:3060] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:31.342856 12449 raft_consensus.cc:1275] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Refusing update from remote peer 8335d0003eb04bb89225bc768e89326f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:31.345265 12666 consensus_queue.cc:1048] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:31.350828 12448 raft_consensus.cc:2468] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 2.
I20260709 12:26:31.352036 12599 leader_election.cc:290] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 2 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:31.352619 12563 leader_election.cc:304] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:31.350708 12523 raft_consensus.cc:2468] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 2.
I20260709 12:26:31.354305 12648 raft_consensus.cc:2804] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:31.354995 12648 raft_consensus.cc:697] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:31.356429 12648 consensus_queue.cc:237] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:31.369248 12345 catalog_manager.cc:5697] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 1 to 2, leader changed from dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65) to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:31.576324 12523 raft_consensus.cc:1275] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:31.578358 12692 consensus_queue.cc:1048] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:31.591984 12597 raft_consensus.cc:1275] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:31.593931 12692 consensus_queue.cc:1048] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:31.624449 12597 raft_consensus.cc:1275] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Refusing update from remote peer 8335d0003eb04bb89225bc768e89326f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:31.626550 12666 consensus_queue.cc:1048] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:31.640573 12449 raft_consensus.cc:1275] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Refusing update from remote peer 8335d0003eb04bb89225bc768e89326f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:31.643519 12712 consensus_queue.cc:1048] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:31.663866 12597 raft_consensus.cc:1275] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:31.665604 12692 consensus_queue.cc:1048] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:31.678719 12523 raft_consensus.cc:1275] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:31.682000 12734 consensus_queue.cc:1048] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:31.877389 12449 raft_consensus.cc:1275] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Refusing update from remote peer e63758d19a9946fbb92b90abc13ba7e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:31.879719 12648 consensus_queue.cc:1048] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:31.893397 12523 raft_consensus.cc:1275] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Refusing update from remote peer e63758d19a9946fbb92b90abc13ba7e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:31.897071 12753 consensus_queue.cc:1048] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:33.904277 12345 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55932:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:26:33.906970 12345 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:26:33.969975 12435 tablet_service.cc:1511] Processing CreateTablet for tablet 58e62c187bd14732a717bb655247ae77 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:33.972117 12514 tablet_service.cc:1511] Processing CreateTablet for tablet 58e62c187bd14732a717bb655247ae77 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:33.973510 12432 tablet_service.cc:1511] Processing CreateTablet for tablet fc4e9f70485d46bfb19ac098241535dc (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:33.974269 12588 tablet_service.cc:1511] Processing CreateTablet for tablet 58e62c187bd14732a717bb655247ae77 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:33.974663 12515 tablet_service.cc:1511] Processing CreateTablet for tablet fc4e9f70485d46bfb19ac098241535dc (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:33.976521 12432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc4e9f70485d46bfb19ac098241535dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:33.979277 12584 tablet_service.cc:1511] Processing CreateTablet for tablet fc4e9f70485d46bfb19ac098241535dc (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:33.979852 12434 tablet_service.cc:1511] Processing CreateTablet for tablet c219abe0ae174078b653a16da6f547f0 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:33.980443 12507 tablet_service.cc:1511] Processing CreateTablet for tablet c219abe0ae174078b653a16da6f547f0 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:33.981070 12584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc4e9f70485d46bfb19ac098241535dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:33.982843 12586 tablet_service.cc:1511] Processing CreateTablet for tablet c219abe0ae174078b653a16da6f547f0 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:33.984315 12586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c219abe0ae174078b653a16da6f547f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:33.986756 12439 tablet_service.cc:1511] Processing CreateTablet for tablet dac711d68a3243a5947623fd84723fba (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:33.987943 12510 tablet_service.cc:1511] Processing CreateTablet for tablet dac711d68a3243a5947623fd84723fba (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:33.989401 12440 tablet_service.cc:1511] Processing CreateTablet for tablet 08b989081c2847d1a8b5544686b68dd9 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:33.989996 12582 tablet_service.cc:1511] Processing CreateTablet for tablet dac711d68a3243a5947623fd84723fba (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:33.991030 12511 tablet_service.cc:1511] Processing CreateTablet for tablet 08b989081c2847d1a8b5544686b68dd9 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:33.976521 12514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58e62c187bd14732a717bb655247ae77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:33.992561 12515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc4e9f70485d46bfb19ac098241535dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:33.992300 12583 tablet_service.cc:1511] Processing CreateTablet for tablet 08b989081c2847d1a8b5544686b68dd9 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:33.993362 12511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08b989081c2847d1a8b5544686b68dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:33.996567 12440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08b989081c2847d1a8b5544686b68dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:33.996929 12507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c219abe0ae174078b653a16da6f547f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:33.997668 12439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dac711d68a3243a5947623fd84723fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:33.999912 12434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c219abe0ae174078b653a16da6f547f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.003005 12435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58e62c187bd14732a717bb655247ae77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:33.991508 12582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dac711d68a3243a5947623fd84723fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.019845 12512 tablet_service.cc:1511] Processing CreateTablet for tablet d50730b956af4e7c97163f9383c86b44 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:34.020282 12587 tablet_service.cc:1511] Processing CreateTablet for tablet d50730b956af4e7c97163f9383c86b44 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:34.000420 12438 tablet_service.cc:1511] Processing CreateTablet for tablet d50730b956af4e7c97163f9383c86b44 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:34.021986 12433 tablet_service.cc:1511] Processing CreateTablet for tablet 63f764b58817471d902e9f11db53e854 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:34.023092 12438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50730b956af4e7c97163f9383c86b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.023761 12433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63f764b58817471d902e9f11db53e854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.025714 12512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50730b956af4e7c97163f9383c86b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.030411 12510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dac711d68a3243a5947623fd84723fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.050022 12583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08b989081c2847d1a8b5544686b68dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.051054 12587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50730b956af4e7c97163f9383c86b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.054708 12588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58e62c187bd14732a717bb655247ae77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.062110 12770 tablet_bootstrap.cc:492] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:34.087774 12769 tablet_bootstrap.cc:492] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:34.092494 12585 tablet_service.cc:1511] Processing CreateTablet for tablet 63f764b58817471d902e9f11db53e854 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:34.081650 12515 tablet_service.cc:1511] Processing CreateTablet for tablet 63f764b58817471d902e9f11db53e854 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:34.098685 12585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63f764b58817471d902e9f11db53e854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.099622 12515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63f764b58817471d902e9f11db53e854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.102939 12437 tablet_service.cc:1511] Processing CreateTablet for tablet 64a4b7b1c2c448b7bd06ab5503bd2d64 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:34.105252 12437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64a4b7b1c2c448b7bd06ab5503bd2d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.106007 12509 tablet_service.cc:1511] Processing CreateTablet for tablet 64a4b7b1c2c448b7bd06ab5503bd2d64 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:34.107942 12509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64a4b7b1c2c448b7bd06ab5503bd2d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.140897 12588 tablet_service.cc:1511] Processing CreateTablet for tablet 64a4b7b1c2c448b7bd06ab5503bd2d64 (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:34.142942 12588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64a4b7b1c2c448b7bd06ab5503bd2d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.142668 12436 tablet_service.cc:1511] Processing CreateTablet for tablet 5623b2867b034017baf0e8f69807eb2c (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:34.144272 12436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5623b2867b034017baf0e8f69807eb2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.179823 12769 tablet_bootstrap.cc:654] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.201488 12510 tablet_service.cc:1511] Processing CreateTablet for tablet 5623b2867b034017baf0e8f69807eb2c (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:34.203531 12510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5623b2867b034017baf0e8f69807eb2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.199715 12584 tablet_service.cc:1511] Processing CreateTablet for tablet 5623b2867b034017baf0e8f69807eb2c (DEFAULT_TABLE table=soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:34.204195 12770 tablet_bootstrap.cc:654] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.205521 12584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5623b2867b034017baf0e8f69807eb2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:34.203562 12771 tablet_bootstrap.cc:492] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:34.239935 12771 tablet_bootstrap.cc:654] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.246253 12769 tablet_bootstrap.cc:492] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:34.247066 12769 ts_tablet_manager.cc:1403] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.160s	user 0.006s	sys 0.013s
I20260709 12:26:34.253762 12770 tablet_bootstrap.cc:492] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:34.254655 12770 ts_tablet_manager.cc:1403] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.193s	user 0.016s	sys 0.002s
I20260709 12:26:34.266100 12769 raft_consensus.cc:359] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.268380 12769 raft_consensus.cc:385] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.268824 12769 raft_consensus.cc:740] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.270067 12769 consensus_queue.cc:260] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.274353 12769 ts_tablet_manager.cc:1434] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 12:26:34.271253 12770 raft_consensus.cc:359] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.278532 12770 raft_consensus.cc:385] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.279006 12770 raft_consensus.cc:740] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.279949 12770 consensus_queue.cc:260] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.282764 12770 ts_tablet_manager.cc:1434] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.002s
I20260709 12:26:34.287537 12770 tablet_bootstrap.cc:492] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:34.296299 12770 tablet_bootstrap.cc:654] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.284219 12769 tablet_bootstrap.cc:492] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:34.304692 12769 tablet_bootstrap.cc:654] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.335119 12757 raft_consensus.cc:493] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.335882 12757 raft_consensus.cc:515] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.341190 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4e9f70485d46bfb19ac098241535dc" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
W20260709 12:26:34.344619 12563 leader_election.cc:343] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.347993 12757 leader_election.cc:290] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:34.350013 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4e9f70485d46bfb19ac098241535dc" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:34.354523 12758 raft_consensus.cc:493] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.362703 12758 raft_consensus.cc:515] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.366241 12758 leader_election.cc:290] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:34.373916 12523 raft_consensus.cc:2468] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
I20260709 12:26:34.405092 12563 leader_election.cc:304] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:34.410640 12757 raft_consensus.cc:2804] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:34.411402 12757 raft_consensus.cc:493] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.411960 12757 raft_consensus.cc:3060] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:26:34.478380 12489 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:45359, user_credentials={real_user=slave}} blocked reactor thread for 206324us
I20260709 12:26:34.482913 12771 tablet_bootstrap.cc:492] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:34.484130 12771 ts_tablet_manager.cc:1403] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.281s	user 0.094s	sys 0.139s
I20260709 12:26:34.486184 12769 tablet_bootstrap.cc:492] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:34.486896 12769 ts_tablet_manager.cc:1403] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.203s	user 0.080s	sys 0.089s
I20260709 12:26:34.488144 12771 raft_consensus.cc:359] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.489166 12771 raft_consensus.cc:385] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.489842 12770 tablet_bootstrap.cc:492] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:34.489179 12757 raft_consensus.cc:515] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.490689 12770 ts_tablet_manager.cc:1403] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.203s	user 0.069s	sys 0.111s
I20260709 12:26:34.489886 12771 raft_consensus.cc:740] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.491106 12769 raft_consensus.cc:359] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.492129 12769 raft_consensus.cc:385] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.492519 12769 raft_consensus.cc:740] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.492674 12771 consensus_queue.cc:260] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.493451 12769 consensus_queue.cc:260] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.495740 12770 raft_consensus.cc:359] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.496850 12770 raft_consensus.cc:385] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.497236 12770 raft_consensus.cc:740] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.498773 12770 consensus_queue.cc:260] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.499383 12771 ts_tablet_manager.cc:1434] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 12:26:34.503225 12770 ts_tablet_manager.cc:1434] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.006s
I20260709 12:26:34.503403 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4e9f70485d46bfb19ac098241535dc" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:34.505187 12770 tablet_bootstrap.cc:492] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:34.506166 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4e9f70485d46bfb19ac098241535dc" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:34.507280 12599 raft_consensus.cc:2393] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8335d0003eb04bb89225bc768e89326f in current term 1: Already voted for candidate e63758d19a9946fbb92b90abc13ba7e5 in this term.
I20260709 12:26:34.508049 12771 tablet_bootstrap.cc:492] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
W20260709 12:26:34.509450 12489 leader_election.cc:343] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.513582 12769 ts_tablet_manager.cc:1434] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.026s	user 0.012s	sys 0.006s
I20260709 12:26:34.515369 12769 tablet_bootstrap.cc:492] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:34.520448 12770 tablet_bootstrap.cc:654] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.521690 12757 leader_election.cc:290] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:34.521919 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4e9f70485d46bfb19ac098241535dc" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:34.522854 12525 raft_consensus.cc:3060] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:34.524534 12769 tablet_bootstrap.cc:654] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.526942 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4e9f70485d46bfb19ac098241535dc" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
W20260709 12:26:34.529214 12563 leader_election.cc:343] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.534947 12770 tablet_bootstrap.cc:492] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:34.535899 12770 ts_tablet_manager.cc:1403] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.011s
I20260709 12:26:34.541042 12770 raft_consensus.cc:359] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.542141 12770 raft_consensus.cc:385] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.542601 12770 raft_consensus.cc:740] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.542654 12771 tablet_bootstrap.cc:654] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.543741 12770 consensus_queue.cc:260] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.549491 12770 ts_tablet_manager.cc:1434] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.002s
I20260709 12:26:34.551041 12770 tablet_bootstrap.cc:492] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:34.570178 12757 raft_consensus.cc:493] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.570873 12757 raft_consensus.cc:515] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.574234 12757 leader_election.cc:290] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:34.574376 12770 tablet_bootstrap.cc:654] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.576664 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e62c187bd14732a717bb655247ae77" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
W20260709 12:26:34.581403 12563 leader_election.cc:343] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.583359 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e62c187bd14732a717bb655247ae77" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:34.584534 12449 raft_consensus.cc:2468] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
I20260709 12:26:34.587095 12563 leader_election.cc:304] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 8335d0003eb04bb89225bc768e89326f
I20260709 12:26:34.588801 12757 raft_consensus.cc:2804] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:34.589377 12757 raft_consensus.cc:493] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.590173 12757 raft_consensus.cc:3060] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:34.634722 12757 raft_consensus.cc:515] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
W20260709 12:26:34.636754 12489 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:45359, user_credentials={real_user=slave}} blocked reactor thread for 126285us
I20260709 12:26:34.639654 12489 leader_election.cc:304] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:34.649446 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e62c187bd14732a717bb655247ae77" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:34.650748 12449 raft_consensus.cc:3060] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:34.655906 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e62c187bd14732a717bb655247ae77" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:34.649356 12771 tablet_bootstrap.cc:492] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:34.659053 12757 leader_election.cc:290] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:34.647291 12525 raft_consensus.cc:2468] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:34.660261 12775 raft_consensus.cc:2749] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:26:34.660475 12563 leader_election.cc:343] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.661758 12563 leader_election.cc:304] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:34.657866 12771 ts_tablet_manager.cc:1403] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.150s	user 0.032s	sys 0.033s
I20260709 12:26:34.663700 12757 raft_consensus.cc:2804] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:34.664528 12757 raft_consensus.cc:697] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:34.670810 12757 consensus_queue.cc:237] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.675504 12770 tablet_bootstrap.cc:492] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:34.680521 12770 ts_tablet_manager.cc:1403] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.130s	user 0.033s	sys 0.018s
I20260709 12:26:34.683807 12770 raft_consensus.cc:359] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.684520 12770 raft_consensus.cc:385] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.684799 12770 raft_consensus.cc:740] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.679509 12769 tablet_bootstrap.cc:492] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:34.686115 12769 ts_tablet_manager.cc:1403] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.171s	user 0.030s	sys 0.041s
I20260709 12:26:34.685639 12770 consensus_queue.cc:260] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.689064 12770 ts_tablet_manager.cc:1434] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:26:34.690274 12769 raft_consensus.cc:359] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.690936 12770 tablet_bootstrap.cc:492] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:34.691219 12769 raft_consensus.cc:385] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.691555 12769 raft_consensus.cc:740] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.692497 12769 consensus_queue.cc:260] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.695655 12769 ts_tablet_manager.cc:1434] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:26:34.697176 12769 tablet_bootstrap.cc:492] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:34.701123 12770 tablet_bootstrap.cc:654] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.706442 12775 raft_consensus.cc:493] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.706895 12769 tablet_bootstrap.cc:654] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.707175 12775 raft_consensus.cc:515] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.709883 12775 leader_election.cc:290] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:34.713783 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:34.714272 12450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
W20260709 12:26:34.716511 12489 leader_election.cc:343] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.686264 12771 raft_consensus.cc:359] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.718516 12771 raft_consensus.cc:385] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.719136 12771 raft_consensus.cc:740] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
W20260709 12:26:34.720046 12489 leader_election.cc:343] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.721712 12489 leader_election.cc:304] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:34.723397 12779 raft_consensus.cc:2749] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:34.720803 12449 raft_consensus.cc:2468] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:34.728094 12563 leader_election.cc:304] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 8335d0003eb04bb89225bc768e89326f
I20260709 12:26:34.729337 12778 raft_consensus.cc:2804] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:34.721972 12771 consensus_queue.cc:260] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.734607 12771 ts_tablet_manager.cc:1434] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.005s
I20260709 12:26:34.735970 12771 tablet_bootstrap.cc:492] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:34.740545 12769 tablet_bootstrap.cc:492] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:34.741411 12769 ts_tablet_manager.cc:1403] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.000s
W20260709 12:26:34.748003 12396 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:34.745409 12769 raft_consensus.cc:359] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.751729 12769 raft_consensus.cc:385] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.752312 12769 raft_consensus.cc:740] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.753172 12770 tablet_bootstrap.cc:492] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:34.753844 12770 ts_tablet_manager.cc:1403] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.063s	user 0.022s	sys 0.018s
I20260709 12:26:34.754734 12769 consensus_queue.cc:260] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.758293 12769 ts_tablet_manager.cc:1434] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 12:26:34.760340 12769 tablet_bootstrap.cc:492] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:34.762168 12346 catalog_manager.cc:5697] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:34.768983 12770 raft_consensus.cc:359] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.769375 12778 raft_consensus.cc:697] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:34.770232 12770 raft_consensus.cc:385] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.770717 12770 raft_consensus.cc:740] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.771620 12784 raft_consensus.cc:493] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.771548 12770 consensus_queue.cc:260] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.771180 12778 consensus_queue.cc:237] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.772375 12784 raft_consensus.cc:515] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.774580 12770 ts_tablet_manager.cc:1434] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:26:34.776712 12770 tablet_bootstrap.cc:492] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:34.776882 12775 raft_consensus.cc:493] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.777849 12775 raft_consensus.cc:515] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.782691 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50730b956af4e7c97163f9383c86b44" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:34.785516 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50730b956af4e7c97163f9383c86b44" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:34.786898 12770 tablet_bootstrap.cc:654] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
W20260709 12:26:34.788405 12414 leader_election.cc:343] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:26:34.789290 12414 leader_election.cc:343] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.789734 12414 leader_election.cc:304] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:34.775141 12784 leader_election.cc:290] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:34.792165 12775 leader_election.cc:290] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:34.794253 12784 raft_consensus.cc:2749] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:34.795058 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b989081c2847d1a8b5544686b68dd9" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:34.796016 12597 raft_consensus.cc:2468] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 0.
I20260709 12:26:34.797485 12489 leader_election.cc:304] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:34.797942 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b989081c2847d1a8b5544686b68dd9" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:34.799139 12775 raft_consensus.cc:2804] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:34.798676 12769 tablet_bootstrap.cc:654] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.799671 12775 raft_consensus.cc:493] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.800151 12775 raft_consensus.cc:3060] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:34.803759 12771 tablet_bootstrap.cc:654] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
W20260709 12:26:34.809453 12489 leader_election.cc:343] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.829723 12775 raft_consensus.cc:515] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.832338 12769 tablet_bootstrap.cc:492] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:34.832867 12775 leader_election.cc:290] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:34.833910 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b989081c2847d1a8b5544686b68dd9" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:34.834831 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b989081c2847d1a8b5544686b68dd9" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:34.833024 12769 ts_tablet_manager.cc:1403] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.073s	user 0.018s	sys 0.004s
I20260709 12:26:34.835531 12597 raft_consensus.cc:3060] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:26:34.841992 12489 leader_election.cc:343] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.842046 12769 raft_consensus.cc:359] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.843747 12769 raft_consensus.cc:385] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.844084 12769 raft_consensus.cc:740] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.844195 12770 tablet_bootstrap.cc:492] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:34.844997 12770 ts_tablet_manager.cc:1403] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.069s	user 0.020s	sys 0.001s
I20260709 12:26:34.845245 12769 consensus_queue.cc:260] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.849426 12770 raft_consensus.cc:359] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.862084 12770 raft_consensus.cc:385] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.857363 12769 ts_tablet_manager.cc:1434] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.001s
I20260709 12:26:34.862287 12597 raft_consensus.cc:2468] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 1.
I20260709 12:26:34.863618 12769 tablet_bootstrap.cc:492] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:34.864316 12489 leader_election.cc:304] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:34.866140 12775 raft_consensus.cc:2804] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:34.869530 12775 raft_consensus.cc:697] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 1 LEADER]: Becoming Leader. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Running, Role: LEADER
I20260709 12:26:34.871073 12775 consensus_queue.cc:237] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.875058 12769 tablet_bootstrap.cc:654] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.867060 12770 raft_consensus.cc:740] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.876432 12770 consensus_queue.cc:260] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.880266 12770 ts_tablet_manager.cc:1434] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.008s
I20260709 12:26:34.882529 12770 tablet_bootstrap.cc:492] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:34.888806 12771 tablet_bootstrap.cc:492] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:34.888300 12346 catalog_manager.cc:5697] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:34.889719 12771 ts_tablet_manager.cc:1403] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.154s	user 0.028s	sys 0.016s
I20260709 12:26:34.895269 12778 raft_consensus.cc:493] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.895949 12778 raft_consensus.cc:515] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.897367 12769 tablet_bootstrap.cc:492] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:34.898514 12770 tablet_bootstrap.cc:654] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.899323 12778 leader_election.cc:290] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:34.901485 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac711d68a3243a5947623fd84723fba" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:34.898555 12769 ts_tablet_manager.cc:1403] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.007s
I20260709 12:26:34.910779 12525 raft_consensus.cc:2468] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
I20260709 12:26:34.913455 12563 leader_election.cc:304] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:34.914556 12778 raft_consensus.cc:2804] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:34.915091 12778 raft_consensus.cc:493] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.916036 12778 raft_consensus.cc:3060] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:34.916791 12345 catalog_manager.cc:5697] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8335d0003eb04bb89225bc768e89326f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "8335d0003eb04bb89225bc768e89326f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:34.922374 12770 tablet_bootstrap.cc:492] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:34.923288 12770 ts_tablet_manager.cc:1403] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.005s
I20260709 12:26:34.901623 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac711d68a3243a5947623fd84723fba" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:34.905689 12771 raft_consensus.cc:359] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.927168 12771 raft_consensus.cc:385] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.928083 12771 raft_consensus.cc:740] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
W20260709 12:26:34.929376 12563 leader_election.cc:343] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.929872 12771 consensus_queue.cc:260] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.933121 12771 ts_tablet_manager.cc:1434] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.000s
I20260709 12:26:34.935039 12769 raft_consensus.cc:359] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.939455 12769 raft_consensus.cc:385] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.939858 12769 raft_consensus.cc:740] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.940785 12769 consensus_queue.cc:260] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.944077 12769 ts_tablet_manager.cc:1434] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:26:34.945557 12769 tablet_bootstrap.cc:492] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:34.946488 12770 raft_consensus.cc:359] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.947604 12770 raft_consensus.cc:385] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:34.947957 12770 raft_consensus.cc:740] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:34.938460 12778 raft_consensus.cc:515] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.949084 12770 consensus_queue.cc:260] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.951817 12778 leader_election.cc:290] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:34.954326 12771 tablet_bootstrap.cc:492] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:34.956609 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac711d68a3243a5947623fd84723fba" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:34.957830 12525 raft_consensus.cc:3060] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:34.962438 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac711d68a3243a5947623fd84723fba" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:34.964614 12769 tablet_bootstrap.cc:654] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
W20260709 12:26:34.968647 12563 leader_election.cc:343] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:34.971313 12771 tablet_bootstrap.cc:654] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:34.983297 12525 raft_consensus.cc:2468] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:34.984814 12770 ts_tablet_manager.cc:1434] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.061s	user 0.012s	sys 0.000s
I20260709 12:26:34.986580 12770 tablet_bootstrap.cc:492] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:34.989995 12769 tablet_bootstrap.cc:492] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:34.991334 12769 ts_tablet_manager.cc:1403] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.005s
I20260709 12:26:34.996186 12769 raft_consensus.cc:359] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.000933 12771 tablet_bootstrap.cc:492] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:35.002620 12771 ts_tablet_manager.cc:1403] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.009s
I20260709 12:26:35.010602 12771 raft_consensus.cc:359] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.011814 12771 raft_consensus.cc:385] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:35.012104 12771 raft_consensus.cc:740] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:35.012854 12771 consensus_queue.cc:260] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:34.997216 12769 raft_consensus.cc:385] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:35.013873 12769 raft_consensus.cc:740] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:35.014786 12769 consensus_queue.cc:260] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.016629 12771 ts_tablet_manager.cc:1434] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.004s
I20260709 12:26:35.018419 12771 tablet_bootstrap.cc:492] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:35.018654 12769 ts_tablet_manager.cc:1434] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.027s	user 0.011s	sys 0.000s
I20260709 12:26:35.020064 12769 tablet_bootstrap.cc:492] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:35.022024 12563 leader_election.cc:304] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:35.023514 12778 raft_consensus.cc:2804] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:35.024116 12778 raft_consensus.cc:697] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:35.024995 12778 consensus_queue.cc:237] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.042337 12770 tablet_bootstrap.cc:654] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:35.046100 12771 tablet_bootstrap.cc:654] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:35.060982 12770 tablet_bootstrap.cc:492] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:35.060930 12771 tablet_bootstrap.cc:492] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:35.062144 12770 ts_tablet_manager.cc:1403] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.076s	user 0.023s	sys 0.007s
I20260709 12:26:35.063140 12771 ts_tablet_manager.cc:1403] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.011s
I20260709 12:26:35.066655 12770 raft_consensus.cc:359] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.068151 12770 raft_consensus.cc:385] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:35.068491 12770 raft_consensus.cc:740] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:35.068603 12771 raft_consensus.cc:359] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.059911 12344 catalog_manager.cc:5697] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:35.069722 12770 consensus_queue.cc:260] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.072484 12770 ts_tablet_manager.cc:1434] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:26:35.076498 12769 tablet_bootstrap.cc:654] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:35.080194 12771 raft_consensus.cc:385] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:35.080791 12771 raft_consensus.cc:740] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:35.082319 12770 tablet_bootstrap.cc:492] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f: Bootstrap starting.
I20260709 12:26:35.082489 12771 consensus_queue.cc:260] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.089407 12771 ts_tablet_manager.cc:1434] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.026s	user 0.002s	sys 0.010s
I20260709 12:26:35.091611 12771 tablet_bootstrap.cc:492] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:35.096117 12770 tablet_bootstrap.cc:654] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:35.100993 12771 tablet_bootstrap.cc:654] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:35.110706 12778 raft_consensus.cc:493] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.111891 12778 raft_consensus.cc:515] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.112947 12771 tablet_bootstrap.cc:492] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:35.112789 12769 tablet_bootstrap.cc:492] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:35.113601 12771 ts_tablet_manager.cc:1403] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.005s
I20260709 12:26:35.116995 12769 ts_tablet_manager.cc:1403] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.097s	user 0.025s	sys 0.000s
I20260709 12:26:35.120781 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:35.121819 12778 leader_election.cc:290] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:35.124017 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:35.123704 12771 raft_consensus.cc:359] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.125454 12771 raft_consensus.cc:385] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:35.126066 12771 raft_consensus.cc:740] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
W20260709 12:26:35.126473 12563 leader_election.cc:343] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:35.124969 12525 raft_consensus.cc:2468] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
I20260709 12:26:35.128595 12771 consensus_queue.cc:260] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.130942 12563 leader_election.cc:304] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:35.121881 12769 raft_consensus.cc:359] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.134708 12771 ts_tablet_manager.cc:1434] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 12:26:35.134640 12769 raft_consensus.cc:385] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:35.135927 12769 raft_consensus.cc:740] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:35.136942 12771 tablet_bootstrap.cc:492] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:35.137455 12769 consensus_queue.cc:260] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.141817 12769 ts_tablet_manager.cc:1434] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 12:26:35.145179 12769 tablet_bootstrap.cc:492] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5: Bootstrap starting.
I20260709 12:26:35.150411 12771 tablet_bootstrap.cc:654] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:35.155435 12755 raft_consensus.cc:493] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.156985 12755 raft_consensus.cc:515] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.159773 12769 tablet_bootstrap.cc:654] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:35.168064 12787 raft_consensus.cc:493] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.169641 12787 raft_consensus.cc:515] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.172070 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63f764b58817471d902e9f11db53e854" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:35.173452 12787 leader_election.cc:290] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:35.173451 12769 tablet_bootstrap.cc:492] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5: No bootstrap required, opened a new log
I20260709 12:26:35.176455 12769 ts_tablet_manager.cc:1403] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5: Time spent bootstrapping tablet: real 0.032s	user 0.020s	sys 0.006s
I20260709 12:26:35.133325 12778 raft_consensus.cc:2804] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:35.177048 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63f764b58817471d902e9f11db53e854" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:35.178856 12449 raft_consensus.cc:2468] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
I20260709 12:26:35.177459 12770 tablet_bootstrap.cc:492] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f: No bootstrap required, opened a new log
I20260709 12:26:35.180219 12770 ts_tablet_manager.cc:1403] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f: Time spent bootstrapping tablet: real 0.098s	user 0.026s	sys 0.044s
I20260709 12:26:35.179298 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63f764b58817471d902e9f11db53e854" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:35.177238 12778 raft_consensus.cc:493] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.180397 12769 raft_consensus.cc:359] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.181258 12523 raft_consensus.cc:2468] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
I20260709 12:26:35.181505 12769 raft_consensus.cc:385] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:35.182945 12525 raft_consensus.cc:2468] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 0.
I20260709 12:26:35.183692 12563 leader_election.cc:304] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:35.185006 12787 raft_consensus.cc:2804] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:35.185465 12787 raft_consensus.cc:493] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.186352 12787 raft_consensus.cc:3060] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.186265 12414 leader_election.cc:304] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:35.187639 12784 raft_consensus.cc:2804] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:35.188042 12784 raft_consensus.cc:493] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.188393 12784 raft_consensus.cc:3060] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.191174 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63f764b58817471d902e9f11db53e854" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:35.185616 12778 raft_consensus.cc:3060] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.185288 12755 leader_election.cc:290] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:35.182072 12769 raft_consensus.cc:740] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Initialized, Role: FOLLOWER
I20260709 12:26:35.199796 12784 raft_consensus.cc:515] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.210043 12784 leader_election.cc:290] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Requested vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:35.201022 12770 raft_consensus.cc:359] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.211972 12770 raft_consensus.cc:385] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:35.212280 12770 raft_consensus.cc:740] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Initialized, Role: FOLLOWER
I20260709 12:26:35.206682 12769 consensus_queue.cc:260] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.213827 12787 raft_consensus.cc:515] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.216850 12787 leader_election.cc:290] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:35.217728 12597 raft_consensus.cc:2393] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in current term 1: Already voted for candidate e63758d19a9946fbb92b90abc13ba7e5 in this term.
I20260709 12:26:35.219507 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63f764b58817471d902e9f11db53e854" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:35.221578 12769 ts_tablet_manager.cc:1434] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.002s
I20260709 12:26:35.220438 12525 raft_consensus.cc:3060] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.220537 12778 raft_consensus.cc:515] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.222965 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63f764b58817471d902e9f11db53e854" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:35.225347 12778 leader_election.cc:290] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:35.226089 12784 raft_consensus.cc:493] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.226202 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63f764b58817471d902e9f11db53e854" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:35.227218 12784 raft_consensus.cc:515] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.226527 12450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
W20260709 12:26:35.229516 12563 leader_election.cc:343] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:35.229943 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:35.230943 12523 raft_consensus.cc:3060] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.234445 12787 consensus_queue.cc:1048] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:35.222888 12770 consensus_queue.cc:260] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.238372 12771 tablet_bootstrap.cc:492] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:35.239573 12771 ts_tablet_manager.cc:1403] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.103s	user 0.014s	sys 0.020s
I20260709 12:26:35.240096 12770 ts_tablet_manager.cc:1434] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 12:26:35.242000 12525 raft_consensus.cc:2468] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:35.243484 12784 leader_election.cc:290] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:35.244167 12563 leader_election.cc:304] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:35.244566 12771 raft_consensus.cc:359] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.245684 12787 raft_consensus.cc:2804] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:35.245795 12771 raft_consensus.cc:385] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:35.246619 12787 raft_consensus.cc:697] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:35.246461 12771 raft_consensus.cc:740] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:35.247980 12787 consensus_queue.cc:237] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.248607 12771 consensus_queue.cc:260] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.253753 12771 ts_tablet_manager.cc:1434] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.002s
I20260709 12:26:35.254192 12448 raft_consensus.cc:3060] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.255425 12771 tablet_bootstrap.cc:492] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:35.257807 12779 raft_consensus.cc:493] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.258819 12779 raft_consensus.cc:515] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.266628 12779 leader_election.cc:290] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:35.269323 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50730b956af4e7c97163f9383c86b44" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:35.270071 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63f764b58817471d902e9f11db53e854" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:35.271029 12771 tablet_bootstrap.cc:654] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:35.274302 12449 raft_consensus.cc:2393] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e63758d19a9946fbb92b90abc13ba7e5 in current term 1: Already voted for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in this term.
I20260709 12:26:35.276487 12787 consensus_queue.cc:1048] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:35.270311 12597 raft_consensus.cc:2468] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 0.
I20260709 12:26:35.278916 12489 leader_election.cc:304] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:35.279964 12779 raft_consensus.cc:2804] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:35.280030 12523 raft_consensus.cc:2468] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:35.280483 12779 raft_consensus.cc:493] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.280887 12779 raft_consensus.cc:3060] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.282884 12346 catalog_manager.cc:5697] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:35.284987 12414 leader_election.cc:304] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:35.286814 12563 leader_election.cc:304] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:35.288021 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4b7b1c2c448b7bd06ab5503bd2d64" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:35.288780 12597 raft_consensus.cc:2468] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 0.
I20260709 12:26:35.291507 12778 raft_consensus.cc:2804] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:35.292258 12778 raft_consensus.cc:697] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:35.294853 12779 raft_consensus.cc:515] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.285640 12771 tablet_bootstrap.cc:492] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:35.298046 12784 raft_consensus.cc:2749] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:26:35.300166 12771 ts_tablet_manager.cc:1403] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.013s
I20260709 12:26:35.304296 12775 consensus_queue.cc:1048] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:35.304322 12771 raft_consensus.cc:359] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.306226 12771 raft_consensus.cc:385] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:35.306524 12771 raft_consensus.cc:740] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:35.306686 12779 leader_election.cc:290] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:35.307343 12525 raft_consensus.cc:3060] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.293206 12778 consensus_queue.cc:237] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.307367 12771 consensus_queue.cc:260] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.312604 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4b7b1c2c448b7bd06ab5503bd2d64" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:35.314646 12771 ts_tablet_manager.cc:1434] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 12:26:35.315557 12448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50730b956af4e7c97163f9383c86b44" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:35.319017 12523 raft_consensus.cc:2468] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 0.
I20260709 12:26:35.319090 12448 raft_consensus.cc:2468] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 0.
I20260709 12:26:35.322993 12771 tablet_bootstrap.cc:492] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Bootstrap starting.
I20260709 12:26:35.333278 12771 tablet_bootstrap.cc:654] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:35.330361 12346 catalog_manager.cc:5697] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:35.335695 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50730b956af4e7c97163f9383c86b44" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:35.336445 12597 raft_consensus.cc:3060] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.349735 12597 raft_consensus.cc:2468] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 1.
I20260709 12:26:35.351895 12489 leader_election.cc:304] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:35.353128 12775 raft_consensus.cc:2804] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:35.353937 12775 raft_consensus.cc:697] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 1 LEADER]: Becoming Leader. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Running, Role: LEADER
I20260709 12:26:35.358212 12775 consensus_queue.cc:237] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.338871 12414 leader_election.cc:304] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:35.371230 12810 raft_consensus.cc:2804] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:35.371727 12810 raft_consensus.cc:493] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.372189 12810 raft_consensus.cc:3060] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.374665 12787 consensus_queue.cc:1048] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:35.406363 12448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50730b956af4e7c97163f9383c86b44" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:35.410619 12448 raft_consensus.cc:3060] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.418380 12809 raft_consensus.cc:493] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.419265 12809 raft_consensus.cc:515] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.420586 12810 raft_consensus.cc:515] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.439947 12771 tablet_bootstrap.cc:492] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: No bootstrap required, opened a new log
I20260709 12:26:35.439704 12809 leader_election.cc:290] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:35.429306 12449 raft_consensus.cc:3060] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.440852 12771 ts_tablet_manager.cc:1403] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent bootstrapping tablet: real 0.118s	user 0.032s	sys 0.019s
I20260709 12:26:35.442269 12778 raft_consensus.cc:493] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.443213 12778 raft_consensus.cc:515] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.444789 12771 raft_consensus.cc:359] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.445708 12771 raft_consensus.cc:385] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:35.445916 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac711d68a3243a5947623fd84723fba" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:35.449488 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac711d68a3243a5947623fd84723fba" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:35.451839 12446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5623b2867b034017baf0e8f69807eb2c" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:35.452602 12525 raft_consensus.cc:2393] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in current term 1: Already voted for candidate e63758d19a9946fbb92b90abc13ba7e5 in this term.
I20260709 12:26:35.457620 12778 leader_election.cc:290] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:35.472116 12814 raft_consensus.cc:493] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.446014 12771 raft_consensus.cc:740] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Initialized, Role: FOLLOWER
I20260709 12:26:35.477713 12771 consensus_queue.cc:260] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.479384 12446 raft_consensus.cc:2468] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
W20260709 12:26:35.487749 12819 log.cc:927] Time spent T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:26:35.455513 12448 raft_consensus.cc:2468] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 1.
I20260709 12:26:35.493638 12809 raft_consensus.cc:493] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.494835 12809 raft_consensus.cc:515] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.471264 12414 leader_election.cc:304] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:35.499215 12755 raft_consensus.cc:3060] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.473999 12814 raft_consensus.cc:515] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.502753 12771 ts_tablet_manager.cc:1434] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.003s
I20260709 12:26:35.499966 12810 leader_election.cc:290] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Requested vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:35.510751 12814 leader_election.cc:290] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:35.514755 12345 catalog_manager.cc:5697] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8335d0003eb04bb89225bc768e89326f (127.1.123.66). New cstate: current_term: 1 leader_uuid: "8335d0003eb04bb89225bc768e89326f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:35.533726 12563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:41231, user_credentials={real_user=slave}} blocked reactor thread for 87256.5us
I20260709 12:26:35.549654 12787 consensus_queue.cc:1048] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:35.567305 12755 raft_consensus.cc:2749] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:35.585002 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e62c187bd14732a717bb655247ae77" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:35.599004 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5623b2867b034017baf0e8f69807eb2c" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:35.600078 12525 raft_consensus.cc:2468] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 0.
W20260709 12:26:35.596179 12414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.66:41219, user_credentials={real_user=slave}} blocked reactor thread for 125004us
I20260709 12:26:35.603804 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4b7b1c2c448b7bd06ab5503bd2d64" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:35.608784 12598 raft_consensus.cc:3060] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.606153 12448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e62c187bd14732a717bb655247ae77" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" is_pre_election: true
I20260709 12:26:35.614127 12779 consensus_queue.cc:1048] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:35.608839 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4b7b1c2c448b7bd06ab5503bd2d64" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:35.622035 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b989081c2847d1a8b5544686b68dd9" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:35.615048 12809 leader_election.cc:290] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:35.625401 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b989081c2847d1a8b5544686b68dd9" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:35.627708 12563 leader_election.cc:304] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:35.631632 12824 raft_consensus.cc:2804] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:35.632198 12824 raft_consensus.cc:493] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.632663 12824 raft_consensus.cc:3060] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.628547 12525 raft_consensus.cc:3060] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.633973 12598 raft_consensus.cc:2468] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 1.
I20260709 12:26:35.643620 12414 leader_election.cc:304] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:35.644395 12755 raft_consensus.cc:2804] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:35.644869 12414 leader_election.cc:304] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:35.644902 12755 raft_consensus.cc:697] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Becoming Leader. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:35.645802 12809 raft_consensus.cc:2749] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:26:35.646538 12755 consensus_queue.cc:237] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.647214 12489 leader_election.cc:304] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f; no voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:35.651348 12778 consensus_queue.cc:1048] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:35.667092 12775 raft_consensus.cc:2749] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:26:35.659570 12824 raft_consensus.cc:515] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.670266 12787 consensus_queue.cc:1048] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:35.672816 12787 consensus_queue.cc:1048] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:35.675468 12824 leader_election.cc:290] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:35.684530 12525 raft_consensus.cc:2468] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 1.
I20260709 12:26:35.687444 12449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5623b2867b034017baf0e8f69807eb2c" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:35.688196 12449 raft_consensus.cc:3060] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.697731 12755 raft_consensus.cc:493] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.698583 12755 raft_consensus.cc:515] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.701159 12755 leader_election.cc:290] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:35.701658 12345 catalog_manager.cc:5697] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:35.704764 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5" is_pre_election: true
I20260709 12:26:35.712785 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5623b2867b034017baf0e8f69807eb2c" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:35.714622 12524 raft_consensus.cc:3060] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:35.721997 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8335d0003eb04bb89225bc768e89326f" is_pre_election: true
I20260709 12:26:35.706259 12449 raft_consensus.cc:2468] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:35.734889 12525 raft_consensus.cc:2393] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in current term 1: Already voted for candidate e63758d19a9946fbb92b90abc13ba7e5 in this term.
I20260709 12:26:35.742038 12787 consensus_queue.cc:1048] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:26:35.748123 12524 raft_consensus.cc:2468] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 1.
I20260709 12:26:35.755039 12563 leader_election.cc:304] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:35.756486 12787 raft_consensus.cc:2804] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:35.757079 12787 raft_consensus.cc:697] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:35.746788 12414 leader_election.cc:304] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:35.758514 12787 consensus_queue.cc:237] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:35.763556 12755 raft_consensus.cc:3060] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:26:35.770064 12396 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:35.777086 12755 raft_consensus.cc:2749] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:35.780591 12824 consensus_queue.cc:1048] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:35.800087 12814 consensus_queue.cc:1048] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:35.806529 12757 consensus_queue.cc:1048] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:35.811074 12346 catalog_manager.cc:5697] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:35.825922 12779 consensus_queue.cc:1048] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:36.031504 12599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58e62c187bd14732a717bb655247ae77"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
mode: GRACEFUL
new_leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:47384
I20260709 12:26:36.032398 12599 raft_consensus.cc:606] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Received request to transfer leadership to TS dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:36.041419 12599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc4e9f70485d46bfb19ac098241535dc"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
mode: GRACEFUL
new_leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:47384
I20260709 12:26:36.042287 12599 raft_consensus.cc:606] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Received request to transfer leadership to TS dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:36.049966 12599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
mode: GRACEFUL
new_leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:47384
I20260709 12:26:36.051085 12599 raft_consensus.cc:606] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Received request to transfer leadership to TS dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:36.060920 12599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dac711d68a3243a5947623fd84723fba"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
mode: GRACEFUL
new_leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:47384
I20260709 12:26:36.061748 12599 raft_consensus.cc:606] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Received request to transfer leadership to TS dc9b2c6f2dbc4f7f888b1b24a0b6fed5
I20260709 12:26:36.078661 12599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63f764b58817471d902e9f11db53e854"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
mode: GRACEFUL
new_leader_uuid: "8335d0003eb04bb89225bc768e89326f"
 from {username='slave'} at 127.0.0.1:47384
I20260709 12:26:36.079404 12599 raft_consensus.cc:606] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Received request to transfer leadership to TS 8335d0003eb04bb89225bc768e89326f
I20260709 12:26:36.087788 12450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64a4b7b1c2c448b7bd06ab5503bd2d64"
dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
mode: GRACEFUL
new_leader_uuid: "8335d0003eb04bb89225bc768e89326f"
 from {username='slave'} at 127.0.0.1:34204
I20260709 12:26:36.088497 12450 raft_consensus.cc:606] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Received request to transfer leadership to TS 8335d0003eb04bb89225bc768e89326f
I20260709 12:26:36.139344 12755 consensus_queue.cc:1048] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:36.171674 12755 consensus_queue.cc:1048] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:36.184621 12787 consensus_queue.cc:1048] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:36.187419 12784 raft_consensus.cc:993] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: : Instructing follower 8335d0003eb04bb89225bc768e89326f to start an election
I20260709 12:26:36.189150 12784 raft_consensus.cc:1081] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Signalling peer 8335d0003eb04bb89225bc768e89326f to start an election
I20260709 12:26:36.194715 12525 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64a4b7b1c2c448b7bd06ab5503bd2d64"
dest_uuid: "8335d0003eb04bb89225bc768e89326f"
 from {username='slave'} at 127.0.0.1:46818
I20260709 12:26:36.195744 12525 raft_consensus.cc:493] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:36.196144 12525 raft_consensus.cc:3060] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.208173 12824 consensus_queue.cc:1048] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:36.208313 12525 raft_consensus.cc:515] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.216460 12450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4b7b1c2c448b7bd06ab5503bd2d64" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:36.217159 12525 leader_election.cc:290] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 2 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:36.217844 12450 raft_consensus.cc:3055] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:36.218465 12450 raft_consensus.cc:740] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:36.218751 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4b7b1c2c448b7bd06ab5503bd2d64" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:36.219461 12450 consensus_queue.cc:260] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.220089 12599 raft_consensus.cc:3060] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.221310 12450 raft_consensus.cc:3060] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.230686 12450 raft_consensus.cc:2468] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 2.
I20260709 12:26:36.233345 12489 leader_election.cc:304] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:36.235985 12599 raft_consensus.cc:2468] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 2.
I20260709 12:26:36.235255 12779 raft_consensus.cc:2804] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:36.239185 12779 raft_consensus.cc:697] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 2 LEADER]: Becoming Leader. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Running, Role: LEADER
I20260709 12:26:36.240224 12779 consensus_queue.cc:237] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.255486 12824 raft_consensus.cc:993] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5: : Instructing follower dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:36.256160 12787 raft_consensus.cc:1081] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Signalling peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:36.257186 12344 catalog_manager.cc:5697] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f reported cstate change: term changed from 1 to 2, leader changed from dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65) to 8335d0003eb04bb89225bc768e89326f (127.1.123.66). New cstate: current_term: 2 leader_uuid: "8335d0003eb04bb89225bc768e89326f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:36.258944 12450 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dac711d68a3243a5947623fd84723fba"
dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:34188
I20260709 12:26:36.259892 12450 raft_consensus.cc:493] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:36.260365 12450 raft_consensus.cc:3060] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.272478 12450 raft_consensus.cc:515] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.275656 12450 leader_election.cc:290] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Requested vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:36.276129 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac711d68a3243a5947623fd84723fba" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:36.277328 12523 raft_consensus.cc:3060] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.277598 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac711d68a3243a5947623fd84723fba" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:36.278935 12598 raft_consensus.cc:3055] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:36.279284 12598 raft_consensus.cc:740] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:36.280305 12598 consensus_queue.cc:260] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.282116 12598 raft_consensus.cc:3060] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.287473 12523 raft_consensus.cc:2468] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:36.289389 12414 leader_election.cc:304] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:36.290324 12784 raft_consensus.cc:2804] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:36.290917 12784 raft_consensus.cc:697] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Becoming Leader. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:36.292060 12757 raft_consensus.cc:993] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5: : Instructing follower 8335d0003eb04bb89225bc768e89326f to start an election
I20260709 12:26:36.292647 12757 raft_consensus.cc:1081] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Signalling peer 8335d0003eb04bb89225bc768e89326f to start an election
I20260709 12:26:36.293277 12598 raft_consensus.cc:2468] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:36.292081 12784 consensus_queue.cc:237] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.295584 12523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "63f764b58817471d902e9f11db53e854"
dest_uuid: "8335d0003eb04bb89225bc768e89326f"
 from {username='slave'} at 127.0.0.1:46830
I20260709 12:26:36.296274 12523 raft_consensus.cc:493] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:36.296635 12523 raft_consensus.cc:3060] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.305738 12523 raft_consensus.cc:515] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.306718 12344 catalog_manager.cc:5697] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 reported cstate change: term changed from 1 to 2, leader changed from e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67) to dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:36.308257 12523 leader_election.cc:290] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 2 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:36.310671 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63f764b58817471d902e9f11db53e854" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:36.311497 12598 raft_consensus.cc:3055] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:36.311882 12598 raft_consensus.cc:740] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:36.311952 12450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63f764b58817471d902e9f11db53e854" candidate_uuid: "8335d0003eb04bb89225bc768e89326f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:36.312770 12450 raft_consensus.cc:3060] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.313005 12598 consensus_queue.cc:260] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.315022 12598 raft_consensus.cc:3060] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.323757 12450 raft_consensus.cc:2468] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 2.
I20260709 12:26:36.324622 12598 raft_consensus.cc:2468] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8335d0003eb04bb89225bc768e89326f in term 2.
I20260709 12:26:36.325668 12489 leader_election.cc:304] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:36.326561 12779 raft_consensus.cc:2804] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:36.327085 12779 raft_consensus.cc:697] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 2 LEADER]: Becoming Leader. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Running, Role: LEADER
I20260709 12:26:36.328269 12779 consensus_queue.cc:237] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.338634 12344 catalog_manager.cc:5697] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f reported cstate change: term changed from 1 to 2, leader changed from e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67) to 8335d0003eb04bb89225bc768e89326f (127.1.123.66). New cstate: current_term: 2 leader_uuid: "8335d0003eb04bb89225bc768e89326f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:36.378342 12787 raft_consensus.cc:993] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5: : Instructing follower dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:36.378924 12826 raft_consensus.cc:1081] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Signalling peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:36.381361 12450 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0"
dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:34188
I20260709 12:26:36.382333 12450 raft_consensus.cc:493] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:36.382687 12450 raft_consensus.cc:3060] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.391129 12450 raft_consensus.cc:515] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.393621 12450 leader_election.cc:290] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Requested vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:36.394881 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:36.395083 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:36.395664 12523 raft_consensus.cc:3060] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.396274 12598 raft_consensus.cc:3055] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:36.396701 12598 raft_consensus.cc:740] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:36.397680 12598 consensus_queue.cc:260] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.399730 12598 raft_consensus.cc:3060] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.405236 12523 raft_consensus.cc:2468] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:36.408229 12414 leader_election.cc:304] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:36.409595 12784 raft_consensus.cc:2804] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:36.410175 12784 raft_consensus.cc:697] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Becoming Leader. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:36.410404 12598 raft_consensus.cc:2468] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:36.410962 12784 consensus_queue.cc:237] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.423097 12344 catalog_manager.cc:5697] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 reported cstate change: term changed from 1 to 2, leader changed from e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67) to dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:36.475102 12787 raft_consensus.cc:993] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5: : Instructing follower dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:36.476235 12826 raft_consensus.cc:1081] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Signalling peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:36.478557 12450 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc4e9f70485d46bfb19ac098241535dc"
dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:34188
I20260709 12:26:36.479532 12450 raft_consensus.cc:493] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:36.479990 12450 raft_consensus.cc:3060] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.489626 12450 raft_consensus.cc:515] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.491844 12450 leader_election.cc:290] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Requested vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:36.492892 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4e9f70485d46bfb19ac098241535dc" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:36.493903 12523 raft_consensus.cc:3060] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.493571 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4e9f70485d46bfb19ac098241535dc" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:36.494788 12598 raft_consensus.cc:3055] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:36.495150 12598 raft_consensus.cc:740] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:36.495971 12598 consensus_queue.cc:260] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.497527 12598 raft_consensus.cc:3060] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.500228 12523 raft_consensus.cc:2468] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:36.502173 12414 leader_election.cc:304] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:36.503608 12784 raft_consensus.cc:2804] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:36.504118 12784 raft_consensus.cc:697] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Becoming Leader. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:36.505571 12784 consensus_queue.cc:237] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.508189 12598 raft_consensus.cc:2468] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:36.509330 12787 raft_consensus.cc:993] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5: : Instructing follower dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:36.510109 12826 raft_consensus.cc:1081] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Signalling peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 to start an election
I20260709 12:26:36.513137 12450 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58e62c187bd14732a717bb655247ae77"
dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
 from {username='slave'} at 127.0.0.1:34188
I20260709 12:26:36.514424 12450 raft_consensus.cc:493] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:36.514907 12450 raft_consensus.cc:3060] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.516275 12344 catalog_manager.cc:5697] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 reported cstate change: term changed from 1 to 2, leader changed from e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67) to dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:36.524075 12450 raft_consensus.cc:515] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.526953 12450 leader_election.cc:290] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Requested vote from peers 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219), e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67:45359)
I20260709 12:26:36.527915 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e62c187bd14732a717bb655247ae77" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:36.528769 12523 raft_consensus.cc:3060] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.530560 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e62c187bd14732a717bb655247ae77" candidate_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
I20260709 12:26:36.531368 12598 raft_consensus.cc:3055] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:36.531737 12598 raft_consensus.cc:740] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:36.532752 12598 consensus_queue.cc:260] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.534844 12598 raft_consensus.cc:3060] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:36.539580 12523 raft_consensus.cc:2468] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:36.541494 12414 leader_election.cc:304] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, dc9b2c6f2dbc4f7f888b1b24a0b6fed5; no voters: 
I20260709 12:26:36.542454 12784 raft_consensus.cc:2804] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:36.543114 12784 raft_consensus.cc:697] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Becoming Leader. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:36.544283 12784 consensus_queue.cc:237] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:36.544868 12598 raft_consensus.cc:2468] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9b2c6f2dbc4f7f888b1b24a0b6fed5 in term 2.
I20260709 12:26:36.557816 12344 catalog_manager.cc:5697] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 reported cstate change: term changed from 1 to 2, leader changed from e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67) to dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:36.677037 12598 raft_consensus.cc:1275] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:36.679208 12755 consensus_queue.cc:1048] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:36.695874 12523 raft_consensus.cc:1275] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:36.698001 12809 consensus_queue.cc:1048] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:36.794983 12598 raft_consensus.cc:1275] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Refusing update from remote peer 8335d0003eb04bb89225bc768e89326f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:36.796363 12599 raft_consensus.cc:1275] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:36.796846 12775 consensus_queue.cc:1048] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:36.798221 12809 consensus_queue.cc:1048] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:36.811105 12598 raft_consensus.cc:1275] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Refusing update from remote peer 8335d0003eb04bb89225bc768e89326f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:36.814966 12775 consensus_queue.cc:1048] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:26:36.829994 12523 raft_consensus.cc:1275] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:36.834818 12450 raft_consensus.cc:1275] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Refusing update from remote peer 8335d0003eb04bb89225bc768e89326f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:36.837021 12784 consensus_queue.cc:1048] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:36.842311 12849 consensus_queue.cc:1048] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:36.842480 12450 raft_consensus.cc:1275] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Refusing update from remote peer 8335d0003eb04bb89225bc768e89326f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:36.852840 12849 consensus_queue.cc:1048] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:36.957080 12525 raft_consensus.cc:1275] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:36.958791 12755 consensus_queue.cc:1048] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:36.970515 12598 raft_consensus.cc:1275] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:36.973242 12755 consensus_queue.cc:1048] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:37.006881 12598 raft_consensus.cc:1275] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:37.008689 12755 consensus_queue.cc:1048] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:37.024361 12525 raft_consensus.cc:1275] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Refusing update from remote peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:37.027158 12809 consensus_queue.cc:1048] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:39.101080 12346 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55932:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:26:39.146061 12346 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55932:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:26:39.235777  1517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:39.745851 12914 tablet.cc:1724] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:26:39.776120 12913 log.cc:927] Time spent T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5: Append to log took a long time: real 0.079s	user 0.000s	sys 0.002s
W20260709 12:26:39.803136 12910 log.cc:927] Time spent T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Append to log took a long time: real 0.073s	user 0.000s	sys 0.002s
W20260709 12:26:39.840485 12898 log.cc:927] Time spent T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5: Append to log took a long time: real 0.278s	user 0.004s	sys 0.000s
W20260709 12:26:39.843374 12920 log.cc:927] Time spent T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5: Append to log took a long time: real 0.057s	user 0.003s	sys 0.000s
W20260709 12:26:39.859406 12551 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:08b989081c2847d1a8b5544686b68dd9)
W20260709 12:26:39.918134 12551 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:08b989081c2847d1a8b5544686b68dd9)
I20260709 12:26:39.857299 12930 tablet.cc:1724] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:39.935214 12921 tablet.cc:1724] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:39.941324 12933 tablet.cc:1724] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:39.756299 12917 tablet.cc:1724] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:39.862592 12932 tablet.cc:1724] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:39.818711 12916 tablet.cc:1724] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:26:39.947077 12476 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:08b989081c2847d1a8b5544686b68dd9)
I20260709 12:26:39.948135 12925 tablet.cc:1724] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:26:39.976723 12919 log.cc:927] Time spent T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Append to log took a long time: real 0.191s	user 0.000s	sys 0.003s
I20260709 12:26:39.977634 12924 tablet.cc:1724] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:39.990089 12445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0"
dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
mode: GRACEFUL
new_leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
 from {username='slave'} at 127.0.0.1:58878
I20260709 12:26:39.991372 12445 raft_consensus.cc:606] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Received request to transfer leadership to TS e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:39.899156 12934 tablet.cc:1724] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.022560 12931 tablet.cc:1724] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.027323 12927 tablet.cc:1724] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.033789 12905 raft_consensus.cc:993] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: : Instructing follower e63758d19a9946fbb92b90abc13ba7e5 to start an election
I20260709 12:26:40.053973 12903 raft_consensus.cc:1081] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Signalling peer e63758d19a9946fbb92b90abc13ba7e5 to start an election
I20260709 12:26:40.058504 12594 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
 from {username='slave'} at 127.0.0.1:47380
I20260709 12:26:40.059383 12594 raft_consensus.cc:493] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:40.059913 12594 raft_consensus.cc:3060] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:40.092223 12924 tablet.cc:1724] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.130435 12921 tablet.cc:1724] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.136940 12914 tablet.cc:1724] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.151618 12916 tablet.cc:1724] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.143826 12917 tablet.cc:1724] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.160287 12921 tablet.cc:1724] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.193636 12930 tablet.cc:1724] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.195056 12925 tablet.cc:1724] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.201179 12914 tablet.cc:1724] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.201035 12932 tablet.cc:1724] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.233954 12931 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:59116 (request call id 18) took 1066 ms. Trace:
I20260709 12:26:40.235699 12931 rpcz_store.cc:276] 0709 12:26:39.167726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.168117 (+   391us) service_pool.cc:224] Handling call
0709 12:26:40.233846 (+1065729us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:39.194938 (+     0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0709 12:26:39.196303 (+  1365us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0709 12:26:39.196373 (+    70us) alter_schema_op.cc:66] Acquired schema lock
0709 12:26:39.211156 (+ 14783us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0709 12:26:39.212029 (+   873us) alter_schema_op.cc:128] START. Timestamp: 1783599999211405
0709 12:26:39.212471 (+   442us) op_driver.cc:348] REPLICATION: starting
0709 12:26:39.239223 (+ 26752us) log.cc:844] Serialized 204 byte log entry
0709 12:26:39.448950 (+209727us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b5400027798. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a84a4571 00007fd8a84f77bc 00007fd8a84fa2c0 00007fd8a84fa248 00007fd8a84fa1c8 00007fd8a84fa183 00007fd8a84f944c 00007fd8ae3a62f6 00007fd8ae3a0aa8 00007fd8a80e2b41 00007fd8a80d5f2a
0709 12:26:39.475255 (+ 26305us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b18001f08a0. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8ae316ab1 00007fd8ae323281 00007fd8a53c8c0a 00007fd8aeb08219 00007fd8a5450a40 00007fd8a544b4e4 00007fd8a544b455 00007fd8a8494b0b 00007fd8a8519b63 00007fd8a8536e41 00007fd8abf71581
0709 12:26:39.860309 (+385054us) op_driver.cc:464] REPLICATION: finished
0709 12:26:40.022109 (+161800us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0709 12:26:40.058081 (+ 35972us) log.cc:844] Serialized 5 byte log entry
0709 12:26:40.191933 (+133852us) tablet_metadata.cc:665] Metadata flushed
0709 12:26:40.224348 (+ 32415us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.226049 (+  1701us) alter_schema_op.cc:73] Released schema lock
0709 12:26:40.227226 (+  1177us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":161424,"mutex_wait_us":103805,"prepare.queue_time_us":23092,"prepare.run_cpu_time_us":55150,"prepare.run_wall_time_us":339505,"raft.queue_time_us":219174,"raft.run_cpu_time_us":4859,"raft.run_wall_time_us":84777,"replication_time_us":647653,"spinlock_wait_cycles":42530816,"thread_start_us":156654,"threads_started":4,"wal-append.queue_time_us":30271}]]}
I20260709 12:26:40.240278 12931 tablet.cc:1724] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.264295 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d50730b956af4e7c97163f9383c86b44"
dest_uuid: "8335d0003eb04bb89225bc768e89326f"
mode: GRACEFUL
new_leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
 from {username='slave'} at 127.0.0.1:37078
I20260709 12:26:40.265710 12524 raft_consensus.cc:606] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 1 LEADER]: Received request to transfer leadership to TS e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:40.267902  1517 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:26:40.268594  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:40.206961 12594 raft_consensus.cc:515] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.224339 12934 tablet.cc:1724] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.246961 12933 tablet.cc:1724] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.270145 12927 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:60848 (request call id 20) took 1084 ms. Trace:
I20260709 12:26:40.278862 12446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:40.280210 12446 raft_consensus.cc:3055] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:40.281173 12446 raft_consensus.cc:740] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:40.278360 12927 rpcz_store.cc:276] 0709 12:26:39.185821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.188169 (+  2348us) service_pool.cc:224] Handling call
0709 12:26:40.269999 (+1081830us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:39.273344 (+     0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0709 12:26:39.276974 (+  3630us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0709 12:26:39.277035 (+    61us) alter_schema_op.cc:66] Acquired schema lock
0709 12:26:39.277138 (+   103us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0709 12:26:39.278674 (+  1536us) alter_schema_op.cc:128] START. Timestamp: 1783599999277454
0709 12:26:39.278720 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 12:26:39.304346 (+ 25626us) log.cc:844] Serialized 204 byte log entry
0709 12:26:39.573661 (+269315us) op_driver.cc:464] REPLICATION: finished
0709 12:26:39.731548 (+157887us) spinlock_profiling.cc:243] Waited 7.63 ms on lock 0x7b4400022b4c. stack: 00007fd8a541c343 00007fd8a541c240 00007fd8a4dab1d4 00007fd8aeae9304 00007fd8aeae9299 00007fd8a52d6db1 00007fd8a5374c85 00007fd8a53756a2 00007fd8a80e71db 00007fd8a80e2aaf 00007fd8a54310ac 00007fd8ae28b9a9 00007fd8a544d69b 00007fd8a544b93f 00007fd8a544dc02 00007fd8abf71ef0
0709 12:26:40.026959 (+295411us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0709 12:26:40.030195 (+  3236us) log.cc:844] Serialized 5 byte log entry
0709 12:26:40.257478 (+227283us) tablet_metadata.cc:665] Metadata flushed
0709 12:26:40.258969 (+  1491us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.269390 (+ 10421us) alter_schema_op.cc:73] Released schema lock
0709 12:26:40.269443 (+    53us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":434728,"mutex_wait_us":69843,"prepare.queue_time_us":74381,"prepare.run_cpu_time_us":40472,"prepare.run_wall_time_us":169067,"raft.queue_time_us":104010,"raft.run_cpu_time_us":3358,"raft.run_wall_time_us":8702,"replication_time_us":263966,"spinlock_wait_cycles":17538816,"thread_start_us":421644,"threads_started":5,"wal-append.queue_time_us":5250}]]}
I20260709 12:26:40.282306 12446 consensus_queue.cc:260] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.284020 12446 raft_consensus.cc:3060] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:40.293689 12594 leader_election.cc:290] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 3 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:40.283910 12927 tablet.cc:1724] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.286739 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c219abe0ae174078b653a16da6f547f0" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:40.299504 12524 raft_consensus.cc:3060] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:40.299086 12933 tablet.cc:1724] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:40.308249 12930 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:59116 (request call id 21) took 1133 ms. Trace:
I20260709 12:26:40.311192 12524 raft_consensus.cc:2468] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 3.
I20260709 12:26:40.308893 12930 rpcz_store.cc:276] 0709 12:26:39.175207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.176187 (+   980us) service_pool.cc:224] Handling call
0709 12:26:40.308152 (+1131965us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:39.241664 (+     0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0709 12:26:39.242572 (+   908us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0709 12:26:39.242611 (+    39us) alter_schema_op.cc:66] Acquired schema lock
0709 12:26:39.242690 (+    79us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0709 12:26:39.243042 (+   352us) alter_schema_op.cc:128] START. Timestamp: 1783599999242924
0709 12:26:39.243090 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 12:26:39.256504 (+ 13414us) log.cc:844] Serialized 204 byte log entry
0709 12:26:39.882889 (+626385us) op_driver.cc:464] REPLICATION: finished
0709 12:26:40.174930 (+292041us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0709 12:26:40.197817 (+ 22887us) log.cc:844] Serialized 5 byte log entry
0709 12:26:40.305956 (+108139us) tablet_metadata.cc:665] Metadata flushed
0709 12:26:40.307014 (+  1058us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.307944 (+   930us) alter_schema_op.cc:73] Released schema lock
0709 12:26:40.307985 (+    41us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":281936,"mutex_wait_us":141502,"prepare.queue_time_us":61583,"prepare.run_cpu_time_us":70174,"prepare.run_wall_time_us":310280,"raft.queue_time_us":172414,"raft.run_cpu_time_us":20,"raft.run_wall_time_us":19,"replication_time_us":639498,"thread_start_us":162709,"threads_started":3,"wal-append.queue_time_us":55856}]]}
I20260709 12:26:40.324682 12563 leader_election.cc:304] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8335d0003eb04bb89225bc768e89326f, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:40.326331 12933 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:59116 (request call id 20) took 1156 ms. Trace:
I20260709 12:26:40.327039 12787 raft_consensus.cc:2804] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:40.328027 12787 raft_consensus.cc:697] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:40.326892 12933 rpcz_store.cc:276] 0709 12:26:39.169850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.170166 (+   316us) service_pool.cc:224] Handling call
0709 12:26:40.326250 (+1156084us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:39.261153 (+     0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0709 12:26:39.263078 (+  1925us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0709 12:26:39.263144 (+    66us) alter_schema_op.cc:66] Acquired schema lock
0709 12:26:39.263219 (+    75us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0709 12:26:39.267262 (+  4043us) alter_schema_op.cc:128] START. Timestamp: 1783599999263337
0709 12:26:39.267294 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 12:26:39.292313 (+ 25019us) log.cc:844] Serialized 204 byte log entry
0709 12:26:40.057876 (+765563us) op_driver.cc:464] REPLICATION: finished
0709 12:26:40.298851 (+240975us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0709 12:26:40.301014 (+  2163us) log.cc:844] Serialized 5 byte log entry
0709 12:26:40.324064 (+ 23050us) tablet_metadata.cc:665] Metadata flushed
0709 12:26:40.325085 (+  1021us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.326041 (+   956us) alter_schema_op.cc:73] Released schema lock
0709 12:26:40.326079 (+    38us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":240773,"mutex_wait_us":141713,"prepare.queue_time_us":58874,"prepare.run_cpu_time_us":35182,"prepare.run_wall_time_us":269009,"raft.queue_time_us":180086,"raft.run_cpu_time_us":2848,"raft.run_wall_time_us":4883,"replication_time_us":789904,"spinlock_wait_cycles":33664,"thread_start_us":85295,"threads_started":3,"wal-append.queue_time_us":41416}]]}
I20260709 12:26:40.330353 12787 consensus_queue.cc:237] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.334880 12446 raft_consensus.cc:2468] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 3.
I20260709 12:26:40.360930 12446 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025"
dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
mode: GRACEFUL
new_leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
 from {username='slave'} at 127.0.0.1:58878
I20260709 12:26:40.362332 12446 raft_consensus.cc:606] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Received request to transfer leadership to TS e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:40.372896 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7"
dest_uuid: "8335d0003eb04bb89225bc768e89326f"
mode: GRACEFUL
new_leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
 from {username='slave'} at 127.0.0.1:37078
I20260709 12:26:40.374179 12524 raft_consensus.cc:606] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 2 LEADER]: Received request to transfer leadership to TS e63758d19a9946fbb92b90abc13ba7e5
I20260709 12:26:40.377707  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:26:40.379009  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:40.390151 12345 catalog_manager.cc:6242] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805: soft_deleted_table [id=e6d79bc048e94fe1ab3f10b21ee9e94a] alter complete (version 1)
I20260709 12:26:40.405048 12345 catalog_manager.cc:5697] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 2 to 3, leader changed from dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65) to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 3 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:40.657872 12850 raft_consensus.cc:993] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f: : Instructing follower e63758d19a9946fbb92b90abc13ba7e5 to start an election
I20260709 12:26:40.658677 12850 raft_consensus.cc:1081] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 1 LEADER]: Signalling peer e63758d19a9946fbb92b90abc13ba7e5 to start an election
I20260709 12:26:40.661113 12595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d50730b956af4e7c97163f9383c86b44"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
 from {username='slave'} at 127.0.0.1:47382
I20260709 12:26:40.661976 12595 raft_consensus.cc:493] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:40.662312 12595 raft_consensus.cc:3060] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:40.671413 12595 raft_consensus.cc:515] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.673990 12595 leader_election.cc:290] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 2 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:40.675216 12446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50730b956af4e7c97163f9383c86b44" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:40.675256 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50730b956af4e7c97163f9383c86b44" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:40.676105 12446 raft_consensus.cc:3060] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:40.676474 12524 raft_consensus.cc:3055] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:40.677040 12524 raft_consensus.cc:740] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Running, Role: LEADER
I20260709 12:26:40.677977 12524 consensus_queue.cc:260] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.679473 12524 raft_consensus.cc:3060] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:40.683969 12446 raft_consensus.cc:2468] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 2.
I20260709 12:26:40.685599 12563 leader_election.cc:304] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:40.686590 12787 raft_consensus.cc:2804] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:40.686697 12524 raft_consensus.cc:2468] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 2.
I20260709 12:26:40.687165 12787 raft_consensus.cc:697] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:40.688596 12787 consensus_queue.cc:237] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.700084 12345 catalog_manager.cc:5697] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 1 to 2, leader changed from 8335d0003eb04bb89225bc768e89326f (127.1.123.66) to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 2 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:40.705341 12892 raft_consensus.cc:993] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f: : Instructing follower e63758d19a9946fbb92b90abc13ba7e5 to start an election
I20260709 12:26:40.706138 12850 raft_consensus.cc:1081] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 2 LEADER]: Signalling peer e63758d19a9946fbb92b90abc13ba7e5 to start an election
I20260709 12:26:40.708526 12595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
 from {username='slave'} at 127.0.0.1:47382
I20260709 12:26:40.709436 12595 raft_consensus.cc:493] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:40.709900 12595 raft_consensus.cc:3060] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:40.718075 12595 raft_consensus.cc:515] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.721932 12595 leader_election.cc:290] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 3 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:40.724172 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:40.724258 12446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab58c31975ba419dacc885725f47efc7" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:40.724733 12903 raft_consensus.cc:993] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: : Instructing follower e63758d19a9946fbb92b90abc13ba7e5 to start an election
I20260709 12:26:40.725242 12524 raft_consensus.cc:3055] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:40.725389 12446 raft_consensus.cc:3060] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:40.725790 12903 raft_consensus.cc:1081] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Signalling peer e63758d19a9946fbb92b90abc13ba7e5 to start an election
I20260709 12:26:40.725943 12524 raft_consensus.cc:740] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8335d0003eb04bb89225bc768e89326f, State: Running, Role: LEADER
I20260709 12:26:40.727421 12524 consensus_queue.cc:260] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.729319 12524 raft_consensus.cc:3060] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:40.728590 12595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025"
dest_uuid: "e63758d19a9946fbb92b90abc13ba7e5"
 from {username='slave'} at 127.0.0.1:47380
I20260709 12:26:40.731685 12595 raft_consensus.cc:493] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:40.732139 12595 raft_consensus.cc:3060] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:40.737345 12446 raft_consensus.cc:2468] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 3.
I20260709 12:26:40.739091 12563 leader_election.cc:304] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:40.740767 12524 raft_consensus.cc:2468] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 3.
I20260709 12:26:40.741299 12787 raft_consensus.cc:2804] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:40.742048 12787 raft_consensus.cc:697] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:40.744159 12787 consensus_queue.cc:237] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.745090 12595 raft_consensus.cc:515] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.749150 12595 leader_election.cc:290] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 3 election: Requested vote from peers dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231), 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219)
I20260709 12:26:40.749926 12446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5"
I20260709 12:26:40.750782 12446 raft_consensus.cc:3055] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:40.751150 12446 raft_consensus.cc:740] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, State: Running, Role: LEADER
I20260709 12:26:40.752417 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673ec45ffbab46d3ba81b5680b57e025" candidate_uuid: "e63758d19a9946fbb92b90abc13ba7e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8335d0003eb04bb89225bc768e89326f"
I20260709 12:26:40.752511 12446 consensus_queue.cc:260] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.753464 12524 raft_consensus.cc:3060] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:40.754467 12446 raft_consensus.cc:3060] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:40.761198 12345 catalog_manager.cc:5697] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 2 to 3, leader changed from 8335d0003eb04bb89225bc768e89326f (127.1.123.66) to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 3 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:40.767293 12446 raft_consensus.cc:2468] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 3.
I20260709 12:26:40.771124 12563 leader_election.cc:304] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc9b2c6f2dbc4f7f888b1b24a0b6fed5, e63758d19a9946fbb92b90abc13ba7e5; no voters: 
I20260709 12:26:40.772433 12787 raft_consensus.cc:2804] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:40.773316 12787 raft_consensus.cc:697] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 LEADER]: Becoming Leader. State: Replica: e63758d19a9946fbb92b90abc13ba7e5, State: Running, Role: LEADER
I20260709 12:26:40.773831 12524 raft_consensus.cc:2468] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63758d19a9946fbb92b90abc13ba7e5 in term 3.
I20260709 12:26:40.775065 12787 consensus_queue.cc:237] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } }
I20260709 12:26:40.790406 12446 raft_consensus.cc:1275] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Refusing update from remote peer e63758d19a9946fbb92b90abc13ba7e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:26:40.792145 12787 consensus_queue.cc:1048] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:40.792825 12345 catalog_manager.cc:5697] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 reported cstate change: term changed from 2 to 3, leader changed from dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65) to e63758d19a9946fbb92b90abc13ba7e5 (127.1.123.67). New cstate: current_term: 3 leader_uuid: "e63758d19a9946fbb92b90abc13ba7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e63758d19a9946fbb92b90abc13ba7e5" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:40.814533 12524 raft_consensus.cc:1275] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Refusing update from remote peer e63758d19a9946fbb92b90abc13ba7e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:26:40.816262 12787 consensus_queue.cc:1048] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:41.166461 12524 raft_consensus.cc:1275] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Refusing update from remote peer e63758d19a9946fbb92b90abc13ba7e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:41.168407 12787 consensus_queue.cc:1048] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:41.175506 12522 raft_consensus.cc:1275] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Refusing update from remote peer e63758d19a9946fbb92b90abc13ba7e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:26:41.177330 12926 consensus_queue.cc:1048] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:41.182132 12446 raft_consensus.cc:1275] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Refusing update from remote peer e63758d19a9946fbb92b90abc13ba7e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:41.185858 12446 raft_consensus.cc:1275] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Refusing update from remote peer e63758d19a9946fbb92b90abc13ba7e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:41.188786 12926 consensus_queue.cc:1048] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:41.190603 12787 consensus_queue.cc:1048] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:41.208056 12449 raft_consensus.cc:1275] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Refusing update from remote peer e63758d19a9946fbb92b90abc13ba7e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:26:41.231124 12926 consensus_queue.cc:1048] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9b2c6f2dbc4f7f888b1b24a0b6fed5" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:26:41.259433 12524 raft_consensus.cc:1275] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Refusing update from remote peer e63758d19a9946fbb92b90abc13ba7e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:41.264889 12948 consensus_queue.cc:1048] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8335d0003eb04bb89225bc768e89326f" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 41219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:41.380676  1517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:41.387041  1517 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:41.387516  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:41.392987  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:41.393443  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:42.394898  1517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:42.403079  1517 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:42.403813  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:42.411141  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:42.411883  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:43.413692  1517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:43.421860  1517 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:43.422705  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:43.430308  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:43.430744  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:44.432219  1517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:44.439734  1517 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:44.440322  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:44.447309  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:44.447876  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:45.449661  1517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:45.456169  1517 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:45.456799  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:45.462625  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:45.463272  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:46.465085  1517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:46.472620  1517 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:46.473174  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:46.479445  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:46.479965  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:47.481144  1517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:47.486467  1517 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:47.486991  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:47.491969  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:47.492436  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:48.494012  1517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:48.503113  1517 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:48.503793  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:48.511520  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:48.512218  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:49.514321  1517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:49.520174  1517 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:49.520824  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:49.528436  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:49.529191  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:50.648059  1517 tablet_server.cc:179] TabletServer@127.1.123.65:0 shutting down...
I20260709 12:26:50.717846  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:50.719089  1517 tablet_replica.cc:333] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.720829  1517 raft_consensus.cc:2243] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:50.722572  1517 raft_consensus.cc:2272] T dac711d68a3243a5947623fd84723fba P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.727252  1517 tablet_replica.cc:333] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.728595  1517 raft_consensus.cc:2243] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.729507  1517 raft_consensus.cc:2272] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.734279  1517 tablet_replica.cc:333] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.735576  1517 raft_consensus.cc:2243] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.736423  1517 raft_consensus.cc:2272] T 08b989081c2847d1a8b5544686b68dd9 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.742707  1517 tablet_replica.cc:333] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.744169  1517 raft_consensus.cc:2243] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.745316  1517 raft_consensus.cc:2272] T d50730b956af4e7c97163f9383c86b44 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.750396  1517 tablet_replica.cc:333] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.751590  1517 raft_consensus.cc:2243] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.752461  1517 raft_consensus.cc:2272] T 5623b2867b034017baf0e8f69807eb2c P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.757036  1517 tablet_replica.cc:333] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.758070  1517 raft_consensus.cc:2243] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.758913  1517 raft_consensus.cc:2272] T eb15df4e9b1e44bd9b2dea30b4e72e66 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.762372  1517 tablet_replica.cc:333] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.763199  1517 raft_consensus.cc:2243] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.763923  1517 raft_consensus.cc:2272] T 668bfcf0c9b54599b87a77fab1322ee1 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.767196  1517 tablet_replica.cc:333] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.768047  1517 raft_consensus.cc:2243] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.769016  1517 raft_consensus.cc:2272] T ab58c31975ba419dacc885725f47efc7 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.773062  1517 tablet_replica.cc:333] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.773998  1517 raft_consensus.cc:2243] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:50.775269  1517 raft_consensus.cc:2272] T f32c1c4f8e5a4f4d9314d263fbb06e31 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.778764  1517 tablet_replica.cc:333] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.779703  1517 raft_consensus.cc:2243] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:50.780947  1517 raft_consensus.cc:2272] T 64e61e3600734c4ea12ee399fa362a4e P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.784953  1517 tablet_replica.cc:333] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.785984  1517 raft_consensus.cc:2243] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:50.787377  1517 raft_consensus.cc:2272] T 58e62c187bd14732a717bb655247ae77 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.792239  1517 tablet_replica.cc:333] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.793720  1517 raft_consensus.cc:2243] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.794749  1517 raft_consensus.cc:2272] T 673ec45ffbab46d3ba81b5680b57e025 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:50.795218 12563 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
I20260709 12:26:50.799731  1517 tablet_replica.cc:333] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
W20260709 12:26:50.800369 12563 consensus_peers.cc:597] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:50.800901  1517 raft_consensus.cc:2243] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.801781  1517 raft_consensus.cc:2272] T ba28428998c24aee8321c9ecc9ce919a P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.805387  1517 tablet_replica.cc:333] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.806340  1517 raft_consensus.cc:2243] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.807118  1517 raft_consensus.cc:2272] T 63f764b58817471d902e9f11db53e854 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.811007  1517 tablet_replica.cc:333] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.811969  1517 raft_consensus.cc:2243] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:50.813889  1517 raft_consensus.cc:2272] T d01036a8c7c14415a05dfacc65b7b920 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.818008  1517 tablet_replica.cc:333] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.818876  1517 raft_consensus.cc:2243] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.819897  1517 raft_consensus.cc:2272] T c219abe0ae174078b653a16da6f547f0 P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:50.824000 12563 consensus_peers.cc:597] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:50.825837  1517 tablet_replica.cc:333] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.827387  1517 raft_consensus.cc:2243] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:50.828660  1517 raft_consensus.cc:2272] T fc4e9f70485d46bfb19ac098241535dc P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.831990  1517 tablet_replica.cc:333] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5: stopping tablet replica
I20260709 12:26:50.832736  1517 raft_consensus.cc:2243] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.833415  1517 raft_consensus.cc:2272] T 3473a20c790f484b970e807625ce16cf P dc9b2c6f2dbc4f7f888b1b24a0b6fed5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.904083  1517 tablet_server.cc:196] TabletServer@127.1.123.65:0 shutdown complete.
W20260709 12:26:50.923484 12489 consensus_peers.cc:597] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:41231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:50.946017 12563 consensus_peers.cc:597] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:41231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:50.952211 12489 consensus_peers.cc:597] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:41231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:50.959210 12489 consensus_peers.cc:597] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:41231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:50.990787 12563 consensus_peers.cc:597] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:41231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:51.002161  1517 tablet_server.cc:179] TabletServer@127.1.123.66:0 shutting down...
W20260709 12:26:51.040081 12489 consensus_peers.cc:597] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:41231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:51.048535 12563 consensus_peers.cc:597] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:41231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:51.069511 12489 consensus_peers.cc:597] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:41231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:51.086670 12563 consensus_peers.cc:597] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:41231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:51.097442  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:51.098711  1517 tablet_replica.cc:333] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.100986  1517 raft_consensus.cc:2243] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.102109  1517 raft_consensus.cc:2272] T 58e62c187bd14732a717bb655247ae77 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.105880  1517 tablet_replica.cc:333] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.106892  1517 raft_consensus.cc:2243] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.108137  1517 raft_consensus.cc:2272] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.111964  1517 tablet_replica.cc:333] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.113154  1517 raft_consensus.cc:2243] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.114473  1517 raft_consensus.cc:2272] T d50730b956af4e7c97163f9383c86b44 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.118985  1517 tablet_replica.cc:333] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.119906  1517 raft_consensus.cc:2243] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.121168  1517 raft_consensus.cc:2272] T 08b989081c2847d1a8b5544686b68dd9 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.124672  1517 tablet_replica.cc:333] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.126116  1517 raft_consensus.cc:2243] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.127087  1517 raft_consensus.cc:2272] T dac711d68a3243a5947623fd84723fba P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.130921  1517 tablet_replica.cc:333] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.131861  1517 raft_consensus.cc:2243] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.132573  1517 raft_consensus.cc:2272] T f32c1c4f8e5a4f4d9314d263fbb06e31 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.137359  1517 tablet_replica.cc:333] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.138705  1517 raft_consensus.cc:2243] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.139694  1517 raft_consensus.cc:2272] T 64e61e3600734c4ea12ee399fa362a4e P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.144085  1517 tablet_replica.cc:333] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.145129  1517 raft_consensus.cc:2243] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.146214  1517 raft_consensus.cc:2272] T c219abe0ae174078b653a16da6f547f0 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.151582  1517 tablet_replica.cc:333] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.152752  1517 raft_consensus.cc:2243] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.154003  1517 raft_consensus.cc:2272] T fc4e9f70485d46bfb19ac098241535dc P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.158340  1517 tablet_replica.cc:333] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.159711  1517 raft_consensus.cc:2243] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.161854  1517 raft_consensus.cc:2272] T 3473a20c790f484b970e807625ce16cf P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.167531  1517 tablet_replica.cc:333] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.168975  1517 raft_consensus.cc:2243] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.171825  1517 raft_consensus.cc:2272] T 5623b2867b034017baf0e8f69807eb2c P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:51.172679 12563 consensus_peers.cc:597] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 -> Peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Couldn't send request to peer 8335d0003eb04bb89225bc768e89326f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:51.177466  1517 tablet_replica.cc:333] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.178921  1517 raft_consensus.cc:2243] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:26:51.179237 12489 consensus_peers.cc:597] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f -> Peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5 (127.1.123.65:41231): Couldn't send request to peer dc9b2c6f2dbc4f7f888b1b24a0b6fed5. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:41231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:51.180948  1517 raft_consensus.cc:2272] T eb15df4e9b1e44bd9b2dea30b4e72e66 P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.184473  1517 tablet_replica.cc:333] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.185415  1517 raft_consensus.cc:2243] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.186240  1517 raft_consensus.cc:2272] T 668bfcf0c9b54599b87a77fab1322ee1 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.190021  1517 tablet_replica.cc:333] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.190889  1517 raft_consensus.cc:2243] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.191543  1517 raft_consensus.cc:2272] T ab58c31975ba419dacc885725f47efc7 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.195425  1517 tablet_replica.cc:333] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.196415  1517 raft_consensus.cc:2243] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.197800  1517 raft_consensus.cc:2272] T 63f764b58817471d902e9f11db53e854 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:51.200240 12563 consensus_peers.cc:597] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 -> Peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Couldn't send request to peer 8335d0003eb04bb89225bc768e89326f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:51.202059  1517 tablet_replica.cc:333] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.202940  1517 raft_consensus.cc:2243] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.204008  1517 raft_consensus.cc:2272] T d01036a8c7c14415a05dfacc65b7b920 P 8335d0003eb04bb89225bc768e89326f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.208146  1517 tablet_replica.cc:333] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.209203  1517 raft_consensus.cc:2243] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.210611  1517 raft_consensus.cc:2272] T 673ec45ffbab46d3ba81b5680b57e025 P 8335d0003eb04bb89225bc768e89326f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.216374  1517 tablet_replica.cc:333] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f: stopping tablet replica
I20260709 12:26:51.217159  1517 raft_consensus.cc:2243] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.218907  1517 raft_consensus.cc:2272] T ba28428998c24aee8321c9ecc9ce919a P 8335d0003eb04bb89225bc768e89326f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:51.245021 12563 consensus_peers.cc:597] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 -> Peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Couldn't send request to peer 8335d0003eb04bb89225bc768e89326f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:51.288520  1517 tablet_server.cc:196] TabletServer@127.1.123.66:0 shutdown complete.
I20260709 12:26:51.354804  1517 tablet_server.cc:179] TabletServer@127.1.123.67:0 shutting down...
W20260709 12:26:51.389101 12563 consensus_peers.cc:597] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 -> Peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Couldn't send request to peer 8335d0003eb04bb89225bc768e89326f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:41219: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:51.390254 12563 consensus_peers.cc:597] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 -> Peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Couldn't send request to peer 8335d0003eb04bb89225bc768e89326f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:41219: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:51.434433  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:51.436324  1517 tablet_replica.cc:333] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.437986  1517 raft_consensus.cc:2243] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.439020  1517 raft_consensus.cc:2272] T 64a4b7b1c2c448b7bd06ab5503bd2d64 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.442832  1517 tablet_replica.cc:333] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.443662  1517 raft_consensus.cc:2243] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.444806  1517 raft_consensus.cc:2272] T d50730b956af4e7c97163f9383c86b44 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.447504  1517 tablet_replica.cc:333] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.448225  1517 raft_consensus.cc:2243] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.448863  1517 raft_consensus.cc:2272] T 08b989081c2847d1a8b5544686b68dd9 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.451943  1517 tablet_replica.cc:333] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.452654  1517 raft_consensus.cc:2243] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.453320  1517 raft_consensus.cc:2272] T dac711d68a3243a5947623fd84723fba P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.456105  1517 tablet_replica.cc:333] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.456841  1517 raft_consensus.cc:2243] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.457428  1517 raft_consensus.cc:2272] T f32c1c4f8e5a4f4d9314d263fbb06e31 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.460561  1517 tablet_replica.cc:333] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.461463  1517 raft_consensus.cc:2243] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.462292  1517 raft_consensus.cc:2272] T 64e61e3600734c4ea12ee399fa362a4e P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.465365  1517 tablet_replica.cc:333] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.466249  1517 raft_consensus.cc:2243] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.467248  1517 raft_consensus.cc:2272] T 63f764b58817471d902e9f11db53e854 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.471367  1517 tablet_replica.cc:333] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.472579  1517 raft_consensus.cc:2243] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:26:51.472640 12563 consensus_peers.cc:597] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 -> Peer 8335d0003eb04bb89225bc768e89326f (127.1.123.66:41219): Couldn't send request to peer 8335d0003eb04bb89225bc768e89326f. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:41219: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:51.473448  1517 raft_consensus.cc:2272] T d01036a8c7c14415a05dfacc65b7b920 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.476470  1517 tablet_replica.cc:333] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.477175  1517 raft_consensus.cc:2243] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.477847  1517 raft_consensus.cc:2272] T 58e62c187bd14732a717bb655247ae77 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.480683  1517 tablet_replica.cc:333] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.481374  1517 raft_consensus.cc:2243] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.482518  1517 raft_consensus.cc:2272] T 5623b2867b034017baf0e8f69807eb2c P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.485245  1517 tablet_replica.cc:333] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.486008  1517 raft_consensus.cc:2243] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.487089  1517 raft_consensus.cc:2272] T 668bfcf0c9b54599b87a77fab1322ee1 P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.489881  1517 tablet_replica.cc:333] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.490499  1517 raft_consensus.cc:2243] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.491068  1517 raft_consensus.cc:2272] T eb15df4e9b1e44bd9b2dea30b4e72e66 P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.493610  1517 tablet_replica.cc:333] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.494266  1517 raft_consensus.cc:2243] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.495533  1517 raft_consensus.cc:2272] T 673ec45ffbab46d3ba81b5680b57e025 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.499022  1517 tablet_replica.cc:333] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.499781  1517 raft_consensus.cc:2243] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.500360  1517 raft_consensus.cc:2272] T ba28428998c24aee8321c9ecc9ce919a P e63758d19a9946fbb92b90abc13ba7e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.503551  1517 tablet_replica.cc:333] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.504290  1517 raft_consensus.cc:2243] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.505378  1517 raft_consensus.cc:2272] T c219abe0ae174078b653a16da6f547f0 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.508417  1517 tablet_replica.cc:333] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.509227  1517 raft_consensus.cc:2243] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.510236  1517 raft_consensus.cc:2272] T fc4e9f70485d46bfb19ac098241535dc P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.513293  1517 tablet_replica.cc:333] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.514160  1517 raft_consensus.cc:2243] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:51.514758  1517 raft_consensus.cc:2272] T 3473a20c790f484b970e807625ce16cf P e63758d19a9946fbb92b90abc13ba7e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.518426  1517 tablet_replica.cc:333] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5: stopping tablet replica
I20260709 12:26:51.519135  1517 raft_consensus.cc:2243] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.520396  1517 raft_consensus.cc:2272] T ab58c31975ba419dacc885725f47efc7 P e63758d19a9946fbb92b90abc13ba7e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.572216  1517 tablet_server.cc:196] TabletServer@127.1.123.67:0 shutdown complete.
I20260709 12:26:51.627562  1517 master.cc:562] Master@127.1.123.126:43647 shutting down...
I20260709 12:26:51.675045  1517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:51.675889  1517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:51.676358  1517 tablet_replica.cc:333] T 00000000000000000000000000000000 P f46c096e14be41819c4abf3bda382805: stopping tablet replica
I20260709 12:26:51.702694  1517 master.cc:584] Master@127.1.123.126:43647 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (24424 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:26:51.775679  1517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.123.126:36381
I20260709 12:26:51.777132  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:51.784670 12982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:51.784833 12981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:51.785573 12984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:51.786226  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:51.787377  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:51.787576  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:51.787783  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783600011787748 us; error 0 us; skew 500 ppm
I20260709 12:26:51.788405  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:51.791361  1517 webserver.cc:533] Webserver started at http://127.1.123.126:46381/ using document root <none> and password file <none>
I20260709 12:26:51.791915  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:51.792130  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:51.792438  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:51.793764  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/master-0-root/instance:
uuid: "e25b1bc1a624479fbf4aaf7ccd44f232"
format_stamp: "Formatted at 2026-07-09 12:26:51 on dist-test-slave-08l4"
I20260709 12:26:51.799180  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:26:51.804898 12989 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:51.806259  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:26:51.806740  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/master-0-root
uuid: "e25b1bc1a624479fbf4aaf7ccd44f232"
format_stamp: "Formatted at 2026-07-09 12:26:51 on dist-test-slave-08l4"
I20260709 12:26:51.807096  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:51.828315  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:51.829774  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:51.874725  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.126:36381
I20260709 12:26:51.875056 13040 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.126:36381 every 8 connection(s)
I20260709 12:26:51.879172 13041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:51.893159 13041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232: Bootstrap starting.
I20260709 12:26:51.899300 13041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:51.904915 13041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232: No bootstrap required, opened a new log
I20260709 12:26:51.907907 13041 raft_consensus.cc:359] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25b1bc1a624479fbf4aaf7ccd44f232" member_type: VOTER }
I20260709 12:26:51.908429 13041 raft_consensus.cc:385] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:51.908782 13041 raft_consensus.cc:740] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e25b1bc1a624479fbf4aaf7ccd44f232, State: Initialized, Role: FOLLOWER
I20260709 12:26:51.909514 13041 consensus_queue.cc:260] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25b1bc1a624479fbf4aaf7ccd44f232" member_type: VOTER }
I20260709 12:26:51.910024 13041 raft_consensus.cc:399] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:26:51.910223 13041 raft_consensus.cc:493] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:26:51.910454 13041 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:51.916815 13041 raft_consensus.cc:515] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25b1bc1a624479fbf4aaf7ccd44f232" member_type: VOTER }
I20260709 12:26:51.917459 13041 leader_election.cc:304] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e25b1bc1a624479fbf4aaf7ccd44f232; no voters: 
I20260709 12:26:51.918900 13041 leader_election.cc:290] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:26:51.919512 13044 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:51.921211 13044 raft_consensus.cc:697] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [term 1 LEADER]: Becoming Leader. State: Replica: e25b1bc1a624479fbf4aaf7ccd44f232, State: Running, Role: LEADER
I20260709 12:26:51.921986 13044 consensus_queue.cc:237] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25b1bc1a624479fbf4aaf7ccd44f232" member_type: VOTER }
I20260709 12:26:51.922669 13041 sys_catalog.cc:565] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:26:51.927409 13046 sys_catalog.cc:455] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e25b1bc1a624479fbf4aaf7ccd44f232. Latest consensus state: current_term: 1 leader_uuid: "e25b1bc1a624479fbf4aaf7ccd44f232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25b1bc1a624479fbf4aaf7ccd44f232" member_type: VOTER } }
I20260709 12:26:51.928362 13046 sys_catalog.cc:458] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [sys.catalog]: This master's current role is: LEADER
I20260709 12:26:51.927408 13045 sys_catalog.cc:455] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e25b1bc1a624479fbf4aaf7ccd44f232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25b1bc1a624479fbf4aaf7ccd44f232" member_type: VOTER } }
I20260709 12:26:51.929917 13045 sys_catalog.cc:458] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [sys.catalog]: This master's current role is: LEADER
I20260709 12:26:51.935397 13051 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:26:51.943157 13051 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:26:51.945402  1517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:26:51.955996 13051 catalog_manager.cc:1383] Generated new cluster ID: 8cddb19a8cb241b0ad22097d03aad2cb
I20260709 12:26:51.956399 13051 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:26:51.980331 13051 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:26:51.981901 13051 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:26:51.996816 13051 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232: Generated new TSK 0
I20260709 12:26:51.997489 13051 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:26:52.013823  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:52.021962 13062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:52.023077 13063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:52.025424 13065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:52.025956  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:52.027228  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:52.027456  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:52.027678  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783600012027659 us; error 0 us; skew 500 ppm
I20260709 12:26:52.028278  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:52.031739  1517 webserver.cc:533] Webserver started at http://127.1.123.65:40491/ using document root <none> and password file <none>
I20260709 12:26:52.032421  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:52.032645  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:52.032989  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:52.034543  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-0-root/instance:
uuid: "b0fe69a669ba4dc6b2724c4cff489927"
format_stamp: "Formatted at 2026-07-09 12:26:52 on dist-test-slave-08l4"
I20260709 12:26:52.040565  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:26:52.046631 13070 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:52.047941  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:26:52.048378  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-0-root
uuid: "b0fe69a669ba4dc6b2724c4cff489927"
format_stamp: "Formatted at 2026-07-09 12:26:52 on dist-test-slave-08l4"
I20260709 12:26:52.048681  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:52.074575  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:52.076545  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:52.078526  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:52.081463  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:52.081794  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:26:52.082123  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:52.082314  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:52.141114  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.65:38141
I20260709 12:26:52.141388 13132 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.65:38141 every 8 connection(s)
I20260709 12:26:52.148046  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:52.158587 13137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:52.160419 13138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:52.162478 13140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:52.170159 13133 heartbeater.cc:344] Connected to a master server at 127.1.123.126:36381
I20260709 12:26:52.170923 13133 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:52.171267  1517 server_base.cc:1061] running on GCE node
I20260709 12:26:52.172220 13133 heartbeater.cc:507] Master 127.1.123.126:36381 requested a full tablet report, sending...
I20260709 12:26:52.172768  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:52.173069  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:52.173281  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783600012173242 us; error 0 us; skew 500 ppm
I20260709 12:26:52.174018  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:52.175556 13006 ts_manager.cc:194] Registered new tserver with Master: b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141)
I20260709 12:26:52.177692  1517 webserver.cc:533] Webserver started at http://127.1.123.66:41707/ using document root <none> and password file <none>
I20260709 12:26:52.178519  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:52.178630 13006 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54284
I20260709 12:26:52.178793  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:52.179286  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:52.180985  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-1-root/instance:
uuid: "7092cb56813748718b688edaee860d84"
format_stamp: "Formatted at 2026-07-09 12:26:52 on dist-test-slave-08l4"
I20260709 12:26:52.187156  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:26:52.193039 13145 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:52.195099  1517 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.002s	sys 0.000s
I20260709 12:26:52.195580  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-1-root
uuid: "7092cb56813748718b688edaee860d84"
format_stamp: "Formatted at 2026-07-09 12:26:52 on dist-test-slave-08l4"
I20260709 12:26:52.195921  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:52.211723  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:52.213229  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:52.215246  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:52.219022  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:52.219316  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:52.219642  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:52.219854  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:52.286792  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.66:40767
I20260709 12:26:52.286943 13207 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.66:40767 every 8 connection(s)
I20260709 12:26:52.293356  1517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:52.303077 13211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:52.304996 13212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:52.308151  1517 server_base.cc:1061] running on GCE node
W20260709 12:26:52.309306 13214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:52.310174 13208 heartbeater.cc:344] Connected to a master server at 127.1.123.126:36381
I20260709 12:26:52.310765 13208 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:52.310834  1517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:52.311201  1517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:52.311396  1517 hybrid_clock.cc:648] HybridClock initialized: now 1783600012311377 us; error 0 us; skew 500 ppm
I20260709 12:26:52.311730 13208 heartbeater.cc:507] Master 127.1.123.126:36381 requested a full tablet report, sending...
I20260709 12:26:52.312161  1517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:52.314783 13006 ts_manager.cc:194] Registered new tserver with Master: 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:52.316069  1517 webserver.cc:533] Webserver started at http://127.1.123.67:44459/ using document root <none> and password file <none>
I20260709 12:26:52.316908  1517 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:52.317183  1517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:52.317128 13006 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54300
I20260709 12:26:52.317899  1517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:52.319705  1517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-2-root/instance:
uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
format_stamp: "Formatted at 2026-07-09 12:26:52 on dist-test-slave-08l4"
I20260709 12:26:52.326596  1517 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:26:52.332176 13219 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:52.333351  1517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:26:52.333886  1517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-2-root
uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
format_stamp: "Formatted at 2026-07-09 12:26:52 on dist-test-slave-08l4"
I20260709 12:26:52.334342  1517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskckrDUT/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599626433458-1517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:52.379870  1517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:52.381232  1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:52.382982  1517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:52.385449  1517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:52.385792  1517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:26:52.386054  1517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:52.386236  1517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:52.443550  1517 rpc_server.cc:307] RPC server started. Bound to: 127.1.123.67:45715
I20260709 12:26:52.443703 13281 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.123.67:45715 every 8 connection(s)
I20260709 12:26:52.462231 13282 heartbeater.cc:344] Connected to a master server at 127.1.123.126:36381
I20260709 12:26:52.463218 13282 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:52.464238 13282 heartbeater.cc:507] Master 127.1.123.126:36381 requested a full tablet report, sending...
I20260709 12:26:52.467245 13006 ts_manager.cc:194] Registered new tserver with Master: 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:52.467619  1517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019857066s
I20260709 12:26:52.469877 13006 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54306
I20260709 12:26:52.504446 13006 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54320:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:26:52.508074 13006 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:26:52.620961 13173 tablet_service.cc:1511] Processing CreateTablet for tablet 82ccf5e5e1b14ff98884e5649137d885 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:52.621183 13172 tablet_service.cc:1511] Processing CreateTablet for tablet d7a129a1631a44b8a61c38b7c03a306f (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:52.621882 13171 tablet_service.cc:1511] Processing CreateTablet for tablet 3671b696feea45ef91c742b4bed616b3 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:52.622881 13173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ccf5e5e1b14ff98884e5649137d885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.623687 13172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7a129a1631a44b8a61c38b7c03a306f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.624809 13171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3671b696feea45ef91c742b4bed616b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.629874 13247 tablet_service.cc:1511] Processing CreateTablet for tablet 82ccf5e5e1b14ff98884e5649137d885 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:52.631647 13247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ccf5e5e1b14ff98884e5649137d885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.633816 13170 tablet_service.cc:1511] Processing CreateTablet for tablet aeb326e9f81d4120a8038a64b19111a2 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:52.635440 13170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeb326e9f81d4120a8038a64b19111a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.638448 13169 tablet_service.cc:1511] Processing CreateTablet for tablet bdb2a661794848db8d2c900fee23db77 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:52.640241 13169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdb2a661794848db8d2c900fee23db77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.644452 13168 tablet_service.cc:1511] Processing CreateTablet for tablet 10e5c548e4b14054b2f10d23bbd108bf (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:52.646520 13168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e5c548e4b14054b2f10d23bbd108bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.648980 13246 tablet_service.cc:1511] Processing CreateTablet for tablet d7a129a1631a44b8a61c38b7c03a306f (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:52.650749 13246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7a129a1631a44b8a61c38b7c03a306f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.651844 13167 tablet_service.cc:1511] Processing CreateTablet for tablet 362419b9df1c495ca8c9b744399a3ab1 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:52.653823 13167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362419b9df1c495ca8c9b744399a3ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.660430 13245 tablet_service.cc:1511] Processing CreateTablet for tablet 3671b696feea45ef91c742b4bed616b3 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:52.662290 13245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3671b696feea45ef91c742b4bed616b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.669307 13096 tablet_service.cc:1511] Processing CreateTablet for tablet 3671b696feea45ef91c742b4bed616b3 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:52.669479 13097 tablet_service.cc:1511] Processing CreateTablet for tablet d7a129a1631a44b8a61c38b7c03a306f (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:52.671419 13096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3671b696feea45ef91c742b4bed616b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.671109 13098 tablet_service.cc:1511] Processing CreateTablet for tablet 82ccf5e5e1b14ff98884e5649137d885 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:52.672290 13097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7a129a1631a44b8a61c38b7c03a306f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.674000 13098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ccf5e5e1b14ff98884e5649137d885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.675276 13094 tablet_service.cc:1511] Processing CreateTablet for tablet bdb2a661794848db8d2c900fee23db77 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:52.675741 13092 tablet_service.cc:1511] Processing CreateTablet for tablet 362419b9df1c495ca8c9b744399a3ab1 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:52.678246 13093 tablet_service.cc:1511] Processing CreateTablet for tablet 10e5c548e4b14054b2f10d23bbd108bf (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:52.678598 13243 tablet_service.cc:1511] Processing CreateTablet for tablet bdb2a661794848db8d2c900fee23db77 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:52.680434 13243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdb2a661794848db8d2c900fee23db77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.680991 13095 tablet_service.cc:1511] Processing CreateTablet for tablet aeb326e9f81d4120a8038a64b19111a2 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:52.683008 13242 tablet_service.cc:1511] Processing CreateTablet for tablet 10e5c548e4b14054b2f10d23bbd108bf (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:52.685117 13242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e5c548e4b14054b2f10d23bbd108bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.687211 13241 tablet_service.cc:1511] Processing CreateTablet for tablet 362419b9df1c495ca8c9b744399a3ab1 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:52.687835 13166 tablet_service.cc:1511] Processing CreateTablet for tablet 9676eeaaa50f4a2eba78eccb6c45d70a (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:52.675767 13244 tablet_service.cc:1511] Processing CreateTablet for tablet aeb326e9f81d4120a8038a64b19111a2 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:52.689512 13166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9676eeaaa50f4a2eba78eccb6c45d70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.692629 13244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeb326e9f81d4120a8038a64b19111a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.696926 13241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362419b9df1c495ca8c9b744399a3ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.710530 13240 tablet_service.cc:1511] Processing CreateTablet for tablet 9676eeaaa50f4a2eba78eccb6c45d70a (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:52.712563 13240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9676eeaaa50f4a2eba78eccb6c45d70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.714011 13095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeb326e9f81d4120a8038a64b19111a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.718217 13165 tablet_service.cc:1511] Processing CreateTablet for tablet f27af1b1cca9454e9e1c1feca2bb72f4 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:52.720304 13165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27af1b1cca9454e9e1c1feca2bb72f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.722760 13094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdb2a661794848db8d2c900fee23db77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.728505 13093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e5c548e4b14054b2f10d23bbd108bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.734182 13092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362419b9df1c495ca8c9b744399a3ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.736241 13091 tablet_service.cc:1511] Processing CreateTablet for tablet 9676eeaaa50f4a2eba78eccb6c45d70a (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:52.737882 13090 tablet_service.cc:1511] Processing CreateTablet for tablet f27af1b1cca9454e9e1c1feca2bb72f4 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:52.739127 13091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9676eeaaa50f4a2eba78eccb6c45d70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.740221 13090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27af1b1cca9454e9e1c1feca2bb72f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.747571 13300 tablet_bootstrap.cc:492] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:52.752125 13239 tablet_service.cc:1511] Processing CreateTablet for tablet f27af1b1cca9454e9e1c1feca2bb72f4 (DEFAULT_TABLE table=test-workload [id=7d13119ea964411399400a105bb2d7de]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:52.774859 13239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27af1b1cca9454e9e1c1feca2bb72f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:52.881239 13301 tablet_bootstrap.cc:492] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:52.891916 13301 tablet_bootstrap.cc:654] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:52.896472 13300 tablet_bootstrap.cc:654] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:52.915094 13301 tablet_bootstrap.cc:492] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:52.916018 13301 ts_tablet_manager.cc:1403] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.011s
I20260709 12:26:52.927584 13301 raft_consensus.cc:359] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:52.928552 13301 raft_consensus.cc:385] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:52.928890 13301 raft_consensus.cc:740] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:52.930158 13301 consensus_queue.cc:260] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:52.931062 13302 tablet_bootstrap.cc:492] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:52.934123 13133 heartbeater.cc:499] Master 127.1.123.126:36381 was elected leader, sending a full tablet report...
I20260709 12:26:52.935494 13301 ts_tablet_manager.cc:1434] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.018s	user 0.014s	sys 0.001s
W20260709 12:26:52.938848 13054 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:52.941075 13301 tablet_bootstrap.cc:492] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:52.952116 13300 tablet_bootstrap.cc:492] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:52.952688 13302 tablet_bootstrap.cc:654] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:52.953094 13300 ts_tablet_manager.cc:1403] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.206s	user 0.012s	sys 0.007s
I20260709 12:26:52.953212 13301 tablet_bootstrap.cc:654] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:52.958333 13300 raft_consensus.cc:359] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:52.959410 13300 raft_consensus.cc:385] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:52.959806 13300 raft_consensus.cc:740] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:52.961316 13300 consensus_queue.cc:260] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:52.965822 13302 tablet_bootstrap.cc:492] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:52.970112 13302 ts_tablet_manager.cc:1403] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.004s
I20260709 12:26:52.973941 13301 tablet_bootstrap.cc:492] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:52.974642 13301 ts_tablet_manager.cc:1403] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.034s	user 0.025s	sys 0.003s
I20260709 12:26:52.974548 13302 raft_consensus.cc:359] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:52.976032 13302 raft_consensus.cc:385] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:52.976449 13208 heartbeater.cc:499] Master 127.1.123.126:36381 was elected leader, sending a full tablet report...
I20260709 12:26:52.976454 13302 raft_consensus.cc:740] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:52.978039 13300 ts_tablet_manager.cc:1434] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.024s	user 0.016s	sys 0.000s
I20260709 12:26:52.978248 13302 consensus_queue.cc:260] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:52.979588 13300 tablet_bootstrap.cc:492] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:52.980355 13301 raft_consensus.cc:359] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:52.981835 13301 raft_consensus.cc:385] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:52.982369 13301 raft_consensus.cc:740] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:52.983893 13301 consensus_queue.cc:260] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:52.986984 13301 ts_tablet_manager.cc:1434] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.004s
I20260709 12:26:52.988216 13301 tablet_bootstrap.cc:492] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:52.990514 13300 tablet_bootstrap.cc:654] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:52.996850 13301 tablet_bootstrap.cc:654] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:52.997641 13302 ts_tablet_manager.cc:1434] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.027s	user 0.014s	sys 0.004s
I20260709 12:26:52.998994 13282 heartbeater.cc:499] Master 127.1.123.126:36381 was elected leader, sending a full tablet report...
I20260709 12:26:53.007189 13302 tablet_bootstrap.cc:492] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:53.017953 13300 tablet_bootstrap.cc:492] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:53.018139 13302 tablet_bootstrap.cc:654] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.019075 13300 ts_tablet_manager.cc:1403] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.040s	user 0.022s	sys 0.002s
I20260709 12:26:53.023927 13300 raft_consensus.cc:359] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.025195 13300 raft_consensus.cc:385] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.026705 13300 raft_consensus.cc:740] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.028957 13300 consensus_queue.cc:260] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.032887 13300 ts_tablet_manager.cc:1434] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.013s	user 0.013s	sys 0.000s
I20260709 12:26:53.034762 13300 tablet_bootstrap.cc:492] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:53.034619 13302 tablet_bootstrap.cc:492] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:53.034684 13301 tablet_bootstrap.cc:492] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:53.035998 13302 ts_tablet_manager.cc:1403] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.012s
I20260709 12:26:53.036129 13301 ts_tablet_manager.cc:1403] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.048s	user 0.026s	sys 0.020s
I20260709 12:26:53.041939 13301 raft_consensus.cc:359] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:53.042356 13302 raft_consensus.cc:359] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.043867 13301 raft_consensus.cc:385] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:26:53.044724 13209 tablet.cc:2406] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:26:53.044806 13302 raft_consensus.cc:385] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.045279 13301 raft_consensus.cc:740] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.045859 13302 raft_consensus.cc:740] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.046412 13300 tablet_bootstrap.cc:654] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.047832 13301 consensus_queue.cc:260] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:53.047832 13302 consensus_queue.cc:260] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.050997 13302 ts_tablet_manager.cc:1434] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 12:26:53.052255 13302 tablet_bootstrap.cc:492] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:53.054072 13301 ts_tablet_manager.cc:1434] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 12:26:53.055621 13301 tablet_bootstrap.cc:492] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:53.062294 13302 tablet_bootstrap.cc:654] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.063838 13300 tablet_bootstrap.cc:492] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:53.065008 13300 ts_tablet_manager.cc:1403] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.009s
I20260709 12:26:53.071115 13300 raft_consensus.cc:359] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.072238 13300 raft_consensus.cc:385] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.072760 13300 raft_consensus.cc:740] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.073284 13301 tablet_bootstrap.cc:654] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.074156 13300 consensus_queue.cc:260] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.082921 13300 ts_tablet_manager.cc:1434] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 12:26:53.084502 13300 tablet_bootstrap.cc:492] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:53.093995 13301 tablet_bootstrap.cc:492] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:53.094417 13300 tablet_bootstrap.cc:654] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.095135 13301 ts_tablet_manager.cc:1403] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.040s	user 0.023s	sys 0.007s
I20260709 12:26:53.102258 13301 raft_consensus.cc:359] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.103108 13301 raft_consensus.cc:385] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.103397 13301 raft_consensus.cc:740] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.104233 13301 consensus_queue.cc:260] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.105115 13302 tablet_bootstrap.cc:492] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:53.105937 13302 ts_tablet_manager.cc:1403] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.054s	user 0.023s	sys 0.025s
I20260709 12:26:53.110134 13301 ts_tablet_manager.cc:1434] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 12:26:53.112043 13301 tablet_bootstrap.cc:492] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:53.111814 13302 raft_consensus.cc:359] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:53.113306 13302 raft_consensus.cc:385] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.113828 13302 raft_consensus.cc:740] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.115309 13302 consensus_queue.cc:260] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:53.125195 13302 ts_tablet_manager.cc:1434] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.019s	user 0.012s	sys 0.001s
I20260709 12:26:53.127218 13302 tablet_bootstrap.cc:492] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:53.127799 13301 tablet_bootstrap.cc:654] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.139945 13302 tablet_bootstrap.cc:654] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.142659 13301 tablet_bootstrap.cc:492] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:53.145267 13301 ts_tablet_manager.cc:1403] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.033s	user 0.021s	sys 0.004s
I20260709 12:26:53.150727 13308 raft_consensus.cc:493] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.151500 13308 raft_consensus.cc:515] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.152575 13301 raft_consensus.cc:359] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.153514 13301 raft_consensus.cc:385] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.153846 13301 raft_consensus.cc:740] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.155882 13301 consensus_queue.cc:260] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.157691 13300 tablet_bootstrap.cc:492] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:53.159514 13300 ts_tablet_manager.cc:1403] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.075s	user 0.024s	sys 0.040s
I20260709 12:26:53.160415 13301 ts_tablet_manager.cc:1434] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.003s
I20260709 12:26:53.163508 13301 tablet_bootstrap.cc:492] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:53.165069 13300 raft_consensus.cc:359] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.168479 13308 leader_election.cc:290] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:53.174507 13300 raft_consensus.cc:385] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.178051 13301 tablet_bootstrap.cc:654] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.177982 13309 raft_consensus.cc:493] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.180605 13300 raft_consensus.cc:740] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.181344 13309 raft_consensus.cc:515] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.184139 13300 consensus_queue.cc:260] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.189720 13309 leader_election.cc:290] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:53.195405 13300 ts_tablet_manager.cc:1434] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.034s	user 0.018s	sys 0.001s
I20260709 12:26:53.198323 13300 tablet_bootstrap.cc:492] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:53.179883 13302 tablet_bootstrap.cc:492] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:53.199436 13302 ts_tablet_manager.cc:1403] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.073s	user 0.016s	sys 0.027s
I20260709 12:26:53.208421 13308 raft_consensus.cc:493] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.210417 13308 raft_consensus.cc:515] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.215070 13308 leader_election.cc:290] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:53.223035 13300 tablet_bootstrap.cc:654] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.215706 13302 raft_consensus.cc:359] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.229821 13302 raft_consensus.cc:385] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.230481 13302 raft_consensus.cc:740] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.231616 13302 consensus_queue.cc:260] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.237947 13301 tablet_bootstrap.cc:492] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:53.238822 13301 ts_tablet_manager.cc:1403] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.076s	user 0.026s	sys 0.020s
I20260709 12:26:53.238981 13302 ts_tablet_manager.cc:1434] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.039s	user 0.011s	sys 0.000s
I20260709 12:26:53.240516 13302 tablet_bootstrap.cc:492] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:53.243932 13301 raft_consensus.cc:359] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.244781 13301 raft_consensus.cc:385] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.245107 13301 raft_consensus.cc:740] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.245337 13108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:53.245252 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:53.248271 13108 raft_consensus.cc:2468] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:53.248874 13183 raft_consensus.cc:2468] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:53.250586 13223 leader_election.cc:304] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:53.246134 13301 consensus_queue.cc:260] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.253868 13302 tablet_bootstrap.cc:654] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.255445 13301 ts_tablet_manager.cc:1434] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:26:53.256036 13309 raft_consensus.cc:2804] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:53.256546 13309 raft_consensus.cc:493] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.256929 13309 raft_consensus.cc:3060] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.257095 13301 tablet_bootstrap.cc:492] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:53.267904 13301 tablet_bootstrap.cc:654] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.278810 13309 raft_consensus.cc:515] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.284643 13309 leader_election.cc:290] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:53.285027 13108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:53.287272 13108 raft_consensus.cc:2468] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:53.286258 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:53.286388 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:53.289037 13183 raft_consensus.cc:3060] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.293449 13300 tablet_bootstrap.cc:492] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:53.295823 13149 leader_election.cc:304] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:53.305145 13300 ts_tablet_manager.cc:1403] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.107s	user 0.028s	sys 0.007s
W20260709 12:26:53.305501 13149 leader_election.cc:343] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:53.298534 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:53.302223 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:53.306660 13107 raft_consensus.cc:3060] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.303056 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:53.308462 13257 raft_consensus.cc:2393] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7092cb56813748718b688edaee860d84 in current term 1: Already voted for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in this term.
W20260709 12:26:53.308535 13148 leader_election.cc:343] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:53.309230 13148 leader_election.cc:304] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7092cb56813748718b688edaee860d84; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:53.304190 13302 tablet_bootstrap.cc:492] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:53.311426 13302 ts_tablet_manager.cc:1403] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.071s	user 0.018s	sys 0.022s
I20260709 12:26:53.312278 13329 raft_consensus.cc:2749] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:53.306023 13183 raft_consensus.cc:2468] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:53.313422 13308 raft_consensus.cc:2764] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:26:53.314577 13309 raft_consensus.cc:493] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.315264 13309 raft_consensus.cc:515] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.315866 13329 raft_consensus.cc:493] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.317891 13305 raft_consensus.cc:493] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.318629 13300 raft_consensus.cc:359] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.320463 13108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb326e9f81d4120a8038a64b19111a2" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:53.321026 13300 raft_consensus.cc:385] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.321524 13108 raft_consensus.cc:2468] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:53.321705 13300 raft_consensus.cc:740] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.319940 13329 raft_consensus.cc:515] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.321957 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb326e9f81d4120a8038a64b19111a2" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:53.319196 13305 raft_consensus.cc:515] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.323946 13309 leader_election.cc:290] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:53.324255 13223 leader_election.cc:304] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:53.325304 13329 leader_election.cc:290] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:53.326915 13302 raft_consensus.cc:359] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.328302 13302 raft_consensus.cc:385] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.328327 13330 raft_consensus.cc:493] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.328742 13302 raft_consensus.cc:740] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.329135 13330 raft_consensus.cc:515] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.329990 13302 consensus_queue.cc:260] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.329169 13108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e5c548e4b14054b2f10d23bbd108bf" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:53.331198 13107 raft_consensus.cc:2468] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:53.331431 13108 raft_consensus.cc:2468] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:53.333146 13149 leader_election.cc:304] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:53.334276 13330 leader_election.cc:290] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:53.334786 13329 raft_consensus.cc:2804] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:53.333146 13223 leader_election.cc:304] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:53.339574 13305 leader_election.cc:290] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:53.336848 13329 raft_consensus.cc:493] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.342206 13329 raft_consensus.cc:3060] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.338342 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e5c548e4b14054b2f10d23bbd108bf" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:53.346469 13301 tablet_bootstrap.cc:492] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:53.347738 13301 ts_tablet_manager.cc:1403] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.091s	user 0.028s	sys 0.007s
W20260709 12:26:53.348407 13148 leader_election.cc:343] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:53.350948 13333 raft_consensus.cc:2804] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:53.352450 13333 raft_consensus.cc:697] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Leader. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:26:53.326522 13309 raft_consensus.cc:2804] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:53.353786 13309 raft_consensus.cc:493] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.354810 13309 raft_consensus.cc:3060] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.354619 13333 consensus_queue.cc:237] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.361495 13302 ts_tablet_manager.cc:1434] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.050s	user 0.021s	sys 0.014s
I20260709 12:26:53.364625 13301 raft_consensus.cc:359] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.370891 13302 tablet_bootstrap.cc:492] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:53.369899 13301 raft_consensus.cc:385] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.374971 13301 raft_consensus.cc:740] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.376060 13301 consensus_queue.cc:260] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.378060 13300 consensus_queue.cc:260] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.385851 13182 raft_consensus.cc:2468] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:53.390122 13301 ts_tablet_manager.cc:1434] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.042s	user 0.014s	sys 0.000s
I20260709 12:26:53.391891 13301 tablet_bootstrap.cc:492] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:53.395578 13300 ts_tablet_manager.cc:1434] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.090s	user 0.015s	sys 0.004s
I20260709 12:26:53.396591 13302 tablet_bootstrap.cc:654] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.397403 13300 tablet_bootstrap.cc:492] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:53.406419 13309 raft_consensus.cc:515] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.406549 13329 raft_consensus.cc:515] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.420892 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb326e9f81d4120a8038a64b19111a2" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:53.420892 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e5c548e4b14054b2f10d23bbd108bf" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:53.422636 13182 raft_consensus.cc:3060] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.422878 13107 raft_consensus.cc:3060] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.430483 13309 leader_election.cc:290] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:53.432358 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e5c548e4b14054b2f10d23bbd108bf" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
W20260709 12:26:53.434923 13148 leader_election.cc:343] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:53.436087 13329 leader_election.cc:290] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:53.440200 13108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb326e9f81d4120a8038a64b19111a2" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:53.441881 13108 raft_consensus.cc:3060] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.442662 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9676eeaaa50f4a2eba78eccb6c45d70a" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:53.452970 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3671b696feea45ef91c742b4bed616b3" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:53.459529 13329 raft_consensus.cc:493] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.460497 13329 raft_consensus.cc:515] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:53.466861 13182 raft_consensus.cc:2468] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:53.467625 13108 raft_consensus.cc:2468] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:53.467185 13107 raft_consensus.cc:2468] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:53.473951 13222 leader_election.cc:304] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
W20260709 12:26:53.459530 13073 leader_election.cc:343] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:53.476616 13300 tablet_bootstrap.cc:654] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.478324 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ccf5e5e1b14ff98884e5649137d885" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:53.478814 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ccf5e5e1b14ff98884e5649137d885" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:53.478106 13004 catalog_manager.cc:5697] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.483934 13257 raft_consensus.cc:2468] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:53.483934 13106 raft_consensus.cc:2468] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:53.479506 13329 leader_election.cc:290] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715), b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141)
I20260709 12:26:53.478826 13302 tablet_bootstrap.cc:492] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:53.487109 13302 ts_tablet_manager.cc:1403] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.117s	user 0.017s	sys 0.012s
I20260709 12:26:53.482267 13149 leader_election.cc:304] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99
W20260709 12:26:53.482409 13073 leader_election.cc:343] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:53.482450 13333 raft_consensus.cc:2804] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:53.488127 13333 raft_consensus.cc:697] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Leader. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:26:53.490279 13333 consensus_queue.cc:237] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.486001 13148 leader_election.cc:304] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
I20260709 12:26:53.469048 13301 tablet_bootstrap.cc:654] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.492724 13300 tablet_bootstrap.cc:492] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:53.493213 13308 raft_consensus.cc:2804] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:53.494850 13308 raft_consensus.cc:493] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.495718 13308 raft_consensus.cc:3060] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.496833 13300 ts_tablet_manager.cc:1403] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.100s	user 0.014s	sys 0.009s
I20260709 12:26:53.490309 13329 raft_consensus.cc:2804] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:53.504793 13329 raft_consensus.cc:697] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Becoming Leader. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:26:53.508152 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9676eeaaa50f4a2eba78eccb6c45d70a" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:53.507884 13329 consensus_queue.cc:237] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
W20260709 12:26:53.512297 13073 leader_election.cc:343] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:53.513224 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3671b696feea45ef91c742b4bed616b3" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:53.515045 13073 leader_election.cc:304] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0fe69a669ba4dc6b2724c4cff489927; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84
I20260709 12:26:53.515331 13257 raft_consensus.cc:2468] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 0.
I20260709 12:26:53.517028 13305 raft_consensus.cc:2749] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:53.517951 13302 raft_consensus.cc:359] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.520762 13302 raft_consensus.cc:385] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.522938 13302 raft_consensus.cc:740] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.524366 13300 raft_consensus.cc:359] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.525043 13302 consensus_queue.cc:260] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.526959 13301 tablet_bootstrap.cc:492] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:53.525389 13300 raft_consensus.cc:385] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.528107 13300 raft_consensus.cc:740] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.528297 13301 ts_tablet_manager.cc:1403] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.137s	user 0.016s	sys 0.008s
I20260709 12:26:53.528648 13302 ts_tablet_manager.cc:1434] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 12:26:53.529763 13300 consensus_queue.cc:260] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.540923 13302 tablet_bootstrap.cc:492] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:53.520690 13073 leader_election.cc:304] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 7092cb56813748718b688edaee860d84
I20260709 12:26:53.543483 13308 raft_consensus.cc:515] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:53.547067 13305 raft_consensus.cc:2804] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:53.549180 13305 raft_consensus.cc:493] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.549923 13305 raft_consensus.cc:3060] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.560495 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ccf5e5e1b14ff98884e5649137d885" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:53.561461 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ccf5e5e1b14ff98884e5649137d885" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:53.561908 13257 raft_consensus.cc:3060] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.562355 13106 raft_consensus.cc:3060] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.545069 13300 ts_tablet_manager.cc:1434] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.048s	user 0.014s	sys 0.000s
I20260709 12:26:53.544548 13301 raft_consensus.cc:359] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.567584 13301 raft_consensus.cc:385] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.570917 13301 raft_consensus.cc:740] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.567589 13308 leader_election.cc:290] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Requested vote from peers 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715), b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141)
W20260709 12:26:53.569306 13073 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.1.123.67:45715, user_credentials={real_user=slave}} blocked reactor thread for 48018.4us
I20260709 12:26:53.575675 13301 consensus_queue.cc:260] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.578967 13302 tablet_bootstrap.cc:654] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.587182 13005 catalog_manager.cc:5697] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7092cb56813748718b688edaee860d84 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "7092cb56813748718b688edaee860d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:53.590610 13300 tablet_bootstrap.cc:492] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:53.588135 13305 raft_consensus.cc:515] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.593757 13305 leader_election.cc:290] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 election: Requested vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:53.596448 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3671b696feea45ef91c742b4bed616b3" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:53.597254 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3671b696feea45ef91c742b4bed616b3" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:53.588183 13257 raft_consensus.cc:2468] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:53.599200 13301 ts_tablet_manager.cc:1434] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.070s	user 0.016s	sys 0.001s
W20260709 12:26:53.602164 13073 leader_election.cc:343] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:53.605361 13301 tablet_bootstrap.cc:492] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:53.600332 13256 raft_consensus.cc:3060] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.612010 13300 tablet_bootstrap.cc:654] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.615702 13301 tablet_bootstrap.cc:654] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.590636 13106 raft_consensus.cc:2468] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:53.641974 13149 leader_election.cc:304] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:53.642925 13308 raft_consensus.cc:2804] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:53.643523 13308 raft_consensus.cc:697] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Becoming Leader. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:26:53.644568 13308 consensus_queue.cc:237] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:53.646145 13333 raft_consensus.cc:493] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.651897 13300 tablet_bootstrap.cc:492] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:53.652786 13300 ts_tablet_manager.cc:1403] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.062s	user 0.022s	sys 0.019s
I20260709 12:26:53.647511 13333 raft_consensus.cc:515] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.665268 13300 raft_consensus.cc:359] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.666519 13300 raft_consensus.cc:385] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.666822 13302 tablet_bootstrap.cc:492] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:53.666922 13300 raft_consensus.cc:740] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.667935 13302 ts_tablet_manager.cc:1403] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.127s	user 0.032s	sys 0.035s
I20260709 12:26:53.668414 13300 consensus_queue.cc:260] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.672387 13300 ts_tablet_manager.cc:1434] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.019s	user 0.012s	sys 0.000s
I20260709 12:26:53.678136 13305 raft_consensus.cc:493] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.681299 13305 raft_consensus.cc:515] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.681349 13302 raft_consensus.cc:359] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.683629 13302 raft_consensus.cc:385] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.684849 13305 leader_election.cc:290] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:53.685806 13301 tablet_bootstrap.cc:492] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:53.686129 13256 raft_consensus.cc:2468] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 1.
I20260709 12:26:53.686002 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:53.687036 13301 ts_tablet_manager.cc:1403] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.082s	user 0.035s	sys 0.007s
I20260709 12:26:53.687098 13106 raft_consensus.cc:2468] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:53.687716 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27af1b1cca9454e9e1c1feca2bb72f4" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:53.678190 13005 catalog_manager.cc:5697] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.689278 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:53.690220 13183 raft_consensus.cc:2468] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:53.690394 13223 leader_election.cc:304] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:53.691124 13333 leader_election.cc:290] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:53.691664 13073 leader_election.cc:304] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 7092cb56813748718b688edaee860d84
I20260709 12:26:53.692962 13309 raft_consensus.cc:2804] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:53.693521 13309 raft_consensus.cc:493] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.693933 13309 raft_consensus.cc:3060] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.695307 13300 tablet_bootstrap.cc:492] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:53.695063 13301 raft_consensus.cc:359] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.695937 13301 raft_consensus.cc:385] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.696249 13301 raft_consensus.cc:740] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.697122 13301 consensus_queue.cc:260] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.697872 13330 raft_consensus.cc:2804] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:53.698777 13302 raft_consensus.cc:740] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.700095 13302 consensus_queue.cc:260] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
W20260709 12:26:53.702443 13073 leader_election.cc:343] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:53.703150 13302 ts_tablet_manager.cc:1434] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 12:26:53.715481 13302 tablet_bootstrap.cc:492] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:53.703636 13300 tablet_bootstrap.cc:654] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.705622 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27af1b1cca9454e9e1c1feca2bb72f4" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:53.725657 13330 raft_consensus.cc:697] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 LEADER]: Becoming Leader. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Running, Role: LEADER
I20260709 12:26:53.710461 13005 catalog_manager.cc:5697] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7092cb56813748718b688edaee860d84 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "7092cb56813748718b688edaee860d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:53.726876 13073 leader_election.cc:343] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:53.726805 13330 consensus_queue.cc:237] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.727609 13073 leader_election.cc:304] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0fe69a669ba4dc6b2724c4cff489927; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84
I20260709 12:26:53.732259 13302 tablet_bootstrap.cc:654] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.732596 13351 raft_consensus.cc:2749] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:53.739163 13309 raft_consensus.cc:515] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.743331 13301 ts_tablet_manager.cc:1434] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.056s	user 0.020s	sys 0.003s
I20260709 12:26:53.757637 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:53.758831 13106 raft_consensus.cc:3060] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.759930 13302 tablet_bootstrap.cc:492] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:53.760917 13302 ts_tablet_manager.cc:1403] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.003s
I20260709 12:26:53.761181 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:53.762132 13183 raft_consensus.cc:3060] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.765923 13302 raft_consensus.cc:359] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.767021 13302 raft_consensus.cc:385] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.767513 13302 raft_consensus.cc:740] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.768668 13302 consensus_queue.cc:260] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.772096 13302 ts_tablet_manager.cc:1434] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:26:53.772855 13183 raft_consensus.cc:2468] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:53.773775 13302 tablet_bootstrap.cc:492] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:53.773283 13309 leader_election.cc:290] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:53.776684 13222 leader_election.cc:304] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
I20260709 12:26:53.778035 13309 raft_consensus.cc:2804] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:53.779704 13309 raft_consensus.cc:697] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Leader. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:26:53.781014 13309 consensus_queue.cc:237] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.785154 13302 tablet_bootstrap.cc:654] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.785799 13006 catalog_manager.cc:5697] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:53.791059 13106 raft_consensus.cc:2468] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:53.798550 13300 tablet_bootstrap.cc:492] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:53.799337 13300 ts_tablet_manager.cc:1403] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.104s	user 0.020s	sys 0.001s
I20260709 12:26:53.804034 13300 raft_consensus.cc:359] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.805814 13300 raft_consensus.cc:385] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.806502 13300 raft_consensus.cc:740] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.805730 13004 catalog_manager.cc:5697] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.807934 13300 consensus_queue.cc:260] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.814953 13300 ts_tablet_manager.cc:1434] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:26:53.817317 13300 tablet_bootstrap.cc:492] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:53.820152 13302 tablet_bootstrap.cc:492] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:53.821269 13302 ts_tablet_manager.cc:1403] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.048s	user 0.021s	sys 0.021s
I20260709 12:26:53.825726 13300 tablet_bootstrap.cc:654] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:53.826146 13302 raft_consensus.cc:359] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.827951 13302 raft_consensus.cc:385] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.828594 13302 raft_consensus.cc:740] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.829968 13302 consensus_queue.cc:260] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.832512 13302 ts_tablet_manager.cc:1434] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:26:53.833900 13300 tablet_bootstrap.cc:492] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:53.835075 13300 ts_tablet_manager.cc:1403] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 12:26:53.838497 13300 raft_consensus.cc:359] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.839497 13300 raft_consensus.cc:385] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:53.839779 13308 raft_consensus.cc:493] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.840039 13300 raft_consensus.cc:740] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:53.841256 13300 consensus_queue.cc:260] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.843613 13300 ts_tablet_manager.cc:1434] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:26:53.840670 13308 raft_consensus.cc:515] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.848376 13308 leader_election.cc:290] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:53.850765 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9676eeaaa50f4a2eba78eccb6c45d70a" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:53.851212 13309 raft_consensus.cc:493] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.851917 13256 raft_consensus.cc:2468] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:53.852041 13309 raft_consensus.cc:515] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.850765 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9676eeaaa50f4a2eba78eccb6c45d70a" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:53.855623 13106 raft_consensus.cc:2468] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:53.857426 13148 leader_election.cc:304] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
I20260709 12:26:53.859030 13308 raft_consensus.cc:2804] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:53.859534 13308 raft_consensus.cc:493] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.859521 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27af1b1cca9454e9e1c1feca2bb72f4" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:53.859058 13309 leader_election.cc:290] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:53.860739 13308 raft_consensus.cc:3060] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.860996 13183 raft_consensus.cc:2468] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:53.859939 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27af1b1cca9454e9e1c1feca2bb72f4" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:53.862691 13107 raft_consensus.cc:2468] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:53.864820 13223 leader_election.cc:304] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:53.867677 13309 raft_consensus.cc:2804] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:53.868323 13309 raft_consensus.cc:493] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.868738 13309 raft_consensus.cc:3060] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.872591 13333 consensus_queue.cc:1048] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:53.873413 13308 raft_consensus.cc:515] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.877429 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9676eeaaa50f4a2eba78eccb6c45d70a" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:53.877960 13308 leader_election.cc:290] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:53.878181 13107 raft_consensus.cc:3060] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.879305 13309 raft_consensus.cc:515] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.878064 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9676eeaaa50f4a2eba78eccb6c45d70a" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:53.881989 13256 raft_consensus.cc:3060] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.883100 13309 leader_election.cc:290] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:53.886590 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27af1b1cca9454e9e1c1feca2bb72f4" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:53.886956 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27af1b1cca9454e9e1c1feca2bb72f4" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:53.887434 13106 raft_consensus.cc:3060] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.887569 13182 raft_consensus.cc:3060] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.893107 13107 raft_consensus.cc:2468] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:53.895444 13149 leader_election.cc:304] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:53.896977 13308 raft_consensus.cc:2804] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:53.897639 13308 raft_consensus.cc:697] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Becoming Leader. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:26:53.898731 13256 raft_consensus.cc:2468] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:53.898648 13308 consensus_queue.cc:237] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.907650 13106 raft_consensus.cc:2468] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:53.910224 13223 leader_election.cc:304] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:53.911957 13182 raft_consensus.cc:2468] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:53.912581 13333 raft_consensus.cc:2804] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:53.913813 13333 raft_consensus.cc:697] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Leader. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:26:53.918124 13333 consensus_queue.cc:237] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.921729 13331 consensus_queue.cc:1048] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:53.936419 13006 catalog_manager.cc:5697] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7092cb56813748718b688edaee860d84 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "7092cb56813748718b688edaee860d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:53.941726 13331 raft_consensus.cc:493] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.942380 13331 raft_consensus.cc:515] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.944743 13331 leader_election.cc:290] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:53.946197 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e5c548e4b14054b2f10d23bbd108bf" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:53.947425 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e5c548e4b14054b2f10d23bbd108bf" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:53.948351 13106 raft_consensus.cc:2393] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in current term 1: Already voted for candidate 7092cb56813748718b688edaee860d84 in this term.
I20260709 12:26:53.953054 13223 leader_election.cc:304] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99; no voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:53.954317 13333 raft_consensus.cc:3060] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.955798 13004 catalog_manager.cc:5697] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.962786 13308 consensus_queue.cc:1048] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:53.969452 13333 raft_consensus.cc:2749] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:26:53.979838 13054 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:53.982954 13333 raft_consensus.cc:493] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.983754 13333 raft_consensus.cc:515] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:53.987387 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb2a661794848db8d2c900fee23db77" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:53.988575 13183 raft_consensus.cc:2468] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:53.991318 13222 leader_election.cc:304] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
I20260709 12:26:53.991935 13339 consensus_queue.cc:1048] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:53.992254 13108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb2a661794848db8d2c900fee23db77" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:53.992671 13331 raft_consensus.cc:2804] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:53.993191 13331 raft_consensus.cc:493] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.993997 13331 raft_consensus.cc:3060] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:53.993227 13108 raft_consensus.cc:2468] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:54.000784 13330 raft_consensus.cc:493] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.001629 13330 raft_consensus.cc:515] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.003178 13309 consensus_queue.cc:1048] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:54.005961 13331 raft_consensus.cc:515] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.011076 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb2a661794848db8d2c900fee23db77" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:54.011637 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb2a661794848db8d2c900fee23db77" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:54.012123 13183 raft_consensus.cc:3060] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:54.015035 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb2a661794848db8d2c900fee23db77" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:54.016047 13106 raft_consensus.cc:3060] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:54.021721 13331 leader_election.cc:290] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:54.022603 13330 leader_election.cc:290] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:54.024937 13183 raft_consensus.cc:2468] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:54.026255 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb2a661794848db8d2c900fee23db77" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:54.027096 13222 leader_election.cc:304] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
I20260709 12:26:54.027657 13257 raft_consensus.cc:2393] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b0fe69a669ba4dc6b2724c4cff489927 in current term 1: Already voted for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in this term.
I20260709 12:26:54.028847 13331 raft_consensus.cc:2804] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:54.029753 13331 raft_consensus.cc:697] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Leader. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:26:54.029911 13308 raft_consensus.cc:493] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.030855 13308 raft_consensus.cc:515] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.031351 13331 consensus_queue.cc:237] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.001678 13333 leader_election.cc:290] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:54.041726 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3671b696feea45ef91c742b4bed616b3" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:54.042358 13106 raft_consensus.cc:2468] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:54.046947 13073 leader_election.cc:304] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0fe69a669ba4dc6b2724c4cff489927; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84
I20260709 12:26:54.048182 13308 leader_election.cc:290] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:54.049902 13330 raft_consensus.cc:2749] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:54.050608 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3671b696feea45ef91c742b4bed616b3" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:54.051946 13257 raft_consensus.cc:2393] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7092cb56813748718b688edaee860d84 in current term 1: Already voted for candidate b0fe69a669ba4dc6b2724c4cff489927 in this term.
I20260709 12:26:54.053892 13148 leader_election.cc:304] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7092cb56813748718b688edaee860d84; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:54.055780 13308 raft_consensus.cc:3060] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:54.061831 13309 consensus_queue.cc:1048] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:54.073340 13308 raft_consensus.cc:2749] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:54.084748 13004 catalog_manager.cc:5697] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:54.124920 13308 consensus_queue.cc:1048] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:54.142405 13339 consensus_queue.cc:1048] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:54.171861 13309 consensus_queue.cc:1048] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:54.195662 13309 consensus_queue.cc:1048] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:54.207746 13330 consensus_queue.cc:1048] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:26:54.228091 13330 consensus_queue.cc:1048] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:54.298183 13339 consensus_queue.cc:1048] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:54.315721 13378 consensus_queue.cc:1048] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:54.332598 13309 consensus_queue.cc:1048] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:54.346259 13309 consensus_queue.cc:1048] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:54.389254 13183 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82ccf5e5e1b14ff98884e5649137d885"
dest_uuid: "7092cb56813748718b688edaee860d84"
mode: GRACEFUL
new_leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:41588
I20260709 12:26:54.390506 13183 raft_consensus.cc:606] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Received request to transfer leadership to TS b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:54.417972 13256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f"
dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
mode: GRACEFUL
new_leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:42976
I20260709 12:26:54.418931 13256 raft_consensus.cc:606] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Received request to transfer leadership to TS b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:54.435703 13256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aeb326e9f81d4120a8038a64b19111a2"
dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
mode: GRACEFUL
new_leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:42976
I20260709 12:26:54.436457 13256 raft_consensus.cc:606] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Received request to transfer leadership to TS b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:54.449285 13256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdb2a661794848db8d2c900fee23db77"
dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
mode: GRACEFUL
new_leader_uuid: "7092cb56813748718b688edaee860d84"
 from {username='slave'} at 127.0.0.1:42976
I20260709 12:26:54.450330 13256 raft_consensus.cc:606] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Received request to transfer leadership to TS 7092cb56813748718b688edaee860d84
I20260709 12:26:54.463601 13256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1"
dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
mode: GRACEFUL
new_leader_uuid: "7092cb56813748718b688edaee860d84"
 from {username='slave'} at 127.0.0.1:42976
I20260709 12:26:54.464531 13256 raft_consensus.cc:606] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Received request to transfer leadership to TS 7092cb56813748718b688edaee860d84
I20260709 12:26:54.467305 13309 consensus_queue.cc:1048] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:54.477680 13331 raft_consensus.cc:993] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99: : Instructing follower b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:26:54.478276 13331 raft_consensus.cc:1081] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Signalling peer b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:26:54.480553 13369 raft_consensus.cc:993] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99: : Instructing follower 7092cb56813748718b688edaee860d84 to start an election
I20260709 12:26:54.481117 13369 consensus_queue.cc:1048] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:54.481292 13309 raft_consensus.cc:1081] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Signalling peer 7092cb56813748718b688edaee860d84 to start an election
I20260709 12:26:54.482252 13108 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aeb326e9f81d4120a8038a64b19111a2"
dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:45710
I20260709 12:26:54.485723 13181 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bdb2a661794848db8d2c900fee23db77"
dest_uuid: "7092cb56813748718b688edaee860d84"
 from {username='slave'} at 127.0.0.1:41572
I20260709 12:26:54.486766 13181 raft_consensus.cc:493] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:54.487321 13181 raft_consensus.cc:3060] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.483131 13108 raft_consensus.cc:493] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:54.489849 13108 raft_consensus.cc:3060] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.502949 13108 raft_consensus.cc:515] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.502831 13181 raft_consensus.cc:515] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.507225 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb326e9f81d4120a8038a64b19111a2" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:54.507973 13182 raft_consensus.cc:3060] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.509171 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb2a661794848db8d2c900fee23db77" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:54.509371 13108 leader_election.cc:290] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Requested vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:54.511250 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb326e9f81d4120a8038a64b19111a2" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:54.512215 13256 raft_consensus.cc:3055] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:54.512586 13256 raft_consensus.cc:740] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:26:54.513499 13256 consensus_queue.cc:260] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.514539 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb2a661794848db8d2c900fee23db77" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:54.515493 13257 raft_consensus.cc:3055] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:54.515627 13181 leader_election.cc:290] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 2 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:54.516256 13257 raft_consensus.cc:740] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:26:54.521636 13182 raft_consensus.cc:2468] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:26:54.526729 13073 leader_election.cc:304] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:54.527866 13330 raft_consensus.cc:2804] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:54.528962 13330 raft_consensus.cc:697] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Becoming Leader. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Running, Role: LEADER
I20260709 12:26:54.531486 13330 consensus_queue.cc:237] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.533233 13257 consensus_queue.cc:260] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.515352 13256 raft_consensus.cc:3060] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.538601 13339 raft_consensus.cc:993] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84: : Instructing follower b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:26:54.539340 13378 raft_consensus.cc:1081] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Signalling peer b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:26:54.541885 13107 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82ccf5e5e1b14ff98884e5649137d885"
dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:45704
I20260709 12:26:54.542924 13107 raft_consensus.cc:493] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:54.543555 13107 raft_consensus.cc:3060] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.536698 13257 raft_consensus.cc:3060] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.544778 13006 catalog_manager.cc:5697] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 reported cstate change: term changed from 1 to 2, leader changed from 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67) to b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:54.549328 13256 raft_consensus.cc:2468] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:26:54.558051 13257 raft_consensus.cc:2468] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 2.
I20260709 12:26:54.558079 13107 raft_consensus.cc:515] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:54.560317 13148 leader_election.cc:304] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:54.561102 13107 leader_election.cc:290] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Requested vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:54.562150 13339 raft_consensus.cc:2804] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:54.562953 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ccf5e5e1b14ff98884e5649137d885" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:54.563459 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ccf5e5e1b14ff98884e5649137d885" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:54.562954 13339 raft_consensus.cc:697] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 2 LEADER]: Becoming Leader. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:26:54.563817 13257 raft_consensus.cc:3060] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.564626 13182 raft_consensus.cc:3055] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:54.565152 13182 raft_consensus.cc:740] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:26:54.565210 13339 consensus_queue.cc:237] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.566282 13182 consensus_queue.cc:260] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:54.567862 13182 raft_consensus.cc:3060] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.575049 13257 raft_consensus.cc:2468] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:26:54.578320 13073 leader_election.cc:304] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:54.578397 13006 catalog_manager.cc:5697] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 reported cstate change: term changed from 1 to 2, leader changed from 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67) to 7092cb56813748718b688edaee860d84 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "7092cb56813748718b688edaee860d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:54.580488 13330 raft_consensus.cc:2804] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:54.581395 13330 raft_consensus.cc:697] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Becoming Leader. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Running, Role: LEADER
I20260709 12:26:54.581606 13182 raft_consensus.cc:2468] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:26:54.584030 13330 consensus_queue.cc:237] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } }
I20260709 12:26:54.598502 13006 catalog_manager.cc:5697] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 reported cstate change: term changed from 1 to 2, leader changed from 7092cb56813748718b688edaee860d84 (127.1.123.66) to b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:54.634413 13309 raft_consensus.cc:993] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99: : Instructing follower 7092cb56813748718b688edaee860d84 to start an election
I20260709 12:26:54.635087 13309 raft_consensus.cc:1081] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Signalling peer 7092cb56813748718b688edaee860d84 to start an election
I20260709 12:26:54.637194 13182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1"
dest_uuid: "7092cb56813748718b688edaee860d84"
 from {username='slave'} at 127.0.0.1:41572
I20260709 12:26:54.637982 13182 raft_consensus.cc:493] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:54.639173 13182 raft_consensus.cc:3060] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.648983 13182 raft_consensus.cc:515] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.651701 13182 leader_election.cc:290] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 2 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:54.653036 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:54.654107 13107 raft_consensus.cc:3060] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.654883 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:54.655748 13257 raft_consensus.cc:3055] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:54.656080 13257 raft_consensus.cc:740] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:26:54.656963 13257 consensus_queue.cc:260] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.659158 13257 raft_consensus.cc:3060] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.663780 13107 raft_consensus.cc:2468] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 2.
I20260709 12:26:54.666363 13149 leader_election.cc:304] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:54.667593 13339 raft_consensus.cc:2804] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:54.668252 13339 raft_consensus.cc:697] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 2 LEADER]: Becoming Leader. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:26:54.669231 13339 consensus_queue.cc:237] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.670311 13257 raft_consensus.cc:2468] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 2.
I20260709 12:26:54.681686 13006 catalog_manager.cc:5697] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 reported cstate change: term changed from 1 to 2, leader changed from 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67) to 7092cb56813748718b688edaee860d84 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "7092cb56813748718b688edaee860d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:54.941969 13333 raft_consensus.cc:993] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99: : Instructing follower b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:26:54.942776 13309 raft_consensus.cc:1081] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Signalling peer b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:26:54.945027 13107 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f"
dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:45710
I20260709 12:26:54.945756 13107 raft_consensus.cc:493] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:54.946064 13107 raft_consensus.cc:3060] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.953583 13107 raft_consensus.cc:515] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.955950 13107 leader_election.cc:290] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Requested vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:54.957141 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:54.957577 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:54.958007 13182 raft_consensus.cc:3060] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.958434 13257 raft_consensus.cc:3055] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:54.958858 13257 raft_consensus.cc:740] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:26:54.959874 13257 consensus_queue.cc:260] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.961078 13257 raft_consensus.cc:3060] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:54.964697 13182 raft_consensus.cc:2468] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:26:54.966571 13073 leader_election.cc:304] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:54.967916 13387 raft_consensus.cc:2804] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:54.968525 13387 raft_consensus.cc:697] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Becoming Leader. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Running, Role: LEADER
I20260709 12:26:54.969645 13257 raft_consensus.cc:2468] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:26:54.969648 13387 consensus_queue.cc:237] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:54.979231 13006 catalog_manager.cc:5697] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 reported cstate change: term changed from 1 to 2, leader changed from 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67) to b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:55.054245 13107 raft_consensus.cc:3060] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:55.063714 13107 raft_consensus.cc:1275] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Refusing update from remote peer 7092cb56813748718b688edaee860d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:55.066233 13339 consensus_queue.cc:1048] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:55.074925 13257 raft_consensus.cc:1275] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:55.078346 13387 consensus_queue.cc:1048] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:55.080602 13257 raft_consensus.cc:1275] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Refusing update from remote peer 7092cb56813748718b688edaee860d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:55.082337 13339 consensus_queue.cc:1048] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:55.097503 13182 raft_consensus.cc:1275] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:55.100103 13256 raft_consensus.cc:1275] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:55.100425 13387 consensus_queue.cc:1048] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:55.104029 13182 raft_consensus.cc:1275] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:55.108868 13351 consensus_queue.cc:1048] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:55.109747 13387 consensus_queue.cc:1048] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:55.229645 13107 raft_consensus.cc:1275] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Refusing update from remote peer 7092cb56813748718b688edaee860d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:55.231626 13339 consensus_queue.cc:1048] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:55.242905 13255 raft_consensus.cc:1275] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Refusing update from remote peer 7092cb56813748718b688edaee860d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:55.244673 13339 consensus_queue.cc:1048] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:55.403084 13182 raft_consensus.cc:1275] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:55.404501 13392 consensus_queue.cc:1048] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:55.417699 13255 raft_consensus.cc:1275] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:55.419554 13392 consensus_queue.cc:1048] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:57.532848 13004 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54326:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:26:57.535696 13004 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:26:57.603782 13090 tablet_service.cc:1511] Processing CreateTablet for tablet c3c69181eb6e4eff8fc932f21978aba0 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:57.604709 13165 tablet_service.cc:1511] Processing CreateTablet for tablet c3c69181eb6e4eff8fc932f21978aba0 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:57.605387 13090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c69181eb6e4eff8fc932f21978aba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.606441 13165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c69181eb6e4eff8fc932f21978aba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.607167 13239 tablet_service.cc:1511] Processing CreateTablet for tablet c3c69181eb6e4eff8fc932f21978aba0 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:57.608968 13239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c69181eb6e4eff8fc932f21978aba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.608753 13094 tablet_service.cc:1511] Processing CreateTablet for tablet a96a664188314072a99803d265c020a6 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:57.610491 13094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a96a664188314072a99803d265c020a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.614573 13092 tablet_service.cc:1511] Processing CreateTablet for tablet abbf041ad34d4b16adf6e446c8f8ca68 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:57.615589 13240 tablet_service.cc:1511] Processing CreateTablet for tablet a96a664188314072a99803d265c020a6 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:57.616979 13092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abbf041ad34d4b16adf6e446c8f8ca68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.619338 13169 tablet_service.cc:1511] Processing CreateTablet for tablet a96a664188314072a99803d265c020a6 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:57.629966 13171 tablet_service.cc:1511] Processing CreateTablet for tablet abbf041ad34d4b16adf6e446c8f8ca68 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:57.640014 13241 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfdb12d53dc484197c250705a4385f2 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:57.641111 13095 tablet_service.cc:1511] Processing CreateTablet for tablet bd98068120554eda83a086c1b57fc3be (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:57.632066 13244 tablet_service.cc:1511] Processing CreateTablet for tablet abbf041ad34d4b16adf6e446c8f8ca68 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:57.646795 13168 tablet_service.cc:1511] Processing CreateTablet for tablet bd98068120554eda83a086c1b57fc3be (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:57.626924 13240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a96a664188314072a99803d265c020a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.648249 13095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd98068120554eda83a086c1b57fc3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.649935 13242 tablet_service.cc:1511] Processing CreateTablet for tablet bd98068120554eda83a086c1b57fc3be (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:57.650780 13244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abbf041ad34d4b16adf6e446c8f8ca68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.652064 13242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd98068120554eda83a086c1b57fc3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.655131 13241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfdb12d53dc484197c250705a4385f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.634881 13093 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfdb12d53dc484197c250705a4385f2 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:57.660597 13093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfdb12d53dc484197c250705a4385f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.636346 13166 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfdb12d53dc484197c250705a4385f2 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:57.663323 13170 tablet_service.cc:1511] Processing CreateTablet for tablet 5b336d26784a4500817fda148c468562 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:57.634572 13169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a96a664188314072a99803d265c020a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.669952 13170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b336d26784a4500817fda148c468562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.669173 13096 tablet_service.cc:1511] Processing CreateTablet for tablet 5b336d26784a4500817fda148c468562 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:57.672513 13096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b336d26784a4500817fda148c468562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.674028 13171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abbf041ad34d4b16adf6e446c8f8ca68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.675642 13243 tablet_service.cc:1511] Processing CreateTablet for tablet 5b336d26784a4500817fda148c468562 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:57.677255 13243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b336d26784a4500817fda148c468562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.677999 13166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfdb12d53dc484197c250705a4385f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.682056 13168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd98068120554eda83a086c1b57fc3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.694250 13098 tablet_service.cc:1511] Processing CreateTablet for tablet f5dc20f40888477caf0843336f9cf5c9 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:57.696388 13098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5dc20f40888477caf0843336f9cf5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.698632 13167 tablet_service.cc:1511] Processing CreateTablet for tablet f5dc20f40888477caf0843336f9cf5c9 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:57.701159 13167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5dc20f40888477caf0843336f9cf5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.708948 13245 tablet_service.cc:1511] Processing CreateTablet for tablet f5dc20f40888477caf0843336f9cf5c9 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:57.710824 13245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5dc20f40888477caf0843336f9cf5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.718936 13091 tablet_service.cc:1511] Processing CreateTablet for tablet f41b34915733403b87cd109271ee0945 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:57.720882 13091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f41b34915733403b87cd109271ee0945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.718936 13172 tablet_service.cc:1511] Processing CreateTablet for tablet f41b34915733403b87cd109271ee0945 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:57.734472 13172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f41b34915733403b87cd109271ee0945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.737123 13247 tablet_service.cc:1511] Processing CreateTablet for tablet f41b34915733403b87cd109271ee0945 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:57.739245 13247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f41b34915733403b87cd109271ee0945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.749156 13173 tablet_service.cc:1511] Processing CreateTablet for tablet fa85a653952647ff9fc4db109a8e5211 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:57.751286 13173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa85a653952647ff9fc4db109a8e5211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.752722 13246 tablet_service.cc:1511] Processing CreateTablet for tablet fa85a653952647ff9fc4db109a8e5211 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:57.754523 13246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa85a653952647ff9fc4db109a8e5211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.754606 13097 tablet_service.cc:1511] Processing CreateTablet for tablet fa85a653952647ff9fc4db109a8e5211 (DEFAULT_TABLE table=soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:57.756387 13097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa85a653952647ff9fc4db109a8e5211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:57.832680 13410 tablet_bootstrap.cc:492] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:57.898960 13410 tablet_bootstrap.cc:654] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:57.962785 13413 tablet_bootstrap.cc:492] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84: Bootstrap starting.
W20260709 12:26:58.058207 13054 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:58.168848 13415 tablet_bootstrap.cc:492] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:58.263556 13413 tablet_bootstrap.cc:654] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
W20260709 12:26:58.277078 13073 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.66:40767, user_credentials={real_user=slave}} blocked reactor thread for 121124us
I20260709 12:26:58.290881 13410 tablet_bootstrap.cc:492] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:58.291988 13410 ts_tablet_manager.cc:1403] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.460s	user 0.020s	sys 0.016s
I20260709 12:26:58.289088 13415 tablet_bootstrap.cc:654] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.308524 13413 tablet_bootstrap.cc:492] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:58.308683 13415 tablet_bootstrap.cc:492] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:58.309733 13413 ts_tablet_manager.cc:1403] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.347s	user 0.026s	sys 0.035s
I20260709 12:26:58.310092 13415 ts_tablet_manager.cc:1403] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.143s	user 0.018s	sys 0.004s
I20260709 12:26:58.309013 13410 raft_consensus.cc:359] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.310825 13410 raft_consensus.cc:385] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.311244 13410 raft_consensus.cc:740] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.312327 13410 consensus_queue.cc:260] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.313387 13413 raft_consensus.cc:359] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.315059 13413 raft_consensus.cc:385] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.314889 13415 raft_consensus.cc:359] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.315706 13413 raft_consensus.cc:740] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.316069 13410 ts_tablet_manager.cc:1434] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.003s
I20260709 12:26:58.316316 13415 raft_consensus.cc:385] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.316740 13415 raft_consensus.cc:740] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.317040 13413 consensus_queue.cc:260] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.318295 13410 tablet_bootstrap.cc:492] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:58.317718 13415 consensus_queue.cc:260] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.321147 13413 ts_tablet_manager.cc:1434] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 12:26:58.323549 13420 raft_consensus.cc:493] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.323730 13415 ts_tablet_manager.cc:1434] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:26:58.324350 13420 raft_consensus.cc:515] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.325740 13415 tablet_bootstrap.cc:492] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:58.332409 13410 tablet_bootstrap.cc:654] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.334612 13415 tablet_bootstrap.cc:654] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.338111 13255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abbf041ad34d4b16adf6e446c8f8ca68" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:58.337422 13420 leader_election.cc:290] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
W20260709 12:26:58.345896 13073 leader_election.cc:343] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:58.348989 13415 tablet_bootstrap.cc:492] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:58.349355 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abbf041ad34d4b16adf6e446c8f8ca68" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:58.349794 13415 ts_tablet_manager.cc:1403] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.004s
I20260709 12:26:58.355301 13415 raft_consensus.cc:359] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.356487 13415 raft_consensus.cc:385] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.356890 13415 raft_consensus.cc:740] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
W20260709 12:26:58.360713 13073 leader_election.cc:343] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:58.359337 13415 consensus_queue.cc:260] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.358772 13413 tablet_bootstrap.cc:492] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:58.361598 13073 leader_election.cc:304] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0fe69a669ba4dc6b2724c4cff489927; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84
I20260709 12:26:58.367558 13414 raft_consensus.cc:2749] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:58.368113 13415 ts_tablet_manager.cc:1434] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 12:26:58.371994 13415 tablet_bootstrap.cc:492] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:58.386256 13415 tablet_bootstrap.cc:654] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.387326 13413 tablet_bootstrap.cc:654] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.394845 13415 tablet_bootstrap.cc:492] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:58.394950 13410 tablet_bootstrap.cc:492] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:58.395653 13415 ts_tablet_manager.cc:1403] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.008s
I20260709 12:26:58.395951 13410 ts_tablet_manager.cc:1403] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.078s	user 0.043s	sys 0.020s
I20260709 12:26:58.399360 13415 raft_consensus.cc:359] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.400166 13415 raft_consensus.cc:385] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.400432 13415 raft_consensus.cc:740] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.401249 13415 consensus_queue.cc:260] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.401373 13410 raft_consensus.cc:359] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.402530 13410 raft_consensus.cc:385] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.402890 13410 raft_consensus.cc:740] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.404001 13410 consensus_queue.cc:260] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.405483 13413 tablet_bootstrap.cc:492] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:58.406270 13413 ts_tablet_manager.cc:1403] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.015s
I20260709 12:26:58.407390 13415 ts_tablet_manager.cc:1434] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.005s
I20260709 12:26:58.406734 13410 ts_tablet_manager.cc:1434] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.003s
I20260709 12:26:58.409637 13415 tablet_bootstrap.cc:492] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:58.410355 13410 tablet_bootstrap.cc:492] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:58.413374 13413 raft_consensus.cc:359] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.415563 13413 raft_consensus.cc:385] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.415977 13413 raft_consensus.cc:740] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.419495 13415 tablet_bootstrap.cc:654] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.419449 13413 consensus_queue.cc:260] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.425096 13413 ts_tablet_manager.cc:1434] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 12:26:58.425235 13410 tablet_bootstrap.cc:654] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.427079 13413 tablet_bootstrap.cc:492] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:58.440436 13413 tablet_bootstrap.cc:654] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.440701 13415 tablet_bootstrap.cc:492] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:58.441926 13415 ts_tablet_manager.cc:1403] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.004s
I20260709 12:26:58.442999 13420 raft_consensus.cc:493] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.443643 13420 raft_consensus.cc:515] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.445019 13410 tablet_bootstrap.cc:492] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:58.448602 13420 leader_election.cc:290] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:58.450755 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
W20260709 12:26:58.458221 13073 leader_election.cc:343] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:58.459352 13410 ts_tablet_manager.cc:1403] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.005s
I20260709 12:26:58.447889 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
W20260709 12:26:58.462716 13073 leader_election.cc:343] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:58.463505 13073 leader_election.cc:304] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0fe69a669ba4dc6b2724c4cff489927; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84
I20260709 12:26:58.464501 13414 raft_consensus.cc:2749] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:58.447074 13415 raft_consensus.cc:359] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.471100 13415 raft_consensus.cc:385] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.471480 13415 raft_consensus.cc:740] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.472347 13415 consensus_queue.cc:260] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.474706 13410 raft_consensus.cc:359] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.476642 13410 raft_consensus.cc:385] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.477116 13410 raft_consensus.cc:740] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.479068 13415 ts_tablet_manager.cc:1434] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 12:26:58.478899 13410 consensus_queue.cc:260] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.482491 13415 tablet_bootstrap.cc:492] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:58.483136 13410 ts_tablet_manager.cc:1434] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.005s
I20260709 12:26:58.484453 13410 tablet_bootstrap.cc:492] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:58.492120 13415 tablet_bootstrap.cc:654] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.492703 13410 tablet_bootstrap.cc:654] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.495633 13413 tablet_bootstrap.cc:492] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:58.496333 13413 ts_tablet_manager.cc:1403] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.070s	user 0.027s	sys 0.018s
I20260709 12:26:58.501943 13413 raft_consensus.cc:359] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.502987 13413 raft_consensus.cc:385] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.503291 13413 raft_consensus.cc:740] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.504177 13413 consensus_queue.cc:260] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.507884 13413 ts_tablet_manager.cc:1434] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:26:58.509783 13413 tablet_bootstrap.cc:492] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:58.511004 13309 raft_consensus.cc:493] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.511771 13309 raft_consensus.cc:515] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.514755 13420 raft_consensus.cc:493] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.515465 13420 raft_consensus.cc:515] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.517901 13420 leader_election.cc:290] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:58.518194 13410 tablet_bootstrap.cc:492] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:58.519066 13410 ts_tablet_manager.cc:1403] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.015s
I20260709 12:26:58.519207 13413 tablet_bootstrap.cc:654] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.520905 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b336d26784a4500817fda148c468562" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:58.523587 13410 raft_consensus.cc:359] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.524897 13410 raft_consensus.cc:385] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.525408 13410 raft_consensus.cc:740] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.524493 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b336d26784a4500817fda148c468562" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:58.526703 13256 raft_consensus.cc:2468] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 0.
I20260709 12:26:58.526769 13410 consensus_queue.cc:260] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
W20260709 12:26:58.527922 13073 leader_election.cc:343] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:58.529784 13073 leader_election.cc:304] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 7092cb56813748718b688edaee860d84
I20260709 12:26:58.533197 13413 tablet_bootstrap.cc:492] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:58.533774 13420 raft_consensus.cc:2804] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:58.534180 13413 ts_tablet_manager.cc:1403] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.008s
I20260709 12:26:58.534801 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69181eb6e4eff8fc932f21978aba0" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:58.536674 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69181eb6e4eff8fc932f21978aba0" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
W20260709 12:26:58.537096 13223 leader_election.cc:343] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:26:58.538496 13222 leader_election.cc:343] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:58.538959 13222 leader_election.cc:304] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99; no voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:58.539415 13413 raft_consensus.cc:359] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.540763 13413 raft_consensus.cc:385] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.541152 13413 raft_consensus.cc:740] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.542393 13413 consensus_queue.cc:260] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.543639 13309 leader_election.cc:290] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:58.547256 13434 raft_consensus.cc:2749] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:58.548394 13410 ts_tablet_manager.cc:1434] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.018s
I20260709 12:26:58.534451 13420 raft_consensus.cc:493] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.549276 13420 raft_consensus.cc:3060] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.549382 13413 ts_tablet_manager.cc:1434] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 12:26:58.552930 13413 tablet_bootstrap.cc:492] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:58.563210 13413 tablet_bootstrap.cc:654] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.569432 13415 tablet_bootstrap.cc:492] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:58.574579 13420 raft_consensus.cc:515] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.577273 13415 ts_tablet_manager.cc:1403] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.097s	user 0.017s	sys 0.008s
I20260709 12:26:58.579358 13410 tablet_bootstrap.cc:492] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:58.586580 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b336d26784a4500817fda148c468562" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:58.587476 13420 leader_election.cc:290] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 election: Requested vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
W20260709 12:26:58.588788 13073 leader_election.cc:343] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:58.589164 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b336d26784a4500817fda148c468562" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:58.589926 13256 raft_consensus.cc:3060] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.593189 13415 raft_consensus.cc:359] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.594393 13415 raft_consensus.cc:385] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.594796 13415 raft_consensus.cc:740] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.593797 13410 tablet_bootstrap.cc:654] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.595752 13415 consensus_queue.cc:260] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.597625 13413 tablet_bootstrap.cc:492] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:58.598335 13413 ts_tablet_manager.cc:1403] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.046s	user 0.025s	sys 0.020s
I20260709 12:26:58.601104 13256 raft_consensus.cc:2468] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 1.
I20260709 12:26:58.603385 13073 leader_election.cc:304] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 7092cb56813748718b688edaee860d84
I20260709 12:26:58.603075 13413 raft_consensus.cc:359] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.605032 13413 raft_consensus.cc:385] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.605607 13413 raft_consensus.cc:740] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.606596 13413 consensus_queue.cc:260] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.608737 13415 ts_tablet_manager.cc:1434] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.001s
I20260709 12:26:58.609869 13410 tablet_bootstrap.cc:492] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:58.610383 13413 ts_tablet_manager.cc:1434] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 12:26:58.610602 13410 ts_tablet_manager.cc:1403] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.010s
I20260709 12:26:58.611857 13413 tablet_bootstrap.cc:492] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:58.614163 13420 raft_consensus.cc:2804] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:58.614467 13410 raft_consensus.cc:359] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.615242 13420 raft_consensus.cc:697] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 LEADER]: Becoming Leader. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Running, Role: LEADER
I20260709 12:26:58.615500 13410 raft_consensus.cc:385] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.616189 13410 raft_consensus.cc:740] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.616650 13420 consensus_queue.cc:237] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.617411 13410 consensus_queue.cc:260] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.620617 13410 ts_tablet_manager.cc:1434] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 12:26:58.623106 13413 tablet_bootstrap.cc:654] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.624096 13410 tablet_bootstrap.cc:492] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:58.629949 13415 tablet_bootstrap.cc:492] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:58.634150 13410 tablet_bootstrap.cc:654] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.644588 13413 tablet_bootstrap.cc:492] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:58.645241 13413 ts_tablet_manager.cc:1403] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.010s
I20260709 12:26:58.644805 13006 catalog_manager.cc:5697] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:58.650089 13413 raft_consensus.cc:359] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.651003 13413 raft_consensus.cc:385] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.651522 13413 raft_consensus.cc:740] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.652607 13413 consensus_queue.cc:260] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.655640 13413 ts_tablet_manager.cc:1434] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:26:58.656926 13413 tablet_bootstrap.cc:492] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:58.662804 13415 tablet_bootstrap.cc:654] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.664336 13416 raft_consensus.cc:493] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.665170 13416 raft_consensus.cc:515] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.667737 13413 tablet_bootstrap.cc:654] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.667867 13418 raft_consensus.cc:493] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.668648 13418 raft_consensus.cc:515] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.670334 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:58.672155 13416 leader_election.cc:290] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:58.675345 13255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:58.672683 13433 raft_consensus.cc:493] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.672801 13149 leader_election.cc:343] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:58.677999 13433 raft_consensus.cc:515] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.680763 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd98068120554eda83a086c1b57fc3be" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:58.682039 13107 raft_consensus.cc:2468] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:58.682902 13108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:58.683580 13255 raft_consensus.cc:2468] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:58.684177 13149 leader_election.cc:304] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:58.674415 13418 leader_election.cc:290] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
W20260709 12:26:58.685220 13223 leader_election.cc:343] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:58.685643 13416 raft_consensus.cc:2804] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:58.683147 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd98068120554eda83a086c1b57fc3be" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:58.686095 13433 leader_election.cc:290] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:58.686412 13416 raft_consensus.cc:493] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.686708 13256 raft_consensus.cc:2468] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:58.687062 13416 raft_consensus.cc:3060] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.690279 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:58.674647 13410 tablet_bootstrap.cc:492] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:58.691474 13182 raft_consensus.cc:2468] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:58.692715 13410 ts_tablet_manager.cc:1403] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.069s	user 0.031s	sys 0.007s
I20260709 12:26:58.693652 13418 raft_consensus.cc:493] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.696964 13222 leader_election.cc:304] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:58.698724 13433 raft_consensus.cc:2804] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:58.699174 13433 raft_consensus.cc:493] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.699542 13433 raft_consensus.cc:3060] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.700685 13410 raft_consensus.cc:359] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.696979 13418 raft_consensus.cc:515] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.704679 13148 leader_election.cc:304] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:58.706490 13411 raft_consensus.cc:2804] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:58.706669 13418 leader_election.cc:290] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:58.707029 13411 raft_consensus.cc:493] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.707487 13410 raft_consensus.cc:385] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.707638 13411 raft_consensus.cc:3060] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.707985 13410 raft_consensus.cc:740] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.708194 13108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abbf041ad34d4b16adf6e446c8f8ca68" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:58.710418 13108 raft_consensus.cc:2468] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:58.712221 13255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abbf041ad34d4b16adf6e446c8f8ca68" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:58.713130 13149 leader_election.cc:304] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:58.709472 13410 consensus_queue.cc:260] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.715286 13418 raft_consensus.cc:2804] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:58.715835 13418 raft_consensus.cc:493] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.716187 13418 raft_consensus.cc:3060] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.718951 13410 ts_tablet_manager.cc:1434] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.001s
I20260709 12:26:58.720583 13410 tablet_bootstrap.cc:492] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:58.724359 13413 tablet_bootstrap.cc:492] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:58.725080 13413 ts_tablet_manager.cc:1403] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.068s	user 0.021s	sys 0.016s
I20260709 12:26:58.713147 13255 raft_consensus.cc:2468] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:58.729839 13418 raft_consensus.cc:515] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.737499 13108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abbf041ad34d4b16adf6e446c8f8ca68" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:58.738454 13108 raft_consensus.cc:3060] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.739554 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abbf041ad34d4b16adf6e446c8f8ca68" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:58.740703 13256 raft_consensus.cc:3060] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.708194 13416 raft_consensus.cc:515] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.729420 13433 raft_consensus.cc:515] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.753183 13416 leader_election.cc:290] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:58.754087 13108 raft_consensus.cc:2468] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:58.754719 13255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd98068120554eda83a086c1b57fc3be" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:58.756184 13410 tablet_bootstrap.cc:654] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.755460 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd98068120554eda83a086c1b57fc3be" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:58.758209 13107 raft_consensus.cc:3060] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.760053 13415 tablet_bootstrap.cc:492] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:58.760960 13415 ts_tablet_manager.cc:1403] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.131s	user 0.023s	sys 0.013s
I20260709 12:26:58.755668 13255 raft_consensus.cc:3060] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.757051 13418 leader_election.cc:290] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:58.781251 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:58.778129 13433 leader_election.cc:290] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:58.798585 13413 raft_consensus.cc:359] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.799975 13413 raft_consensus.cc:385] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.802065 13413 raft_consensus.cc:740] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.805465 13413 consensus_queue.cc:260] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.810875 13256 raft_consensus.cc:2468] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:58.812218 13413 ts_tablet_manager.cc:1434] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.087s	user 0.004s	sys 0.008s
I20260709 12:26:58.843487 13413 tablet_bootstrap.cc:492] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:58.843760 13148 leader_election.cc:304] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
I20260709 12:26:58.845191 13448 raft_consensus.cc:2804] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:58.845935 13448 raft_consensus.cc:697] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Becoming Leader. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:26:58.846716 13410 tablet_bootstrap.cc:492] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:58.846911 13448 consensus_queue.cc:237] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.814114 13415 raft_consensus.cc:359] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.830130 13411 raft_consensus.cc:515] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.849618 13415 raft_consensus.cc:385] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.850150 13415 raft_consensus.cc:740] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.851172 13415 consensus_queue.cc:260] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.833760 13255 raft_consensus.cc:2468] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:58.847581 13410 ts_tablet_manager.cc:1403] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.127s	user 0.027s	sys 0.030s
I20260709 12:26:58.833437 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:58.855326 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:58.860662 13182 raft_consensus.cc:2393] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in current term 1: Already voted for candidate 7092cb56813748718b688edaee860d84 in this term.
I20260709 12:26:58.861070 13256 raft_consensus.cc:2393] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7092cb56813748718b688edaee860d84 in current term 1: Already voted for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in this term.
I20260709 12:26:58.855827 13413 tablet_bootstrap.cc:654] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.859508 13148 leader_election.cc:304] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
I20260709 12:26:58.865036 13449 raft_consensus.cc:2804] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:58.864108 13410 raft_consensus.cc:359] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.865784 13410 raft_consensus.cc:385] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.865994 13449 raft_consensus.cc:697] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Becoming Leader. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:26:58.867765 13449 consensus_queue.cc:237] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.860019 13433 raft_consensus.cc:493] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.856390 13415 ts_tablet_manager.cc:1434] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.095s	user 0.011s	sys 0.000s
I20260709 12:26:58.871246 13433 raft_consensus.cc:515] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.872704 13415 tablet_bootstrap.cc:492] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:58.874684 13433 leader_election.cc:290] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:58.859934 13411 leader_election.cc:290] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:58.879868 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa85a653952647ff9fc4db109a8e5211" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:58.866346 13410 raft_consensus.cc:740] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.881122 13182 raft_consensus.cc:2468] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:58.881649 13410 consensus_queue.cc:260] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.882917 13222 leader_election.cc:304] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
I20260709 12:26:58.884460 13433 raft_consensus.cc:2804] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:58.884876 13433 raft_consensus.cc:493] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.885202 13410 ts_tablet_manager.cc:1434] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.004s
I20260709 12:26:58.885234 13433 raft_consensus.cc:3060] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.887020 13410 tablet_bootstrap.cc:492] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
I20260709 12:26:58.891104 13448 raft_consensus.cc:493] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.891918 13448 raft_consensus.cc:515] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.897399 13104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b336d26784a4500817fda148c468562" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:58.901221 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b336d26784a4500817fda148c468562" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:58.901067 13433 raft_consensus.cc:515] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.902372 13256 raft_consensus.cc:2393] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7092cb56813748718b688edaee860d84 in current term 1: Already voted for candidate b0fe69a669ba4dc6b2724c4cff489927 in this term.
I20260709 12:26:58.903206 13413 tablet_bootstrap.cc:492] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:58.903947 13413 ts_tablet_manager.cc:1403] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.004s
I20260709 12:26:58.903795 13148 leader_election.cc:304] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7092cb56813748718b688edaee860d84; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:58.905431 13433 leader_election.cc:290] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:58.907361 13413 raft_consensus.cc:359] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.909024 13413 raft_consensus.cc:385] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.909418 13413 raft_consensus.cc:740] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.910604 13413 consensus_queue.cc:260] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.907544 13448 leader_election.cc:290] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:58.907692 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa85a653952647ff9fc4db109a8e5211" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:58.912971 13413 ts_tablet_manager.cc:1434] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.002s
I20260709 12:26:58.913430 13182 raft_consensus.cc:3060] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.914175 13413 tablet_bootstrap.cc:492] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84: Bootstrap starting.
I20260709 12:26:58.915339 13410 tablet_bootstrap.cc:654] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.922709 13413 tablet_bootstrap.cc:654] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.908339 13449 raft_consensus.cc:3060] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.908603 13415 tablet_bootstrap.cc:654] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.927626 13410 tablet_bootstrap.cc:492] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:58.928511 13410 ts_tablet_manager.cc:1403] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.009s
I20260709 12:26:58.930509 13182 raft_consensus.cc:2468] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:58.931205 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:58.942876 13105 raft_consensus.cc:3060] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:58.944133 13413 tablet_bootstrap.cc:492] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84: No bootstrap required, opened a new log
I20260709 12:26:58.944823 13413 ts_tablet_manager.cc:1403] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.002s
I20260709 12:26:58.948048 13413 raft_consensus.cc:359] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.949162 13222 leader_election.cc:304] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
I20260709 12:26:58.951284 13433 raft_consensus.cc:2804] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:58.952579 13433 raft_consensus.cc:697] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Leader. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:26:58.953637 13433 consensus_queue.cc:237] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.955646 13415 tablet_bootstrap.cc:492] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:58.957087 13105 raft_consensus.cc:2468] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:58.963920 13415 ts_tablet_manager.cc:1403] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.092s	user 0.018s	sys 0.009s
I20260709 12:26:58.965896 13149 leader_election.cc:304] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99
I20260709 12:26:58.966423 13107 raft_consensus.cc:2468] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:58.968492 13448 raft_consensus.cc:2804] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:58.969408 13448 raft_consensus.cc:697] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Becoming Leader. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:26:58.937022 13104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa85a653952647ff9fc4db109a8e5211" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:58.970595 13448 consensus_queue.cc:237] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.932286 13410 raft_consensus.cc:359] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.971801 13410 raft_consensus.cc:385] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.972158 13410 raft_consensus.cc:740] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.973124 13410 consensus_queue.cc:260] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.974753 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa85a653952647ff9fc4db109a8e5211" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:58.975195 13415 raft_consensus.cc:359] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:58.975759 13410 ts_tablet_manager.cc:1434] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.003s
I20260709 12:26:58.976023 13415 raft_consensus.cc:385] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.976475 13415 raft_consensus.cc:740] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.977150 13410 tablet_bootstrap.cc:492] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99: Bootstrap starting.
W20260709 12:26:58.980057 13223 leader_election.cc:343] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:58.949204 13413 raft_consensus.cc:385] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:58.983330 13413 raft_consensus.cc:740] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Initialized, Role: FOLLOWER
I20260709 12:26:58.998588 13415 consensus_queue.cc:260] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.000069 13103 raft_consensus.cc:3060] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.001399 13415 ts_tablet_manager.cc:1434] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 12:26:59.002784 13415 tablet_bootstrap.cc:492] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:59.006577 13413 consensus_queue.cc:260] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.011902 13413 ts_tablet_manager.cc:1434] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84: Time spent starting tablet: real 0.067s	user 0.010s	sys 0.003s
I20260709 12:26:59.017107 13415 tablet_bootstrap.cc:654] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:59.021005 13103 raft_consensus.cc:2468] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:59.030238 13415 tablet_bootstrap.cc:492] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:59.030952 13415 ts_tablet_manager.cc:1403] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.002s
I20260709 12:26:59.033871 13415 raft_consensus.cc:359] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.034515 13415 raft_consensus.cc:385] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:59.034858 13415 raft_consensus.cc:740] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:59.035840 13415 consensus_queue.cc:260] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.042600 13415 ts_tablet_manager.cc:1434] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.004s
I20260709 12:26:59.044016 13415 tablet_bootstrap.cc:492] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927: Bootstrap starting.
I20260709 12:26:59.070613 13415 tablet_bootstrap.cc:654] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:58.958097 13449 raft_consensus.cc:2749] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:59.084861 13415 tablet_bootstrap.cc:492] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927: No bootstrap required, opened a new log
I20260709 12:26:59.087738 13415 ts_tablet_manager.cc:1403] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.012s
I20260709 12:26:59.096779 13415 raft_consensus.cc:359] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.097698 13415 raft_consensus.cc:385] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:59.098004 13415 raft_consensus.cc:740] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Initialized, Role: FOLLOWER
I20260709 12:26:59.098843 13415 consensus_queue.cc:260] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.100642 13415 ts_tablet_manager.cc:1434] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
W20260709 12:26:59.127283 13054 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:59.139735 13456 raft_consensus.cc:493] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.140533 13456 raft_consensus.cc:515] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.143342 13456 leader_election.cc:290] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:59.165928 13456 raft_consensus.cc:493] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.166783 13456 raft_consensus.cc:515] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.168627 13449 raft_consensus.cc:493] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.170735 13456 leader_election.cc:290] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:59.169598 13449 raft_consensus.cc:515] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.181034 13449 leader_election.cc:290] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:59.182534 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f41b34915733403b87cd109271ee0945" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:59.183542 13105 raft_consensus.cc:2468] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:59.185452 13149 leader_election.cc:304] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:59.186584 13448 raft_consensus.cc:2804] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:59.187065 13448 raft_consensus.cc:493] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.187422 13448 raft_consensus.cc:3060] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.192467 13449 raft_consensus.cc:493] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.193681 13449 raft_consensus.cc:515] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.198665 13449 leader_election.cc:290] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:59.199153 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69181eb6e4eff8fc932f21978aba0" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:59.200274 13183 raft_consensus.cc:2468] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 0.
I20260709 12:26:59.201684 13073 leader_election.cc:304] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:59.201500 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69181eb6e4eff8fc932f21978aba0" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:59.201987 13448 raft_consensus.cc:515] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.203366 13105 raft_consensus.cc:2468] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:59.205513 13456 raft_consensus.cc:2804] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:59.206626 13456 raft_consensus.cc:493] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.207144 13456 raft_consensus.cc:3060] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.207448 13448 leader_election.cc:290] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:59.215180 13149 leader_election.cc:304] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:59.213840 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f41b34915733403b87cd109271ee0945" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:59.216499 13416 raft_consensus.cc:2804] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:59.216948 13416 raft_consensus.cc:493] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.217741 13416 raft_consensus.cc:3060] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.218968 13105 raft_consensus.cc:3060] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.225171 13106 raft_consensus.cc:2393] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in current term 1: Already voted for candidate 7092cb56813748718b688edaee860d84 in this term.
I20260709 12:26:59.227102 13456 raft_consensus.cc:515] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.230379 13456 leader_election.cc:290] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 election: Requested vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:59.231110 13223 leader_election.cc:304] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99; no voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:59.232215 13435 raft_consensus.cc:2749] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:26:59.232470 13456 consensus_queue.cc:1048] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:59.232808 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69181eb6e4eff8fc932f21978aba0" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:59.235256 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dc20f40888477caf0843336f9cf5c9" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:59.238547 13257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69181eb6e4eff8fc932f21978aba0" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:59.239902 13257 raft_consensus.cc:2468] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 0.
W20260709 12:26:59.241107 13073 leader_election.cc:343] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:59.239109 13255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69181eb6e4eff8fc932f21978aba0" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:59.243034 13255 raft_consensus.cc:3060] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.252560 13105 raft_consensus.cc:2468] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:59.254684 13149 leader_election.cc:304] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:59.256045 13251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f41b34915733403b87cd109271ee0945" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:59.257488 13251 raft_consensus.cc:3060] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.260334 13255 raft_consensus.cc:2468] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 1.
I20260709 12:26:59.256227 13448 raft_consensus.cc:2804] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:59.262707 13448 raft_consensus.cc:697] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Becoming Leader. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:26:59.263264 13449 raft_consensus.cc:493] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.263779 13448 consensus_queue.cc:237] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.263971 13449 raft_consensus.cc:515] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.267305 13251 raft_consensus.cc:2468] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:59.270488 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dc20f40888477caf0843336f9cf5c9" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:59.271461 13103 raft_consensus.cc:2468] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:59.273111 13149 leader_election.cc:304] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:59.277850 13449 leader_election.cc:290] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:59.278553 13449 raft_consensus.cc:2804] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:59.278965 13449 raft_consensus.cc:493] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.279345 13449 raft_consensus.cc:3060] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.279528 13410 tablet_bootstrap.cc:700] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent opening tablet: real 0.179s	user 0.009s	sys 0.000s
I20260709 12:26:59.286751 13410 tablet_bootstrap.cc:654] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:59.290179 13416 raft_consensus.cc:515] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.293496 13433 raft_consensus.cc:493] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.294564 13433 raft_consensus.cc:515] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.298686 13182 raft_consensus.cc:2393] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b0fe69a669ba4dc6b2724c4cff489927 in current term 1: Already voted for candidate 7092cb56813748718b688edaee860d84 in this term.
I20260709 12:26:59.300957 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69181eb6e4eff8fc932f21978aba0" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:59.300119 13073 leader_election.cc:304] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:59.301963 13102 raft_consensus.cc:2393] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7092cb56813748718b688edaee860d84 in current term 1: Already voted for candidate b0fe69a669ba4dc6b2724c4cff489927 in this term.
I20260709 12:26:59.303638 13456 raft_consensus.cc:2804] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:59.304186 13456 raft_consensus.cc:697] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 LEADER]: Becoming Leader. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Running, Role: LEADER
I20260709 12:26:59.304422 13414 consensus_queue.cc:1048] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:59.305136 13456 consensus_queue.cc:237] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.307159 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69181eb6e4eff8fc932f21978aba0" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:59.308085 13256 raft_consensus.cc:2393] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7092cb56813748718b688edaee860d84 in current term 1: Already voted for candidate b0fe69a669ba4dc6b2724c4cff489927 in this term.
I20260709 12:26:59.309401 13148 leader_election.cc:304] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7092cb56813748718b688edaee860d84; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:59.310660 13411 raft_consensus.cc:2749] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:26:59.300757 13251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dc20f40888477caf0843336f9cf5c9" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:59.283138 13252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69181eb6e4eff8fc932f21978aba0" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:59.312146 13252 raft_consensus.cc:2393] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7092cb56813748718b688edaee860d84 in current term 1: Already voted for candidate b0fe69a669ba4dc6b2724c4cff489927 in this term.
I20260709 12:26:59.312938 13449 raft_consensus.cc:515] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
W20260709 12:26:59.315438 13148 leader_election.cc:343] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:59.316234 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dc20f40888477caf0843336f9cf5c9" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:59.317119 13102 raft_consensus.cc:3060] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.319227 13253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f41b34915733403b87cd109271ee0945" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:59.320405 13253 raft_consensus.cc:2376] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1. Re-sending same reply.
I20260709 12:26:59.311973 13416 leader_election.cc:290] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:59.328637 13449 leader_election.cc:290] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:59.333710 13102 raft_consensus.cc:2468] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 1.
I20260709 12:26:59.336328 13149 leader_election.cc:304] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:59.337749 13449 raft_consensus.cc:2804] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:59.338775 13449 raft_consensus.cc:697] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Becoming Leader. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:26:59.349785 13449 consensus_queue.cc:237] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.360699 13416 raft_consensus.cc:493] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.361202 13253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dc20f40888477caf0843336f9cf5c9" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:59.362414 13435 consensus_queue.cc:1048] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260709 12:26:59.367758 13148 leader_election.cc:343] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:59.363314 13181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dc20f40888477caf0843336f9cf5c9" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:59.361632 13416 raft_consensus.cc:515] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.371968 13073 leader_election.cc:304] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0fe69a669ba4dc6b2724c4cff489927; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84
I20260709 12:26:59.373992 13416 leader_election.cc:290] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:59.373366 13414 raft_consensus.cc:2749] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:59.374661 13004 catalog_manager.cc:5697] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7092cb56813748718b688edaee860d84 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "7092cb56813748718b688edaee860d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:59.376749 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:59.377725 13102 raft_consensus.cc:2468] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:59.379911 13149 leader_election.cc:304] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:59.381203 13416 raft_consensus.cc:2804] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:59.381759 13416 raft_consensus.cc:493] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.382195 13416 raft_consensus.cc:3060] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.382580 13253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" is_pre_election: true
I20260709 12:26:59.384961 13004 catalog_manager.cc:5697] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7092cb56813748718b688edaee860d84 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "7092cb56813748718b688edaee860d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:59.397637 13411 consensus_queue.cc:1048] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:59.397847 13253 raft_consensus.cc:2468] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 0.
I20260709 12:26:59.405961 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927" is_pre_election: true
I20260709 12:26:59.406886 13105 raft_consensus.cc:2468] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 0.
I20260709 12:26:59.408942 13223 leader_election.cc:304] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:26:59.410032 13435 raft_consensus.cc:2804] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:59.410549 13435 raft_consensus.cc:493] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.410967 13435 raft_consensus.cc:3060] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.431797 13449 consensus_queue.cc:1048] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:26:59.443712 13433 leader_election.cc:290] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:59.447310 13181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84" is_pre_election: true
I20260709 12:26:59.463955 13435 raft_consensus.cc:515] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.480701 13411 consensus_queue.cc:1048] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:59.494524 13410 tablet_bootstrap.cc:492] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99: No bootstrap required, opened a new log
I20260709 12:26:59.501227 13410 ts_tablet_manager.cc:1403] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent bootstrapping tablet: real 0.524s	user 0.023s	sys 0.001s
I20260709 12:26:59.480650 13449 consensus_queue.cc:1048] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:26:59.485754 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:59.506168 13102 raft_consensus.cc:3060] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.484439 13416 raft_consensus.cc:515] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.514161 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:26:59.490132 13433 consensus_queue.cc:1048] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:26:59.521313 13435 leader_election.cc:290] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:26:59.527294 13181 raft_consensus.cc:2393] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in current term 1: Already voted for candidate 7092cb56813748718b688edaee860d84 in this term.
I20260709 12:26:59.528513 13416 leader_election.cc:290] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:26:59.530979 13410 raft_consensus.cc:359] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.532163 13410 raft_consensus.cc:385] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:59.532555 13410 raft_consensus.cc:740] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Initialized, Role: FOLLOWER
I20260709 12:26:59.537943 13410 consensus_queue.cc:260] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.531900 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:26:59.543313 13256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:26:59.544595 13256 raft_consensus.cc:2393] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7092cb56813748718b688edaee860d84 in current term 1: Already voted for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in this term.
I20260709 12:26:59.520701 13448 consensus_queue.cc:1048] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:26:59.559727 13148 leader_election.cc:304] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7092cb56813748718b688edaee860d84; no voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:59.581073 13448 raft_consensus.cc:2749] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:26:59.588912 13410 ts_tablet_manager.cc:1434] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.000s
I20260709 12:26:59.593700 13102 raft_consensus.cc:2468] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 1.
I20260709 12:26:59.596199 13223 leader_election.cc:304] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 7092cb56813748718b688edaee860d84
I20260709 12:26:59.597319 13473 raft_consensus.cc:2804] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:59.599398 13473 raft_consensus.cc:697] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Leader. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:26:59.606666 13418 consensus_queue.cc:1048] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:59.611825 13473 consensus_queue.cc:237] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:26:59.651670 13006 catalog_manager.cc:5697] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:59.656296 13005 catalog_manager.cc:5697] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65). New cstate: current_term: 1 leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:59.679250 13004 catalog_manager.cc:5697] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7092cb56813748718b688edaee860d84 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "7092cb56813748718b688edaee860d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:59.682395 13004 catalog_manager.cc:5697] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7092cb56813748718b688edaee860d84 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "7092cb56813748718b688edaee860d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:59.684163 13004 catalog_manager.cc:5697] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7092cb56813748718b688edaee860d84 (127.1.123.66). New cstate: current_term: 1 leader_uuid: "7092cb56813748718b688edaee860d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:59.706866 13005 catalog_manager.cc:5697] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67). New cstate: current_term: 1 leader_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:59.741482 13420 consensus_queue.cc:1048] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:59.757938 13420 consensus_queue.cc:1048] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:59.760924 13449 consensus_queue.cc:1048] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:26:59.777853 13251 raft_consensus.cc:3060] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:59.790098 13448 consensus_queue.cc:1048] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:59.809046 13448 consensus_queue.cc:1048] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:59.822233 13449 consensus_queue.cc:1048] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:59.924163 13251 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a96a664188314072a99803d265c020a6"
dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
mode: GRACEFUL
new_leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:42976
I20260709 12:26:59.925354 13251 raft_consensus.cc:606] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Received request to transfer leadership to TS b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:59.933387 13182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abbf041ad34d4b16adf6e446c8f8ca68"
dest_uuid: "7092cb56813748718b688edaee860d84"
mode: GRACEFUL
new_leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:41588
I20260709 12:26:59.934315 13182 raft_consensus.cc:606] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Received request to transfer leadership to TS b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:59.941980 13182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2"
dest_uuid: "7092cb56813748718b688edaee860d84"
mode: GRACEFUL
new_leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:41588
I20260709 12:26:59.942654 13182 raft_consensus.cc:606] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Received request to transfer leadership to TS b0fe69a669ba4dc6b2724c4cff489927
I20260709 12:26:59.971138 13105 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b336d26784a4500817fda148c468562"
dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
mode: GRACEFUL
new_leader_uuid: "7092cb56813748718b688edaee860d84"
 from {username='slave'} at 127.0.0.1:38868
I20260709 12:26:59.971967 13105 raft_consensus.cc:606] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 LEADER]: Received request to transfer leadership to TS 7092cb56813748718b688edaee860d84
I20260709 12:26:59.981933 13456 raft_consensus.cc:993] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927: : Instructing follower 7092cb56813748718b688edaee860d84 to start an election
I20260709 12:26:59.982546 13456 raft_consensus.cc:1081] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 LEADER]: Signalling peer 7092cb56813748718b688edaee860d84 to start an election
I20260709 12:26:59.991531 13182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b336d26784a4500817fda148c468562"
dest_uuid: "7092cb56813748718b688edaee860d84"
 from {username='slave'} at 127.0.0.1:41578
I20260709 12:26:59.992347 13182 raft_consensus.cc:493] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:59.992731 13182 raft_consensus.cc:3060] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.003978 13182 raft_consensus.cc:515] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:00.006510 13182 leader_election.cc:290] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 2 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:27:00.007706 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b336d26784a4500817fda148c468562" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:27:00.008136 13251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b336d26784a4500817fda148c468562" candidate_uuid: "7092cb56813748718b688edaee860d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:27:00.009470 13105 raft_consensus.cc:3055] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:00.009590 13251 raft_consensus.cc:3060] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.010178 13105 raft_consensus.cc:740] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Running, Role: LEADER
I20260709 12:27:00.011118 13105 consensus_queue.cc:260] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:00.012367 13105 raft_consensus.cc:3060] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.021682 13105 raft_consensus.cc:2468] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 2.
I20260709 12:27:00.021919 13251 raft_consensus.cc:2468] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092cb56813748718b688edaee860d84 in term 2.
I20260709 12:27:00.023748 13149 leader_election.cc:304] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:27:00.025159 13416 raft_consensus.cc:2804] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:00.025717 13416 raft_consensus.cc:697] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 2 LEADER]: Becoming Leader. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:27:00.026568 13416 consensus_queue.cc:237] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:00.038661 13005 catalog_manager.cc:5697] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 reported cstate change: term changed from 1 to 2, leader changed from b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65) to 7092cb56813748718b688edaee860d84 (127.1.123.66). New cstate: current_term: 2 leader_uuid: "7092cb56813748718b688edaee860d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:00.051812 13473 consensus_queue.cc:1048] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:00.068136 13472 raft_consensus.cc:993] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99: : Instructing follower b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:27:00.069124 13433 raft_consensus.cc:1081] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Signalling peer b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:27:00.071679 13106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a96a664188314072a99803d265c020a6"
dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:45710
I20260709 12:27:00.076211 13106 raft_consensus.cc:493] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:00.077004 13106 raft_consensus.cc:3060] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.077948 13472 consensus_queue.cc:1048] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:00.091598 13106 raft_consensus.cc:515] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:00.100049 13106 leader_election.cc:290] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Requested vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:27:00.102820 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:27:00.102859 13251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96a664188314072a99803d265c020a6" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:27:00.103502 13182 raft_consensus.cc:3060] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.103904 13251 raft_consensus.cc:3055] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:00.104334 13251 raft_consensus.cc:740] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:27:00.105330 13251 consensus_queue.cc:260] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:00.107061 13251 raft_consensus.cc:3060] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.116173 13251 raft_consensus.cc:2468] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:27:00.117722 13182 raft_consensus.cc:2468] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:27:00.117724 13073 leader_election.cc:304] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
W20260709 12:27:00.119683 13129 debug-util.cc:398] Leaking SignalData structure 0x7b08000829c0 after lost signal to thread 1520
I20260709 12:27:00.119052 13456 raft_consensus.cc:2804] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:00.120888 13456 raft_consensus.cc:697] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Becoming Leader. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Running, Role: LEADER
W20260709 12:27:00.122228 13129 debug-util.cc:398] Leaking SignalData structure 0x7b0800134d40 after lost signal to thread 13040
I20260709 12:27:00.121898 13456 consensus_queue.cc:237] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
W20260709 12:27:00.125394 13129 debug-util.cc:398] Leaking SignalData structure 0x7b08002ddc40 after lost signal to thread 13207
W20260709 12:27:00.127023 13129 debug-util.cc:398] Leaking SignalData structure 0x7b08001102a0 after lost signal to thread 13281
I20260709 12:27:00.142123 13449 raft_consensus.cc:993] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84: : Instructing follower b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:27:00.142720 13418 raft_consensus.cc:1081] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Signalling peer b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:27:00.141450 13006 catalog_manager.cc:5697] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 reported cstate change: term changed from 1 to 2, leader changed from 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67) to b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:00.150569 13106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2"
dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:45704
I20260709 12:27:00.157090 13106 raft_consensus.cc:493] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:00.157755 13106 raft_consensus.cc:3060] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.171001 13106 raft_consensus.cc:515] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:00.174499 13182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:27:00.175201 13182 raft_consensus.cc:3055] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:00.175550 13182 raft_consensus.cc:740] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:27:00.176689 13182 consensus_queue.cc:260] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:00.178072 13182 raft_consensus.cc:3060] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.178006 13251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfdb12d53dc484197c250705a4385f2" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:27:00.179702 13251 raft_consensus.cc:3060] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.182381 13416 raft_consensus.cc:993] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84: : Instructing follower b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:27:00.175446 13106 leader_election.cc:290] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Requested vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:27:00.183334 13416 raft_consensus.cc:1081] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Signalling peer b0fe69a669ba4dc6b2724c4cff489927 to start an election
I20260709 12:27:00.185859 13105 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abbf041ad34d4b16adf6e446c8f8ca68"
dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
 from {username='slave'} at 127.0.0.1:45704
I20260709 12:27:00.186635 13105 raft_consensus.cc:493] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:00.187045 13105 raft_consensus.cc:3060] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.198076 13105 raft_consensus.cc:515] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:00.203924 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abbf041ad34d4b16adf6e446c8f8ca68" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:27:00.204483 13251 raft_consensus.cc:2468] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:27:00.205075 13183 raft_consensus.cc:3055] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:00.205592 13183 raft_consensus.cc:740] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:27:00.206401 13073 leader_election.cc:304] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:27:00.207858 13456 raft_consensus.cc:2804] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:00.207432 13183 consensus_queue.cc:260] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:00.208681 13456 raft_consensus.cc:697] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Becoming Leader. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Running, Role: LEADER
I20260709 12:27:00.201076 13182 raft_consensus.cc:2468] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:27:00.209859 13456 consensus_queue.cc:237] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:00.210456 13183 raft_consensus.cc:3060] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.223470 13183 raft_consensus.cc:2468] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:27:00.222841 13006 catalog_manager.cc:5697] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 reported cstate change: term changed from 1 to 2, leader changed from 7092cb56813748718b688edaee860d84 (127.1.123.66) to b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:00.225757 13073 leader_election.cc:304] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092cb56813748718b688edaee860d84, b0fe69a669ba4dc6b2724c4cff489927; no voters: 
I20260709 12:27:00.226807 13456 raft_consensus.cc:2804] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:00.227341 13251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abbf041ad34d4b16adf6e446c8f8ca68" candidate_uuid: "b0fe69a669ba4dc6b2724c4cff489927" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
I20260709 12:27:00.227728 13456 raft_consensus.cc:697] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Becoming Leader. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Running, Role: LEADER
I20260709 12:27:00.228161 13251 raft_consensus.cc:3060] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:00.229171 13456 consensus_queue.cc:237] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:00.222851 13105 leader_election.cc:290] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [CANDIDATE]: Term 2 election: Requested vote from peers 7092cb56813748718b688edaee860d84 (127.1.123.66:40767), 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67:45715)
I20260709 12:27:00.239698 13251 raft_consensus.cc:2468] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fe69a669ba4dc6b2724c4cff489927 in term 2.
I20260709 12:27:00.247237 13006 catalog_manager.cc:5697] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 reported cstate change: term changed from 1 to 2, leader changed from 7092cb56813748718b688edaee860d84 (127.1.123.66) to b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65). New cstate: current_term: 2 leader_uuid: "b0fe69a669ba4dc6b2724c4cff489927" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:00.430806 13256 raft_consensus.cc:1275] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Refusing update from remote peer 7092cb56813748718b688edaee860d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:00.433104 13418 consensus_queue.cc:1048] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:00.454591 13105 raft_consensus.cc:1275] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Refusing update from remote peer 7092cb56813748718b688edaee860d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:00.456807 13449 consensus_queue.cc:1048] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:00.643549 13251 raft_consensus.cc:1275] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:00.646313 13495 consensus_queue.cc:1048] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:00.662279 13183 raft_consensus.cc:1275] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:00.669711 13456 consensus_queue.cc:1048] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:00.680017 13256 raft_consensus.cc:1275] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:00.682796 13495 consensus_queue.cc:1048] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:27:00.711154 13183 raft_consensus.cc:1275] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:00.712339 13256 raft_consensus.cc:1275] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:00.713390 13456 consensus_queue.cc:1048] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:00.719609 13456 consensus_queue.cc:1048] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:00.788563 13183 raft_consensus.cc:1275] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Refusing update from remote peer b0fe69a669ba4dc6b2724c4cff489927: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:00.791900 13500 consensus_queue.cc:1048] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:03.001202 13005 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54326:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:27:03.056191 13005 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54326:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:27:03.103847  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:03.202626 13223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.65:38141, user_credentials={real_user=slave}} blocked reactor thread for 80261.6us
W20260709 12:27:03.364320 13073 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:45715, user_credentials={real_user=slave}} blocked reactor thread for 75050.4us
W20260709 12:27:03.393471 13526 log.cc:927] Time spent T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927: Append to log took a long time: real 0.166s	user 0.000s	sys 0.005s
W20260709 12:27:03.549491 13546 log.cc:927] Time spent T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927: Append to log took a long time: real 0.122s	user 0.000s	sys 0.002s
I20260709 12:27:03.634558 13565 tablet.cc:1724] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.653862 13561 tablet.cc:1724] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:27:03.690639 13559 log.cc:927] Time spent T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:27:03.710937 13574 tablet.cc:1724] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:27:03.729665 13148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:45715, user_credentials={real_user=slave}} blocked reactor thread for 116308us
W20260709 12:27:03.743347 13560 log.cc:927] Time spent T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:27:03.752223 13183 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1"
dest_uuid: "7092cb56813748718b688edaee860d84"
mode: GRACEFUL
new_leader_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
 from {username='slave'} at 127.0.0.1:47910
I20260709 12:27:03.753350 13183 raft_consensus.cc:606] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 2 LEADER]: Received request to transfer leadership to TS 349a4090b14c4bc091b0aa6cc2b7fa99
I20260709 12:27:03.691816 13567 tablet.cc:1724] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.673103 13564 tablet.cc:1724] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.773938 13575 tablet.cc:1724] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.730535 13566 tablet.cc:1724] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.779474 13569 tablet.cc:1724] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.814296 13005 catalog_manager.cc:5789] TS b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141) does not have the latest schema for tablet a96a664188314072a99803d265c020a6 (table soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]). Expected version 1 got 0
I20260709 12:27:03.818851 13005 catalog_manager.cc:5789] TS b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141) does not have the latest schema for tablet 7dfdb12d53dc484197c250705a4385f2 (table soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]). Expected version 1 got 0
I20260709 12:27:03.813279 13578 tablet.cc:1724] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.831521 13506 raft_consensus.cc:993] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84: : Instructing follower 349a4090b14c4bc091b0aa6cc2b7fa99 to start an election
I20260709 12:27:03.834672 13005 catalog_manager.cc:5789] TS b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141) does not have the latest schema for tablet abbf041ad34d4b16adf6e446c8f8ca68 (table soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c]). Expected version 1 got 0
I20260709 12:27:03.841928 13577 tablet.cc:1724] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.841615 13576 tablet.cc:1724] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.845781 13573 raft_consensus.cc:1081] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 2 LEADER]: Signalling peer 349a4090b14c4bc091b0aa6cc2b7fa99 to start an election
I20260709 12:27:03.858618 13251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1"
dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
 from {username='slave'} at 127.0.0.1:42944
I20260709 12:27:03.859341 13251 raft_consensus.cc:493] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:03.859777 13251 raft_consensus.cc:3060] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:03.861951 13561 tablet.cc:1724] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.875667 13567 tablet.cc:1724] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.893019 13565 tablet.cc:1724] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.895434 13251 raft_consensus.cc:515] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:03.920933 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:27:03.921720 13183 raft_consensus.cc:3055] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:27:03.922119 13183 raft_consensus.cc:740] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7092cb56813748718b688edaee860d84, State: Running, Role: LEADER
I20260709 12:27:03.923008 13183 consensus_queue.cc:260] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:03.924139 13183 raft_consensus.cc:3060] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:03.920303 13251 leader_election.cc:290] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 3 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:27:03.938064 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362419b9df1c495ca8c9b744399a3ab1" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:27:03.938894 13103 raft_consensus.cc:3060] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:03.946934 13578 tablet.cc:1724] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.941134 13579 tablet.cc:1724] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.966836 13569 tablet.cc:1724] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.967607 13561 tablet.cc:1724] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.960749 13576 tablet.cc:1724] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.989145 13574 tablet.cc:1724] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:04.002553 13575 tablet.cc:1724] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:03.967871 13183 raft_consensus.cc:2468] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 3.
I20260709 12:27:03.952955 13564 tablet.cc:1724] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:04.010627 13222 leader_election.cc:304] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
I20260709 12:27:03.995823 13566 tablet.cc:1724] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:04.021423 13521 raft_consensus.cc:2804] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:27:04.022265 13521 raft_consensus.cc:697] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 3 LEADER]: Becoming Leader. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:27:04.021695 13577 tablet.cc:1724] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:04.023305 13521 consensus_queue.cc:237] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:04.031662 13105 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f"
dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
mode: GRACEFUL
new_leader_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
 from {username='slave'} at 127.0.0.1:38876
I20260709 12:27:04.033004 13105 raft_consensus.cc:606] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Received request to transfer leadership to TS 349a4090b14c4bc091b0aa6cc2b7fa99
I20260709 12:27:04.036572  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:27:04.037492  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:04.054803 13103 raft_consensus.cc:2468] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 3.
I20260709 12:27:04.082762 13576 tablet.cc:1724] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:04.056143 13565 tablet.cc:1724] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:04.068110 13579 tablet.cc:1724] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:27:04.112390 13534 raft_consensus.cc:993] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927: : Instructing follower 349a4090b14c4bc091b0aa6cc2b7fa99 to start an election
I20260709 12:27:04.113258 13534 raft_consensus.cc:1081] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Signalling peer 349a4090b14c4bc091b0aa6cc2b7fa99 to start an election
I20260709 12:27:04.113987 13006 catalog_manager.cc:5697] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 reported cstate change: term changed from 2 to 3, leader changed from 7092cb56813748718b688edaee860d84 (127.1.123.66) to 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67). New cstate: current_term: 3 leader_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:04.125236 13251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f"
dest_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99"
 from {username='slave'} at 127.0.0.1:42960
I20260709 12:27:04.125938 13251 raft_consensus.cc:493] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:04.126271 13251 raft_consensus.cc:3060] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:04.127671 13566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:45696 (request call id 20) took 1037 ms. Trace:
I20260709 12:27:04.164063 13251 raft_consensus.cc:515] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:04.167958 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b0fe69a669ba4dc6b2724c4cff489927"
I20260709 12:27:04.169006 13103 raft_consensus.cc:3055] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:27:04.169399 13103 raft_consensus.cc:740] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b0fe69a669ba4dc6b2724c4cff489927, State: Running, Role: LEADER
I20260709 12:27:04.128191 13566 rpcz_store.cc:276] 0709 12:27:03.090529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.091077 (+   548us) service_pool.cc:224] Handling call
0709 12:27:04.127510 (+1036433us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:03.188600 (+     0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0709 12:27:03.190163 (+  1563us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0709 12:27:03.190222 (+    59us) alter_schema_op.cc:66] Acquired schema lock
0709 12:27:03.190602 (+   380us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0709 12:27:03.195544 (+  4942us) alter_schema_op.cc:128] START. Timestamp: 1783600023190735
0709 12:27:03.196617 (+  1073us) op_driver.cc:348] REPLICATION: starting
0709 12:27:03.237665 (+ 41048us) log.cc:844] Serialized 204 byte log entry
0709 12:27:03.761636 (+523971us) op_driver.cc:464] REPLICATION: finished
0709 12:27:03.995692 (+234056us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0709 12:27:04.030013 (+ 34321us) log.cc:844] Serialized 5 byte log entry
0709 12:27:04.125136 (+ 95123us) tablet_metadata.cc:665] Metadata flushed
0709 12:27:04.126883 (+  1747us) log.cc:844] Serialized 14 byte log entry
0709 12:27:04.127048 (+   165us) alter_schema_op.cc:73] Released schema lock
0709 12:27:04.127094 (+    46us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":233731,"prepare.queue_time_us":96273,"prepare.run_cpu_time_us":69175,"prepare.run_wall_time_us":317195,"raft.queue_time_us":28109,"raft.run_cpu_time_us":2033,"raft.run_wall_time_us":2034,"replication_time_us":564829,"thread_start_us":312423,"threads_started":3,"wal-append.queue_time_us":114565}]]}
I20260709 12:27:04.175076 13183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a129a1631a44b8a61c38b7c03a306f" candidate_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7092cb56813748718b688edaee860d84"
I20260709 12:27:04.176307 13183 raft_consensus.cc:3060] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:04.170447 13103 consensus_queue.cc:260] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:04.179486 13103 raft_consensus.cc:3060] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:04.186051 13183 raft_consensus.cc:2468] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 3.
I20260709 12:27:04.173166 13251 leader_election.cc:290] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 3 election: Requested vote from peers b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141), 7092cb56813748718b688edaee860d84 (127.1.123.66:40767)
I20260709 12:27:04.190219 13222 leader_election.cc:304] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349a4090b14c4bc091b0aa6cc2b7fa99, 7092cb56813748718b688edaee860d84; no voters: 
I20260709 12:27:04.191445 13521 raft_consensus.cc:2804] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:27:04.191995 13521 raft_consensus.cc:697] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 3 LEADER]: Becoming Leader. State: Replica: 349a4090b14c4bc091b0aa6cc2b7fa99, State: Running, Role: LEADER
I20260709 12:27:04.193024 13521 consensus_queue.cc:237] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } }
I20260709 12:27:04.208587 13103 raft_consensus.cc:2468] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349a4090b14c4bc091b0aa6cc2b7fa99 in term 3.
I20260709 12:27:04.225370 13004 catalog_manager.cc:6242] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232: soft_deleted_table [id=de8a73bb60064c2e9bffda728e62c78c] alter complete (version 1)
I20260709 12:27:04.251494 13005 catalog_manager.cc:5697] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 reported cstate change: term changed from 2 to 3, leader changed from b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65) to 349a4090b14c4bc091b0aa6cc2b7fa99 (127.1.123.67). New cstate: current_term: 3 leader_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349a4090b14c4bc091b0aa6cc2b7fa99" member_type: VOTER last_known_addr { host: "127.1.123.67" port: 45715 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:04.470717 13179 raft_consensus.cc:1275] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 3 FOLLOWER]: Refusing update from remote peer 349a4090b14c4bc091b0aa6cc2b7fa99: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:04.473291 13521 consensus_queue.cc:1048] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:27:04.479012 13103 raft_consensus.cc:1275] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 3 FOLLOWER]: Refusing update from remote peer 349a4090b14c4bc091b0aa6cc2b7fa99: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:04.482127 13472 consensus_queue.cc:1048] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:27:04.583802 13179 raft_consensus.cc:1275] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 3 FOLLOWER]: Refusing update from remote peer 349a4090b14c4bc091b0aa6cc2b7fa99: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:04.585999 13521 consensus_queue.cc:1048] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092cb56813748718b688edaee860d84" member_type: VOTER last_known_addr { host: "127.1.123.66" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:27:04.612366 13103 raft_consensus.cc:1275] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 3 FOLLOWER]: Refusing update from remote peer 349a4090b14c4bc091b0aa6cc2b7fa99: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:04.622560 13521 consensus_queue.cc:1048] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fe69a669ba4dc6b2724c4cff489927" member_type: VOTER last_known_addr { host: "127.1.123.65" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:27:05.039489  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:05.046324  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:05.046873  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:06.048455  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:06.055291  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:06.056099  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:07.057771  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:07.066952  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:07.068228  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:08.070344  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:08.078418  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:08.079313  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:09.081336  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:09.089277  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:09.090178  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:27:09.204097 13148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.123.67:45715, user_credentials={real_user=slave}} blocked reactor thread for 58242.7us
I20260709 12:27:10.313408  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:10.319090  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:10.319514  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:27:10.424907 13129 debug-util.cc:398] Leaking SignalData structure 0x7b080021a200 after lost signal to thread 1520
W20260709 12:27:10.426195 13129 debug-util.cc:398] Leaking SignalData structure 0x7b0800296660 after lost signal to thread 13040
W20260709 12:27:10.427587 13129 debug-util.cc:398] Leaking SignalData structure 0x7b080017a240 after lost signal to thread 13132
W20260709 12:27:10.428774 13129 debug-util.cc:398] Leaking SignalData structure 0x7b080026ab40 after lost signal to thread 13207
W20260709 12:27:10.429934 13129 debug-util.cc:398] Leaking SignalData structure 0x7b080014bf60 after lost signal to thread 13281
I20260709 12:27:11.320755  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:11.328251  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:11.328744  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:12.330302  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:12.338294  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:12.339145  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:13.340548  1517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:13.346975  1517 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:13.347647  1517 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:14.469583  1517 tablet_server.cc:179] TabletServer@127.1.123.65:0 shutting down...
W20260709 12:27:14.546104 13149 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 19 similar messages]
W20260709 12:27:14.553812 13149 consensus_peers.cc:597] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 -> Peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Couldn't send request to peer b0fe69a669ba4dc6b2724c4cff489927. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:14.561851  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:27:14.564304 13149 consensus_peers.cc:597] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 -> Peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Couldn't send request to peer b0fe69a669ba4dc6b2724c4cff489927. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:14.565431  1517 tablet_replica.cc:333] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.569200  1517 raft_consensus.cc:2243] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.571311  1517 raft_consensus.cc:2272] T c3c69181eb6e4eff8fc932f21978aba0 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:14.577028 13223 consensus_peers.cc:597] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 -> Peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Couldn't send request to peer b0fe69a669ba4dc6b2724c4cff489927. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:14.581823  1517 tablet_replica.cc:333] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
W20260709 12:27:14.583061 13149 consensus_peers.cc:597] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 -> Peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Couldn't send request to peer b0fe69a669ba4dc6b2724c4cff489927. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:14.583797  1517 raft_consensus.cc:2243] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.584971  1517 raft_consensus.cc:2272] T 5b336d26784a4500817fda148c468562 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.591254  1517 tablet_replica.cc:333] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.592360  1517 raft_consensus.cc:2243] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.594096  1517 raft_consensus.cc:2272] T 7dfdb12d53dc484197c250705a4385f2 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.600935  1517 tablet_replica.cc:333] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.602972  1517 raft_consensus.cc:2243] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.604336  1517 raft_consensus.cc:2272] T f5dc20f40888477caf0843336f9cf5c9 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.611418  1517 tablet_replica.cc:333] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.612311  1517 raft_consensus.cc:2243] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.613058  1517 raft_consensus.cc:2272] T f41b34915733403b87cd109271ee0945 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.618114  1517 tablet_replica.cc:333] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.620303  1517 raft_consensus.cc:2243] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.621960  1517 raft_consensus.cc:2272] T 82ccf5e5e1b14ff98884e5649137d885 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.626133  1517 tablet_replica.cc:333] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.627182  1517 raft_consensus.cc:2243] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.628358  1517 raft_consensus.cc:2272] T f27af1b1cca9454e9e1c1feca2bb72f4 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.632308  1517 tablet_replica.cc:333] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.633944  1517 raft_consensus.cc:2243] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.634822  1517 raft_consensus.cc:2272] T 10e5c548e4b14054b2f10d23bbd108bf P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.642122  1517 tablet_replica.cc:333] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.643293  1517 raft_consensus.cc:2243] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:27:14.643847 13149 consensus_peers.cc:597] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 -> Peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Couldn't send request to peer b0fe69a669ba4dc6b2724c4cff489927. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:14.645490  1517 raft_consensus.cc:2272] T aeb326e9f81d4120a8038a64b19111a2 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.651052  1517 tablet_replica.cc:333] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.652132  1517 raft_consensus.cc:2243] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.652973  1517 raft_consensus.cc:2272] T 9676eeaaa50f4a2eba78eccb6c45d70a P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.657053  1517 tablet_replica.cc:333] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.658149  1517 raft_consensus.cc:2243] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.659129  1517 raft_consensus.cc:2272] T bd98068120554eda83a086c1b57fc3be P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.663892  1517 tablet_replica.cc:333] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.664817  1517 raft_consensus.cc:2243] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.667141  1517 raft_consensus.cc:2272] T abbf041ad34d4b16adf6e446c8f8ca68 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.672180  1517 tablet_replica.cc:333] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.674099  1517 raft_consensus.cc:2243] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.674916  1517 raft_consensus.cc:2272] T fa85a653952647ff9fc4db109a8e5211 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.678434  1517 tablet_replica.cc:333] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.679180  1517 raft_consensus.cc:2243] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.679833  1517 raft_consensus.cc:2272] T bdb2a661794848db8d2c900fee23db77 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.685463  1517 tablet_replica.cc:333] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.686575  1517 raft_consensus.cc:2243] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.687402  1517 raft_consensus.cc:2272] T 362419b9df1c495ca8c9b744399a3ab1 P b0fe69a669ba4dc6b2724c4cff489927 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.690728  1517 tablet_replica.cc:333] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.691579  1517 raft_consensus.cc:2243] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.692370  1517 raft_consensus.cc:2272] T d7a129a1631a44b8a61c38b7c03a306f P b0fe69a669ba4dc6b2724c4cff489927 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.697157  1517 tablet_replica.cc:333] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.697880  1517 raft_consensus.cc:2243] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.699246  1517 raft_consensus.cc:2272] T a96a664188314072a99803d265c020a6 P b0fe69a669ba4dc6b2724c4cff489927 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.703028  1517 tablet_replica.cc:333] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927: stopping tablet replica
I20260709 12:27:14.703778  1517 raft_consensus.cc:2243] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.704932  1517 raft_consensus.cc:2272] T 3671b696feea45ef91c742b4bed616b3 P b0fe69a669ba4dc6b2724c4cff489927 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.824036  1517 tablet_server.cc:196] TabletServer@127.1.123.65:0 shutdown complete.
W20260709 12:27:14.869797 13149 consensus_peers.cc:597] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 -> Peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Couldn't send request to peer b0fe69a669ba4dc6b2724c4cff489927. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:38141: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:14.891649 13223 consensus_peers.cc:597] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 -> Peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Couldn't send request to peer b0fe69a669ba4dc6b2724c4cff489927. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:38141: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:14.907286  1517 tablet_server.cc:179] TabletServer@127.1.123.66:0 shutting down...
W20260709 12:27:14.907555 13223 consensus_peers.cc:597] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 -> Peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Couldn't send request to peer b0fe69a669ba4dc6b2724c4cff489927. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:38141: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:14.920953 13223 consensus_peers.cc:597] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 -> Peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Couldn't send request to peer b0fe69a669ba4dc6b2724c4cff489927. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:38141: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:14.942441 13149 consensus_peers.cc:597] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 -> Peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Couldn't send request to peer b0fe69a669ba4dc6b2724c4cff489927. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:38141: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:14.965966  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:14.967082  1517 tablet_replica.cc:333] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:14.968184  1517 raft_consensus.cc:2243] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.969080  1517 raft_consensus.cc:2272] T c3c69181eb6e4eff8fc932f21978aba0 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:14.970731 13222 consensus_peers.cc:597] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 -> Peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Couldn't send request to peer 7092cb56813748718b688edaee860d84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:14.974102  1517 tablet_replica.cc:333] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:14.975648  1517 raft_consensus.cc:2243] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.977075  1517 raft_consensus.cc:2272] T 5b336d26784a4500817fda148c468562 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.980541  1517 tablet_replica.cc:333] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:14.981381  1517 raft_consensus.cc:2243] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.982448  1517 raft_consensus.cc:2272] T f5dc20f40888477caf0843336f9cf5c9 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.987054  1517 tablet_replica.cc:333] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:14.987831  1517 raft_consensus.cc:2243] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.989689  1517 raft_consensus.cc:2272] T 7dfdb12d53dc484197c250705a4385f2 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:14.991669 13149 consensus_peers.cc:597] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 -> Peer b0fe69a669ba4dc6b2724c4cff489927 (127.1.123.65:38141): Couldn't send request to peer b0fe69a669ba4dc6b2724c4cff489927. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.65:38141: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:14.993348  1517 tablet_replica.cc:333] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:14.994254  1517 raft_consensus.cc:2243] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.995834  1517 raft_consensus.cc:2272] T f41b34915733403b87cd109271ee0945 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.000705  1517 tablet_replica.cc:333] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84: stopping tablet replica
W20260709 12:27:15.001451 13222 consensus_peers.cc:597] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 -> Peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Couldn't send request to peer 7092cb56813748718b688edaee860d84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:15.002115  1517 raft_consensus.cc:2243] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.003412  1517 raft_consensus.cc:2272] T 82ccf5e5e1b14ff98884e5649137d885 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.007218  1517 tablet_replica.cc:333] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.008271  1517 raft_consensus.cc:2243] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.009693  1517 raft_consensus.cc:2272] T f27af1b1cca9454e9e1c1feca2bb72f4 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.012977  1517 tablet_replica.cc:333] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.013670  1517 raft_consensus.cc:2243] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.014250  1517 raft_consensus.cc:2272] T a96a664188314072a99803d265c020a6 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.016920  1517 tablet_replica.cc:333] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.017627  1517 raft_consensus.cc:2243] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.018179  1517 raft_consensus.cc:2272] T 3671b696feea45ef91c742b4bed616b3 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.020670  1517 tablet_replica.cc:333] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.021648  1517 raft_consensus.cc:2243] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.022730  1517 raft_consensus.cc:2272] T abbf041ad34d4b16adf6e446c8f8ca68 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.026245  1517 tablet_replica.cc:333] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.027078  1517 raft_consensus.cc:2243] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:15.028172  1517 raft_consensus.cc:2272] T bd98068120554eda83a086c1b57fc3be P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.031901  1517 tablet_replica.cc:333] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.032950  1517 raft_consensus.cc:2243] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:15.034440  1517 raft_consensus.cc:2272] T 9676eeaaa50f4a2eba78eccb6c45d70a P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.037058  1517 tablet_replica.cc:333] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.037942  1517 raft_consensus.cc:2243] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.039258  1517 raft_consensus.cc:2272] T aeb326e9f81d4120a8038a64b19111a2 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.042680  1517 tablet_replica.cc:333] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.043313  1517 raft_consensus.cc:2243] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.043905  1517 raft_consensus.cc:2272] T fa85a653952647ff9fc4db109a8e5211 P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.046710  1517 tablet_replica.cc:333] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.047374  1517 raft_consensus.cc:2243] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:15.048943  1517 raft_consensus.cc:2272] T bdb2a661794848db8d2c900fee23db77 P 7092cb56813748718b688edaee860d84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.052805  1517 tablet_replica.cc:333] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.053646  1517 raft_consensus.cc:2243] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.054417  1517 raft_consensus.cc:2272] T 362419b9df1c495ca8c9b744399a3ab1 P 7092cb56813748718b688edaee860d84 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.057372  1517 tablet_replica.cc:333] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.058001  1517 raft_consensus.cc:2243] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.058590  1517 raft_consensus.cc:2272] T d7a129a1631a44b8a61c38b7c03a306f P 7092cb56813748718b688edaee860d84 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.061070  1517 tablet_replica.cc:333] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84: stopping tablet replica
I20260709 12:27:15.061648  1517 raft_consensus.cc:2243] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:15.062775  1517 raft_consensus.cc:2272] T 10e5c548e4b14054b2f10d23bbd108bf P 7092cb56813748718b688edaee860d84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.125373  1517 tablet_server.cc:196] TabletServer@127.1.123.66:0 shutdown complete.
I20260709 12:27:15.187117  1517 tablet_server.cc:179] TabletServer@127.1.123.67:0 shutting down...
I20260709 12:27:15.247581  1517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:27:15.247707 13222 consensus_peers.cc:597] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 -> Peer 7092cb56813748718b688edaee860d84 (127.1.123.66:40767): Couldn't send request to peer 7092cb56813748718b688edaee860d84. Status: Network error: Client connection negotiation failed: client connection to 127.1.123.66:40767: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:15.248962  1517 tablet_replica.cc:333] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.250309  1517 raft_consensus.cc:2243] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.251073  1517 raft_consensus.cc:2272] T bd98068120554eda83a086c1b57fc3be P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.254192  1517 tablet_replica.cc:333] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.255297  1517 raft_consensus.cc:2243] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.256146  1517 raft_consensus.cc:2272] T abbf041ad34d4b16adf6e446c8f8ca68 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.260181  1517 tablet_replica.cc:333] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.261061  1517 raft_consensus.cc:2243] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.261848  1517 raft_consensus.cc:2272] T f5dc20f40888477caf0843336f9cf5c9 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.265944  1517 tablet_replica.cc:333] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.266964  1517 raft_consensus.cc:2243] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.267762  1517 raft_consensus.cc:2272] T 7dfdb12d53dc484197c250705a4385f2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.271835  1517 tablet_replica.cc:333] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.272953  1517 raft_consensus.cc:2243] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.273797  1517 raft_consensus.cc:2272] T c3c69181eb6e4eff8fc932f21978aba0 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.276934  1517 tablet_replica.cc:333] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.278048  1517 raft_consensus.cc:2243] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.278733  1517 raft_consensus.cc:2272] T f41b34915733403b87cd109271ee0945 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.281800  1517 tablet_replica.cc:333] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.282542  1517 raft_consensus.cc:2243] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.283169  1517 raft_consensus.cc:2272] T 82ccf5e5e1b14ff98884e5649137d885 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.288436  1517 tablet_replica.cc:333] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.289623  1517 raft_consensus.cc:2243] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:15.290827  1517 raft_consensus.cc:2272] T f27af1b1cca9454e9e1c1feca2bb72f4 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.294869  1517 tablet_replica.cc:333] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.295774  1517 raft_consensus.cc:2243] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.296456  1517 raft_consensus.cc:2272] T 10e5c548e4b14054b2f10d23bbd108bf P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.299749  1517 tablet_replica.cc:333] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.300613  1517 raft_consensus.cc:2243] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.301512  1517 raft_consensus.cc:2272] T aeb326e9f81d4120a8038a64b19111a2 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.304589  1517 tablet_replica.cc:333] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.305368  1517 raft_consensus.cc:2243] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.306113  1517 raft_consensus.cc:2272] T 9676eeaaa50f4a2eba78eccb6c45d70a P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.309235  1517 tablet_replica.cc:333] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.310091  1517 raft_consensus.cc:2243] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:15.311353  1517 raft_consensus.cc:2272] T fa85a653952647ff9fc4db109a8e5211 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.314777  1517 tablet_replica.cc:333] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.315553  1517 raft_consensus.cc:2243] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.316264  1517 raft_consensus.cc:2272] T bdb2a661794848db8d2c900fee23db77 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.319303  1517 tablet_replica.cc:333] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.320312  1517 raft_consensus.cc:2243] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:27:15.321798  1517 raft_consensus.cc:2272] T d7a129a1631a44b8a61c38b7c03a306f P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.324885  1517 tablet_replica.cc:333] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.325819  1517 raft_consensus.cc:2243] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:27:15.326959  1517 raft_consensus.cc:2272] T 362419b9df1c495ca8c9b744399a3ab1 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.330384  1517 tablet_replica.cc:333] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.331220  1517 raft_consensus.cc:2243] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.331938  1517 raft_consensus.cc:2272] T a96a664188314072a99803d265c020a6 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.335412  1517 tablet_replica.cc:333] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.336238  1517 raft_consensus.cc:2243] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.336900  1517 raft_consensus.cc:2272] T 3671b696feea45ef91c742b4bed616b3 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.340302  1517 tablet_replica.cc:333] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99: stopping tablet replica
I20260709 12:27:15.341146  1517 raft_consensus.cc:2243] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:15.341967  1517 raft_consensus.cc:2272] T 5b336d26784a4500817fda148c468562 P 349a4090b14c4bc091b0aa6cc2b7fa99 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.405068  1517 tablet_server.cc:196] TabletServer@127.1.123.67:0 shutdown complete.
I20260709 12:27:15.461997  1517 master.cc:562] Master@127.1.123.126:36381 shutting down...
I20260709 12:27:15.507733  1517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:15.508622  1517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:15.509034  1517 tablet_replica.cc:333] T 00000000000000000000000000000000 P e25b1bc1a624479fbf4aaf7ccd44f232: stopping tablet replica
I20260709 12:27:15.536063  1517 master.cc:584] Master@127.1.123.126:36381 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (23839 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (48263 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (408994 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.RestartTserver

 1 FAILED TEST
I20260709 12:27:15.619015  1517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2287 messages since previous log ~35 seconds ago
I20260709 12:27:15.619681  1517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2290 messages since previous log ~35 seconds ago
I20260709 12:27:15.619977  1517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2290 messages since previous log ~35 seconds ago
I20260709 12:27:15.620590  1517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 7 messages since previous log ~124 seconds ago
I20260709 12:27:15.620832  1517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~58 seconds ago
I20260709 12:27:15.621007  1517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~58 seconds ago
I20260709 12:27:15.621186  1517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:5227: suppressed but not reported on 3 messages since previous log ~301 seconds ago
I20260709 12:27:15.621440  1517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 15 messages since previous log ~1 seconds ago
I20260709 12:27:15.622042  1517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~22 seconds ago
